Open Source Firms as Conduits for Local Knowledge

by on March 21, 2007 · 28 comments

An insightful post by Don Marti on the economics of peer production:

Linux started as a peer production project in 1991, and got its first vendor-supported device driver in 1994 and its first full-time paid contributor (Leonard Zubkoff, VA Research) in 1997. Leonard started off writing SCSI drivers in his spare time, then got a job doing RAID support — so that VA could sell high-margin boxes with RAID, not just the commodity one-processor, one-drive machines they started with. Today, there’s a flexible and socially connected interface between the kernel team and the hardware companies. A driver developer has to be part of both organizations. The payoffs for plugging into both your company’s management structure and a peer production project include that you get to eliminate a lot of duplication of effort by having other people help with the infrastructure that supports your driver, and you get free code reviews and developer training. Greg says that at one company, one person maintains the Linux driver, and 150 are needed to maintain the driver for a common proprietary OS.

Ed Cashin explains the process as it works today. Firms benefit from peer production not by throwing money at the peer production project as a whole, but by dragging it in the direction the firm wants it to go. And much of what the project wants from firms is to teach it how to be useful. When firms have attempted to “be good community members” by throwing money at projects, they fail to send that signal, and much of the money ends up being wasted on trying to figure out what the project’s users want.

I hadn’t thought of that last point before. It’s a great example of the Hayekian explanation for the success of non-proprietary development models. Computer scientists like to sit in ivory towers thinking about how an ideal kernel ought to work. But in fact, developing a good kernel requires lots of local knowledge: you have to interact with real users and see what features they need. And companies can be an excellent conduit for that local knowledge, because they’re in direct, daily contact with real users, but they also have engineers with the sophistication to explain to other kernel developers exactly what’s wrong. Hence, it make sense that companies will often have the most to contribute to an open source project if they’re fixing the problems that will, by fortuitous coincidence, also do the most to make their own customers happy.

SHARE: 28 comments

About Tim Lee

Timothy B. Lee (Contributor, 2004-2009) is an adjunct scholar at the Cato Institute. He is currently a PhD student and a member of the Center for Information Technology Policy at Princeton University. He contributes regularly to a variety of online publications, including Ars Technica, Techdirt, Cato @ Liberty, and The Angry Blog. He has been a Mac bigot since 1984, a Unix, vi, and Perl bigot since 1998, and a sworn enemy of HTML-formatted email for as long as certain companies have thought that was a good idea. You can reach him by email at leex1008@umn.edu.

Read more articles by Tim Lee.

  • http://www.codemonkeyramblings.com MikeT

    I disagree that you need a lot of local knowledge to write a kernel. In fact, focusing on local knowledge would be to your detriment because you might focus too much on writing a kernel suited to only a narrow set of needs and hardware, unless that was specifically what your customers wanted. A good kernel must be written to be able to handle the needs that arise from “local knowledge,” but it needs to be abstracted away wherever possible so that it is not bogged down by those needs.

  • http://www.techliberation.com/ Tim Lee

    Mike: I agree with you. However, I think feedback from people who write stuff that interfaces with the kernel is essential to ensure that the functionality the kernel does provide is well-designed to allow others to develop specialized functionality on top of it.

  • http://www.codemonkeyramblings.com MikeT

    I disagree that you need a lot of local knowledge to write a kernel. In fact, focusing on local knowledge would be to your detriment because you might focus too much on writing a kernel suited to only a narrow set of needs and hardware, unless that was specifically what your customers wanted. A good kernel must be written to be able to handle the needs that arise from “local knowledge,” but it needs to be abstracted away wherever possible so that it is not bogged down by those needs.

  • http://www.techliberation.com/ Tim Lee

    Mike: I agree with you. However, I think feedback from people who write stuff that interfaces with the kernel is essential to ensure that the functionality the kernel does provide is well-designed to allow others to develop specialized functionality on top of it.

  • http://www.codemonkeyramblings.com MikeT

    I’m not sure I entirely agree with that. I understand the need to have feedback on the efficacy of a driver model, but that’s about it. Even there, there is a large body of knowledge for kernel writers to draw from when it comes to adding support for new types of hardware. I just don’t see why they would need a lot of feedback to make a driver model that works, except to debug it, since so much can be known in advance by that “ivory tower team.”

    I’m not a fan of Microsoft’s per se, but I have heard far more praise for their device driver method which is “ivory toweresque” than for the one that Linux uses. BeOS really puts Linux to shame, and what’s worse, the BeOS development team was about the size of the Linux kernel team!

  • http://www.techliberation.com/ Tim Lee

    So you think this sentence is just flat-out mistaken? “Greg says that at one company, one person maintains the Linux driver, and 150 are needed to maintain the driver for a common proprietary OS.”

    It sounds plausible to me, but I’ve never done driver development myself, so I could very well be wrong.

  • http://www.codemonkeyramblings.com MikeT

    I’m not sure I entirely agree with that. I understand the need to have feedback on the efficacy of a driver model, but that’s about it. Even there, there is a large body of knowledge for kernel writers to draw from when it comes to adding support for new types of hardware. I just don’t see why they would need a lot of feedback to make a driver model that works, except to debug it, since so much can be known in advance by that “ivory tower team.”


    I’m not a fan of Microsoft’s per se, but I have heard far more praise for their device driver method which is “ivory toweresque” than for the one that Linux uses. BeOS really puts Linux to shame, and what’s worse, the BeOS development team was about the size of the Linux kernel team!

  • http://linuxworld.com/community/ Don Marti

    Greg’s comparison is largely based on Microsoft Windows driver developers having to do their own build, release, and test work, which on Linux tends to be left to the distributions.

  • http://www.techliberation.com/ Tim Lee

    So you think this sentence is just flat-out mistaken? “Greg says that at one company, one person maintains the Linux driver, and 150 are needed to maintain the driver for a common proprietary OS.”

    It sounds plausible to me, but I’ve never done driver development myself, so I could very well be wrong.

  • http://linuxworld.com/community/ Don Marti

    Greg’s comparison is largely based on Microsoft Windows driver developers having to do their own build, release, and test work, which on Linux tends to be left to the distributions.

  • http://www.codemonkeyramblings.com MikeT

    That sentence tells me nothing, except that the engineer working on the Linux driver is probably better than the ones working for the Windows side of things. It also doesn’t tell me what those 150 people are doing, how good they are, etc. Suppose that side of things is run by morons who have morons writing the drivers. Is that a fair comparison?

    I am not making a value judgment of which model is better. There are proprietary platforms that kick Linux squarely in the pants on this front, such as BeOS/ZetaOS. At the same time, it is my understanding that SCO’s flavor of UNIX is trashy when it comes to device drivers. The quality here is a function of the people involved, not the development model used.

    By the way, you might want to check out Syllable or HaikuOS. They are better arguments for desktop open source OSs than Linux at this point. I’m partial to HaikuOS because it is like BeOS, and like BeOS it has a very capable desktop where there is no fiddling with anything to get things to just work.

  • http://www.codemonkeyramblings.com MikeT

    That sentence tells me nothing, except that the engineer working on the Linux driver is probably better than the ones working for the Windows side of things. It also doesn’t tell me what those 150 people are doing, how good they are, etc. Suppose that side of things is run by morons who have morons writing the drivers. Is that a fair comparison?


    I am not making a value judgment of which model is better. There are proprietary platforms that kick Linux squarely in the pants on this front, such as BeOS/ZetaOS. At the same time, it is my understanding that SCO’s flavor of UNIX is trashy when it comes to device drivers. The quality here is a function of the people involved, not the development model used.


    By the way, you might want to check out Syllable or HaikuOS. They are better arguments for desktop open source OSs than Linux at this point. I’m partial to HaikuOS because it is like BeOS, and like BeOS it has a very capable desktop where there is no fiddling with anything to get things to just work.

  • http://bennett.com/blog Richard Bennett

    Don Marti is a flaming liar.

    When I worked for 3Com during their heyday as the world’s leading Ethernet NIC supplier, there was exactly one programmer assigned to each driver: one for NDIS 2 (old Windows driver model), one for NDIS 5 (new Windows driver model), one for Linux, and one for ODI (Novell driver). Most of the driver work was maintenance, as new hardware was released every 4 to 6 months and tweaks had to made to each driver to support the new features.

    When a totally new NIC was introduced, such as the Gigabit Ethernet NIC, in some instances a second person was added to the “team” working on the relevant driver. If you added up all the developers, testers, tool developers, marketing people, technicians, admins, and managers in the Ethernet NIC department you’d still be shy of 30. I’ve done similar work at Intel and a couple of wireless LAN companies and the numbers are similar.

    It’s just not that hard to write a driver, and there aren’t that many ways to do it. Any manager who assigns more than 3 people to design, write, and test a single network driver (one card and one OS) is a fool.

  • http://bennett.com/blog Richard Bennett

    Incidentally, 3Com supported two Linux drivers, an open source driver that didn’t exploit all their hardware’s cool features, and a closed source they shipped as a binary only that was more optimized. They didn’t open source the cool feature support because it would have made it too easy for Intel to copy their design. And that’s the common practice for all the companies that build leading-edge hardware, not just in networking but also in graphics (ATI and nVidia, for example.)

    The FOSS people whine about it, but they don’t make their living from R & D. Which brings us to another benefit of proprietary systems over Linux: Windows supports a larger number of devices in fully-optimized mode than Linux ever has. Before you build a Linux system for your own use, you have to do some thorough checking in order to use only the hardware that the version of Linux you want to use can actually support, and that typically means avoiding any interface cards that haven’t been on the market for at least 12-18 months.

  • http://bennett.com/blog Richard Bennett

    Don Marti is a flaming liar.

    When I worked for 3Com during their heyday as the world’s leading Ethernet NIC supplier, there was exactly one programmer assigned to each driver: one for NDIS 2 (old Windows driver model), one for NDIS 5 (new Windows driver model), one for Linux, and one for ODI (Novell driver). Most of the driver work was maintenance, as new hardware was released every 4 to 6 months and tweaks had to made to each driver to support the new features.

    When a totally new NIC was introduced, such as the Gigabit Ethernet NIC, in some instances a second person was added to the “team” working on the relevant driver. If you added up all the developers, testers, tool developers, marketing people, technicians, admins, and managers in the Ethernet NIC department you’d still be shy of 30. I’ve done similar work at Intel and a couple of wireless LAN companies and the numbers are similar.

    It’s just not that hard to write a driver, and there aren’t that many ways to do it. Any manager who assigns more than 3 people to design, write, and test a single network driver (one card and one OS) is a fool.

  • http://bennett.com/blog Richard Bennett

    Incidentally, 3Com supported two Linux drivers, an open source driver that didn’t exploit all their hardware’s cool features, and a closed source they shipped as a binary only that was more optimized. They didn’t open source the cool feature support because it would have made it too easy for Intel to copy their design. And that’s the common practice for all the companies that build leading-edge hardware, not just in networking but also in graphics (ATI and nVidia, for example.)

    The FOSS people whine about it, but they don’t make their living from R & D. Which brings us to another benefit of proprietary systems over Linux: Windows supports a larger number of devices in fully-optimized mode than Linux ever has. Before you build a Linux system for your own use, you have to do some thorough checking in order to use only the hardware that the version of Linux you want to use can actually support, and that typically means avoiding any interface cards that haven’t been on the market for at least 12-18 months.

  • http://linuxworld.com/community/ Don Marti

    Things have changed since the glory days of 3com.

    In most hardware market segments (3d graphics is still an exception, because ATI and Nvidia are infringing too many of each other’s patents to publish anything) the balance has shifted from value in concealing features to value in making hardware easier to integrate.

  • http://bennett.com/blog Richard Bennett

    I notice you dodged the main point and went for the sympathy vote by invoking patents. That’s not going to work.

    You claimed that it takes 150 people to maintain a Windows driver. That’s an egregious lie, and you need to admit it.

    Only then can you beg for the court’s mercy.

  • http://linuxworld.com/community/ Don Marti

    Things have changed since the glory days of 3com.

    In most hardware market segments (3d graphics is still an exception, because ATI and Nvidia are infringing too many of each other’s patents to publish anything) the balance has shifted from value in concealing features to value in making hardware easier to integrate.

  • http://bennett.com/blog Richard Bennett

    I notice you dodged the main point and went for the sympathy vote by invoking patents. That’s not going to work.

    You claimed that it takes 150 people to maintain a Windows driver. That’s an egregious lie, and you need to admit it.

    Only then can you beg for the court’s mercy.

  • http://linuxworld.com/community/ Don Marti

    Went back and checked the source on this one. The 150:1 is for one manufacturer’s products, not one indivividual driver. Updated the blog entry.

  • http://linuxworld.com/community/ Don Marti

    Went back and checked the source on this one. The 150:1 is for one manufacturer’s products, not one indivividual driver. Updated the blog entry.

  • http://bennett.com/blog Richard Bennett

    That’s a step in the right direction, but 150 still seems awfully high simply for drivers. I know nVidia , ATI, and Intel don’t have that many people doing driver work for Windows, but your friend might be including testers, support staff, the cleaning crew, security guards, and the cafeteria staff in his calculations.

    But the interesting number would be the revenue comparison between Windows and Linux for a hardware manufacturer. It wouldn’t surprise me if 99% of the sales of a given hardware interface would come from Windows customers, hence we would expect to see investment spread accordingly.

    The bottom line on free open source is that you get what you pay for in software. When the quality of the code is sufficient for a task, people use it, otherwise they don’t. Apache is perfectly fine for many small-to-medium size web sites, hence it’s a standard. The Linux desktop is a whole other kettle of fish, of course, hence very few people use it.

    And we have the same sort of relationship in Wikipedia: there are plenty of curiosities that don’t warrant full-on professional fact checking, and Wikipedia serves them. If I want to know the name of the runner-up to Carrie Underwood in Season 4 of American Idol it doesn’t matter if I get the wrong answer, as long as I get an answer of some sort quickly. But for mission-critical information, a higher level of quality is needed.

  • http://bennett.com/blog Richard Bennett

    That’s a step in the right direction, but 150 still seems awfully high simply for drivers. I know nVidia , ATI, and Intel don’t have that many people doing driver work for Windows, but your friend might be including testers, support staff, the cleaning crew, security guards, and the cafeteria staff in his calculations.

    But the interesting number would be the revenue comparison between Windows and Linux for a hardware manufacturer. It wouldn’t surprise me if 99% of the sales of a given hardware interface would come from Windows customers, hence we would expect to see investment spread accordingly.

    The bottom line on free open source is that you get what you pay for in software. When the quality of the code is sufficient for a task, people use it, otherwise they don’t. Apache is perfectly fine for many small-to-medium size web sites, hence it’s a standard. The Linux desktop is a whole other kettle of fish, of course, hence very few people use it.

    And we have the same sort of relationship in Wikipedia: there are plenty of curiosities that don’t warrant full-on professional fact checking, and Wikipedia serves them. If I want to know the name of the runner-up to Carrie Underwood in Season 4 of American Idol it doesn’t matter if I get the wrong answer, as long as I get an answer of some sort quickly. But for mission-critical information, a higher level of quality is needed.

  • http://linuxworld.com/community/ Don Marti

    Richard, it’s possible to get other people to pay for a driver that’s of higher quality than you would be willing to pay for.

    On Linux, when someone gets an in-tree driver up to the quality level needed for an embedded device such as a TiVo or router, PC-architecture Linux customers get that embedded-quality driver, even if it would not be justifiable to build that good of a driver for the PC-architecture server market.

    As far as I can tell by looking at MSDN, if you want to do a driver for WinCE on a non-x86 architecture, that’s a separate project from the regular PC driver. Or am I missing something?

  • http://linuxworld.com/community/ Don Marti

    Richard, it’s possible to get other people to pay for a driver that’s of higher quality than you would be willing to pay for.

    On Linux, when someone gets an in-tree driver up to the quality level needed for an embedded device such as a TiVo or router, PC-architecture Linux customers get that embedded-quality driver, even if it would not be justifiable to build that good of a driver for the PC-architecture server market.

    As far as I can tell by looking at MSDN, if you want to do a driver for WinCE on a non-x86 architecture, that’s a separate project from the regular PC driver. Or am I missing something?

  • http://qykxjxmxpb.com qykxjxmxpb

    Hello! Good Site! Thanks you! ytxenjsgbaf

  • http://qykxjxmxpb.com qykxjxmxpb

    Hello! Good Site! Thanks you! ytxenjsgbaf

Previous post:

Next post: