Wherever you go, companies talk about open source. Sometimes it seems it is everywhere. Everyone is doing it. Microsoft is a Platinum Member of the Linux Foundation! So, open source has won and we all should just rely on upstream to give us the custom-built products that we need and otherwise go home, right?
Well, maybe not so fast.
Open source is an amazing and extremely successful methodology when it comes to innovation, collaboration and rapid development. We see distributed, often very diverse teams create and implement new ideas, respond to feedback, share their experiences, and improve and expand projects at truly impressive speed. But please, stop thinking of open source primarily as a software development methodology; at the core, open source is all about people and relationships. Open source is as much about sociology as it is about software engineering. And if you think about open source development as being “just like” proprietary development, except without a pay-check, then you are seriously missing the point.
So, yes, the software engineering aspects of open source in combination with the social phenomenon that is open source have together lead to engineering feats that are simply incredible and that should be recognized and celebrated. But when we look at these projects it is important to keep in mind how they are created and why and by whom.
Rapidly creating and nearly instantly validating new ideas, implementing, and prototyping them are all tasks that these projects excel at. But let’s look at one simple aspect that is so easy to conveniently ignore. A simplistic way to express this would be to say “Someone has to pay the bill”. But when you look at the situation in a little more detail, it soon becomes obvious that this has a substantial impact on how open source software is created. Obviously, this will involve a bit of generalization (since no two projects are the same here), but especially for larger projects, the type that are likely to be used in an enterprise environment, I believe the general principles to be true.
Some open source contributors do participate in these projects in their spare time. Either as students, or as developers or designers (or any of the many other functions involved in larger open source projects) who are otherwise employed, but not paid to work on this specific project. But in reality, for most larger projects today, the vast majority of the developers do so as part of their day job. For a project like Linux that number is north of 90%, for other projects it may be somewhat smaller, but the principle is fundamentally the same.
Which means that while these teams may be building something that is indeed open source software, covered by an open source license, it is likely that a significant part of the contributions to a project were funded by companies with their own, separate agendas and profit motives. Which typically means that there is a “value line” somewhere. And things above that line are what that company gets paid for – and therefore don’t get done in open source.
I tend to think of this as a typical 80/20 scenario – the first 80 percent, the part that’s exciting, that’s innovation, that creates the APIs and defines the “content” of the project typically are done by the community. Doing this part is exciting for the engineers who work on this in their spare time. And it’s easily rationalized as reasonable “shared engineering” by the companies involved.
But the volunteers typically aren’t all that interested in working on the parts that are maybe a little more tedious, more repetitive, less exciting (I am generalizing here and some people will likely get mad – but by and large I think this is a fair statement). And the companies involved are less incentivized to work on those.
Taking the innovation and continuously running it through a number of integration and testing processes is becoming more typical for open source projects, but the rigor and scale at which this is done tends to be significantly smaller than what a healthy product would expect to see. That part of the work is usually left to companies that are built around the product (and I will write more about this soon – this was actually the topic of my talk at the Open Source Leadership Summit last week)
Don’t get me wrong, that is not a bad thing. This allows projects to grow into products, to create a profit motive for the developers and to create overall opportunities in this industry. But this is a phenomenon you need to clearly understand and acknowledge. Because this implies a different point of view when evaluating a project as something that you want to use in a production environment.
There are, of course, many ways to get to this point; you can use an open source distribution of an open source project and do the hard work yourself, as part of a productization process. But this does put a damper on any hopes that somehow open source implies that there is a free lunch to be had here. In general, there isn’t.
So yes, open source is an amazing methodology to innovate, to collaborate and to rapidly move projects forward. And it’s a fascinating social phenomenon. But in general, without additional work on top of it, it is not resulting in software that you’ll want to run in production.
