by Vladislav Valtchev, VMware Open Source Technology Center
Until two months ago, my whole professional experience was as a proprietary software engineer. Here at VMware, I spent the last 4.5 years working in vSphere as a C++ platform engineer, but I used open source software my whole life. Slackware Linux was my very first operating system back in 1998, but I never tried to actually contribute to any open source project.
Recently, I moved to VMware’s Open Source Technology Center (OSTC) and discovered a whole different software development model. As a proprietary software engineer, I was used to preparing a change and posting a review request for it using a centralized web-based system for code reviews. After addressing reviewers’ comments and updating the change, I usually got a “ship it” from two or more people. That meant I could submit the change to our source control system. A change was allowed to be considerably large, sometimes upwards of 600 lines of code.
The proprietary software model differs substantially from the one used in the open source world.
Storytelling with Code Changes
First, contributors usually don’t submit; the project’s owner merges their patches in a pull versus push model. Secondly, people do not just make changes; people prepare a series of patches, where each patch is a small step towards a final goal. Each patch is expected to have a proper subject and a description, while the whole series should have a cover letter. A patch does only one (small) thing: its scope is expected to be so small that a short 50-72 characters subject must be enough for a maintainer to figure out roughly what the patch does.
I like the idea of using a series of patches. I see it as a way to tell the story behind a given change. The story matters as much as the final shape of the code in open source. During the code review, a contributor addresses the maintainer’s comments by continuously re-writing the whole story until every step looks good and in the right place. That makes sense to me. It is easier for a maintainer to review the code, convince himself or herself that the code is correct and fix any eventual bugs—even in a distant figure.
[Related blog: What Is Software Made Of? An Open Source Maintainer’s Perspective]
In proprietary software development, the story behind each change is not published. In the open source world, that would be like a contributor squashing all of his/her patches and leaving the cover letter as a description of the change before asking for a review.
After observing those differences, at some point a theory explaining the reasons for using different models started to grow in my mind:
To support patches from many contributors, the open source software model needs the patches to be as simple and descriptive as possible, even when that means asking contributors to spend more time.
3 Reasons for the “Short-Patch” Model
Fixing each patch in a series takes more time than just fixing a single cumulative patch. However, this is necessary to allow:
- Higher quality reviews: Short patches are simpler to review.
- Simpler long-term maintenance: Short and descriptive patches allow easier root-causing of a bug once a “bad” commit has been found. It is much simpler for anybody to find a bug in a 10-lines patch versus the same bug in a 500-line patch. A huge patch might have a subtle bug.
- Faster reviews: In popular open source software projects, the contributors are many, while the maintainers are few or even a single person in many cases. The short-patch model allows maintainers to be more efficient, avoiding to become a bottleneck.
In open source, the time versus quality/maintainability trade-off is often possible thanks to the lack of strict deadlines driven by time-to-market goals that developers of commercial software need to pursue. Also, at some point, regular contributors of a project get so used to its coding style and best practices that the extra time spent in using series of patches instead of larger patches gets very low.
On the other side, proprietary software is mostly written by dedicated full-time employees, who are formally committed to support a given project and their own code. Proprietary products and/or components usually do not have a single owner, but a whole team. In many cases, each member may review a change and give a “ship it.” This allows the review process to move fast even with larger patches. Occasionally, that comes at the price of lowering the quality bar a little bit.
Big software systems are pretty complex, and the reviewers might not catch all the subtle issues of a change, particularly if changes are big. This problem is addressed in proprietary software by investing more on a variety of expensive tests, such as unit tests, integration tests, system tests, stress tests, chaos tests, etc.
Testing in Proprietary versus Open Source Models
Many open source projects have good tests, but the average open source project owner cannot afford to spend so much effort in testing as big tech companies do. This is compensated by better reviews (again, the time-versus-quality trade off) and some community testing. That makes perfect sense:
Because of the different development models, the quality of the final product could be kept at the same level.
Many great open source projects have shown that. And over the last few years, commercial software companies have also adopted community testing, as well. Microsoft’s Windows Insider program is an example.
In conclusion, I believe that both the models work pretty well in their own contexts, but it’s exciting to see open source testing and change management entering the proprietary realm. It was so interesting for me as developer to stop for a moment and try to think about the rationale behind each model.
Has your company adopted open source methodologies in their proprietary software development models? Share your stories and thoughts in the comments below.
Learn about VMware’s open source projects on GitHub.
Stay tuned to the Open Source Blog for more reviews of VMware open source projects, and follow us on Twitter @vmwopensource.
Vladislav Valtchev is a C/C++/x86 assembly developer with experience in various multi-platform areas, from OS-abstraction C++ libraries to compiler plugins, dynamic instrumentation & profiling, system programming and recently kernel development at non-professional level thanks to his own educational open source kernel. In the past, he also had experiences with web development and UI applications, that made him to be actually a full-stack developer. After taking a BS in Computer Science Engineering at the Sapienza University of Rome, he joined VMware in 2013 as a C++ engineer.
