Home > Blogs > Open Source @VMware

VMware & ONAP | Defining a Common Foundation for Managing Network Functions

By Constantine Polychronopoulos, VP & CTO, Telco and NFV at VMware

The Open Network Automation Platform (ONAP) merges the Open Source ECOMP effort and Open-O under the auspices of the Linux Foundation and it is wonderful news and a decisive step in the right direction!  (Read more in the Press Release from the Linux Foundation.) Both Open-O and ECOMP have made significant progress in their attempt to define a common approach to network orchestration and automation, but the two efforts also diverged in their respective scope, with Open-O focusing on network functions orchestration and ECOMP taking a broader approach attempting to define an end-to-end architecture that included pretty much every function under the realm of network services automation – leveraging, of course, other open source SDN, virtualization and VIM components.

By coalescing the two into a common effort under the Linux Foundation, ONAP will have a tangible impact in the industry that will be far greater than the sum of its parts. Given the leading role and active participation in ONAP of global operators such as AT&T, China Mobile, China Telecom, Bell Canada, Orange, and the founding role of leading network technologies companies that include VMware, Ericsson, Nokia, Intel, IBM, Cisco, Metaswitch, GigaSpaces and many others, we expect to see accelerated progress toward converging on a flexible and extensible architecture and interface specifications, that will bring significant qualitative and quantitative benefits to the entire industry.

Following its tradition of strong commitment to Open Source projects as both a contributor and an adopter of open source initiatives, VMware, having been a Platinum member of Open-O, takes again the lead in this seminal effort as a Founding and Platinum member of ONAP. Bringing the same innovation engine that made compute, storage and network virtualization and cloud computing commonplace, VMware plans to work closely with AT&T, China Mobile, Intel, Ericsson and the rest of the leading operators and network equipment providers in the ONAP effort, in order to define a common, encompassing and extensible foundation for the management and orchestration of software-defined networks and network functions.

In addition to a reference architecture that, given the composition of ONAP, is certain to account for all use cases including 5G and network functions supporting IoT, ONAP APIs will help all drive toward seamless interoperability across technology vendors and service providers. The ONAP project will also boost innovation as networking companies will be more willing to invest in novel network functions that can leverage the ONAP reference architecture, simplifying both development and deployment of such capabilities. The list can be long, including smart, proactive WAN-based security solutions that rely on machine learning for the effective isolation and mitigation of threats, to innovative approaches to network slicing that will certainly be critical to IoT and enterprise solutions, to personal/private clouds etc.

Our hope and wish is that the ONAP project takes an inclusive approach to defining the reference architecture and APIs for network resource and function orchestration and service automation. That is, an approach that does not result in an architecture hard-wired to specific subcomponents, but rather a design that will enable a plug-and-play environment for conforming best-of-breed solutions regardless of whether they are proprietary or open source based. This is the way to boost developer community and industry innovation while maintaining flexibility and the ability to differentiate. To learn more about ONAP, visit the Linux Foundation.

What to Expect from an Open Source Project

By Dirk Hohndel

Wherever you go, companies talk about open source. Sometimes it seems it is everywhere. Everyone is doing it. Microsoft is a Platinum Member of the Linux Foundation! So, open source has won and we all should just rely on upstream to give us the custom-built products that we need and otherwise go home, right?

Well, maybe not so fast.

Open source is an amazing and extremely successful methodology when it comes to innovation, collaboration and rapid development. We see distributed, often very diverse teams create and implement new ideas, respond to feedback, share their experiences, and improve and expand projects at truly impressive speed. But please, stop thinking of open source primarily as a software development methodology; at the core, open source is all about people and relationships. Open source is as much about sociology as it is about software engineering. And if you think about open source development as being “just like” proprietary development, except without a pay-check, then you are seriously missing the point.

So yes, the software engineering aspects of open source in combination with the social phenomenon that is open source have together lead to engineering feats that are simply incredible and that should be recognized and celebrated. But when we look at these projects it is important to keep in mind how they are created and why and by whom.

Rapidly creating and nearly instantly validating new ideas, implementing, and prototyping them are all tasks that these projects excel at. But let’s look at one simple aspect that is so easy to conveniently ignore. A simplistic way to express this would be to say “Someone has to pay the bill”. But when you look at the situation in a little more detail, it soon becomes obvious that this has a substantial impact on how open source software is created. Obviously, this will involve a bit of generalization (since no two projects are the same here), but especially for larger projects, the type that are likely to be used in an enterprise environment, I believe the general principles to be true.

Coding-Desk

Some open source contributors do participate in these projects in their spare time. Either as students, or as developers or designers (or any of the many other functions involved in larger open source projects) who are otherwise employed, but not paid to work on this specific project. But in reality, for most larger projects today, the vast majority of the developers do so as part of their day job. For a project like Linux that number is north of 90%, for other projects it may be somewhat smaller, but the principle is fundamentally the same.

Which means that while these teams may be building something that is indeed open source software, covered by an open source license, it is likely that a significant part of the contributions to a project were funded by companies with their own, separate agendas and profit motives. Which typically means that there is a “value line” somewhere. And things above that line are what that company gets paid for – and therefore don’t get done in open source.

I tend to think of this as a typical 80/20 scenario – the first 80 percent, the part that’s exciting, that’s innovation, that creates the APIs and defines the “content” of the project typically are done by the community. Doing this part is exciting for the engineers who work on this in their spare time. And it’s easily rationalized as reasonable “shared engineering” by the companies involved.

But the volunteers typically aren’t all that interested in working on the parts that are maybe a little more tedious, more repetitive, less exciting (I am generalizing here and some people will likely get mad – but by and large I think this is a fair statement). And the companies involved are less incentivized to work on those. Taking the innovation and continuously running it through a number of integration and testing processes is becoming more typical for open source projects, but the rigor and scale at which this is done tends to be significantly smaller than what a healthy product would expect to see. That part of the work is usually left to companies that are built around the product (and I will write more about this soon – this was actually the topic of my talk at the Open Source Leadership Summit last week)

Don’t get me wrong, that is not a bad thing. This allows projects to grow into products, to create a profit motive for the developers and to create overall opportunities in this industry. But this is a phenomenon you need to clearly understand and acknowledge. Because this implies a different point of view when evaluating a project as something that you want to use in a production environment. There are, of course, many ways to get to this point; you can use an open source distribution of an open source project and do the hard work yourself, as part of a productization process. But this does put a damper on any hopes that somehow open source implies that there is a free lunch to be had here. In general, there isn’t.

So yes, open source is an amazing methodology to innovate, to collaborate, and to rapidly move projects forward. And it’s a fascinating social phenomenon. But in general, without additional work on top of it, it is not resulting in software that you’ll want to run in production.

When Likely is Likely to be Unlikely

By Steven Rostedt
The Linux kernel, like all quality operating systems, strives to be the most efficient as possible. This is because the kernel is master of the machine, and all applications can be bogged down if the kernel is taking too much time to perform a system task. That could be a latency in scheduling between tasks, handling external interfaces such as the network card, or the hard drive. Or any other service that the kernel provides for applications. Thus, the kernel is coded in a way that it can accomplish tasks as fast as possible, but still be maintainable and reliable in its design.
One optimization that is utilized in the kernel are a couple of macros called likely() and unlikely(). These macros are hints to the compiler (namely gcc) such that it can know the probable path of a conditional statement. These macros are defined as:
#define likely(x)     __builtin_expect(!!(x), 1)
#define unlikely(x)   __builtin_expect(!!(x), 0)
An example of its use is something like:
   ret = some_routine();
   if (unlikely(ret < 0))
      return -ERROR;
The function (some_routine) is very likely to not error out. But in the unlikely case that it does, the code returns an error. The unlikely() macro around the condition tells the compiler to optimize the layout such that it expects the variable ret to always be positive.
So why does all this matter? Well, it matters a lot when it comes to laying out the code such that it can optimize the cache. When a CPU is executing code, the pipeline will pull in memory by cache lines.
Ideally, we want the memory to stay in that cache. The more cache lines the CPU requires to use, the more often some other cache line is going to be flushed out just to be loaded in again. And that takes a lot of time compared to the execution of code that is already in the cache.
The official Linux kernel tracer ftrace was able to increase its performance by as much as 20% by adding in proper likely() and unlikely()s to the code. That’s because the compiler was able to keep the logic that is most often used tightly together, and this cut down on the cache lines needed to execute the recording of the ftrace data.
Now you may think that something as simple as “return -ERROR;” would not need such an annotation. But lets say it was the likely case.
Perhaps we had “return 0”. A function return on an x86 machine in C actually requires quite a bit of work. It must restore saved registers and load a return value before it can exit the function. For example the return of pick_next_task_fair() (a commonly called scheduler
function) has this:
  movq $0x0, -0x48(%rbp)
      mov -0x48(%rbp), %rax
      add $0x68, %rsp
      pop %rbx
      pop %r12
      pop %r13
      pop %r14
      pop %r15
      pop %rbp
      retq
The above takes up a total of 27 bytes. With a cache line size of 64 bytes, that return takes up 42% of the cache line.
For the likely case of a simple “if (x) return 0;”, it would be best to keep that code right after the compare of x. If it is not likely then it should be moved away from the rest of the function code, and have the positive branch case have to jump to it.
Now you can see that adding likely() and unlikely() macros around branch conditions can improve the performance of the kernel. But the question then arises, how do you really know if your likely() and unlikely() placements are correct? Doing micro-benchmarks may help, but those rarely demonstrate real world activity, and benchmarks on real world activity is difficult to pin point what branches are being taking or not. This is where I created the likely/unlikely branch profiler.
By enabling CONFIG_PROFILE_ANNOTATED_BRANCHES in the kernel build config file, all the likely() and unlikely() macros will be recorded to see how many times they were correct or not. Then while the machine is running, examining the tracefs file in:
/sys/kernel/debug/tracing/trace_stat/branch_annotated
This will show what branches are correct or not. For example, doing this on one of my test machines with a distribution config, but with profiling enabled, we have:
Correct Incorrect % Function File Line
0 766059 100 audit_syscall_exit Audit.h 2608
0 204147 100 Sched_info_switch Stats.h 156
0 162121 100 Sched_info_queued Stats.h 108
0 106475 100 Sched_info_dequeued Stats.h 73
0 43415 100 Pick_next_task_dl Deadline.c 1155
0 1515 100 Pte_alloc_one_map Memory. 2909
0 1460 100 Percpu_up_read_preempt_enable Percpu-rwsem.h 95
0 1460 100 Percpu_down_read_preempt_disab Percpu-rwsem.h 47
The above shows that there’s several cases that seem to use the likely() and unlikely() macros 100% incorrectly. Now why is this?
There are several reasons that these are incorrect. One is simply because of human error. Over the years I have enabled this tracer on two boxes of mine that I consider production machines. One of these machines runs various services like a web server, a mail server and even my compiling server. The other is the one I work on on a daily basis, which has me writing and compiling code, web browsing, and reading email. After about 3 weeks I collect all the statistics done by this profiler and start analyzing the ones that are most incorrect. One year, after submitting a patch to turn a unlikely into a likely, the author of the original code told me that they meant to put in “likely” and simply put in “unlikely” by mistake.
One thing to note, do not take the profiler as the reason to change the code. Before sending any patch to update the code, I always examine why it is wrong. There may be a config option I have enabled that will make it always incorrect that is normally not enabled. One example of this is if I enabled CONFIG_SLUB_DEBUG_ON, it will enabled the memory slub algorithm debugging at boot up. This will cause several paths to be hit that would normally not be hit, and that will cause several likely() and unlikely() branches to do the opposite of what it usually does, and this profiler will flag them a such. Never use this profiler by itself as justification to send a patch. Always dig deeper and understand the code, or the history of the code before submitting a patch.
One unlikey() path that was incorrect was simply due to the code calling that function had changed over time. There was a case in the scheduler that the task being processed was pretty much guaranteed to be sleeping, and the path for the running case was unlikely(). But over time, there was two changes made to code that called the function that made it much more likely for it to be running. The final decision was to simply remove the unlikely() and not to replace it with likely(), even though it was now a likely case. But as that branch is volatile to how it is called, it was best to not make a decision one way or the other.
One time I reported an incorrect unlikely() without a patch, because the code looked like that case was suppose to be an exception, but it was constantly being hit. The author of that code found a bug somewhere that was causing this path to be taken when it should not have been.
The unlikely stayed but the reporting of it being hit was able to uncover a bug elsewhere.
Looking at the cases above, first being “audit_syscall_exit”, this code gets called at system call exit when there’s some “work” to do. This slow path of system call exit happens when system calls are being traced, or even if a task is performing a “ptrace” on another task (like gdb). Since the auditing code is trying to be “nice”, it doesn’t assume it is the fast path, and places an “unlikely()” in the test case if it is enabled or not. The machine running this happens to have selinux enabled and that enables the auditing of system call exit.
Here’s one example where a 100% incorrect “unlikely” is the right answer! Note, if I disable auditing in the kernel by passing in “audit=0” to the kernel command line, this branch becomes 100% correct.
The sched_info_*() functions all have 100% unlikely, because the code there is something similar to the audit code. If scheduling statistics are disabled, then the sched_info_*() calls should all be ignored. But this is under discussion because when statistics are disabled, the code that has the “unlikely()” is not compiled in. Which begs the question, why is this incorrect? When statistics are compiled into the kernel, there’s a variable used to enable or disable statistics which the unlikely() macro is around. The discussion over this unlikely() is that the variable is enabled by default when statistics are enabled at build time, and can only be disabled by a kernel command line to turn it off. There is a good argument to change the unlikely, but the owner of that code needs to be convinced first.
Using the likely/unlikely profiler is a good way to understand parts of the kernel. Every time I look at an incorrect macro placement that is located in a part of the kernel I’m unfamiliar with, I learn a bit about the surrounding code. That’s one of my motivations to continue to periodically see where likely and unlikely fail to meet expectations.
It’s also a good excuse to keep my “#define if” still in the kernel. But that’s for another topic.

Product-Based Open Source Development

By Jehad Affoneh, Clarity Project Lead

ProjectClarity

At the beginning of 2016, as the team behind Clarity, VMware’s Design System, started the initial work, our main goal was for Clarity to fill a gap within VMware. Clarity was to provide a layer of common visual language that will bring designers and engineers together as well as serve as a common layer of consistency across all of VMware’s user interfaces. 
 

It was an ambitious mission that needed us to work with dozens of teams and hundreds of engineers and designers across the company to understand their use cases and needs as well as advocate for what we believed is the future of user experience for VMware’s web-based applications. 

Even though we thought on day one about the idea of open sourcing Clarity, open sourcing it wasn’t our top priority at the time. It didn’t make sense to open source a project that is yet to provide any success for our internal products. We needed to prove ourselves first before we’re ready to share our work with the world and before we’re able to advocate for others teams within other companies to adopt Clarity. 

One of the core principles that we’ve held from day one as we worked on Clarity internally was the importance of making other products that are built on top of Clarity successful. Our success wasn’t in the success of how much work we would be able to do or how many releases we’ll be able to ship. Our success is going to be tied to the success of the products which depended on Clarity to deliver modern, well-designed, and well-thoughtout design system. 

This is a risky idea. We’ve basically put our success in the hands of our consumers vs. basing it on the technological success of our framework. 

As we worked closely with dozens of teams across VMware, we realized that we’re already somewhat open source. We were just “open source” within VMware. We were already working with dozens of teams, hundreds of engineers and designers, and dozens of different (and difficult) use cases around the same problem we’re trying to solve. This realization has helped us on our path to open sourcing Clarity.  

On November 15th of last year, we decided to take the official step of open sourcing Clarity. The response has been overwhelmingly positive and we couldn’t be happier with the work we’ve been able to do since then with the community we’ve built around Clarity. 

Our core principle of building our success on top of the success products and teams building on top of Clarity are having remains true. We believe in “product-based open source development” which is fancy talk for: working with real products to build real open source projects. 

We believe in the importance of seeing Clarity actually helping engineers and designers do their work better, faster, and have better results for their teams and companies. 

We continue to believe in working with teams across the industry to build modern, well-designed, and well-engineered experiences that delight customers (both VMware and otherwise). 

One aspect that continue to be a challenge that we’re working through is the priority of requirements between “internal” VMware projects and “external” open source consumers. 

As we grow in the open source community, more requirements continue to land in our issue tracker on GitHub and via all other communication tools. We work hard to solve all the problems that we run across, but prioritization is key when running an open source project. What we’ve figured out as a solution so far is to do a few things:  

  1. Prioritize based on the vision of Clarity vs. the type of project requesting a feature or asking us to fix an issue. Being an external or internal project doesn’t play much in our decision in comparison to the issue itself and how it serves our vision. 
  2. The more an issue is requested by different teams and products, the more we look into it. It is important to listen to the engineers and designers using Clarity and the more of them working on different products that are telling us we have an issue or need to change or add something, the more the chance they’re absolutely right. 
  3. We work with our teams inside of VMware as well as contributors externally to continue to encourage contributions. We need another blog post here to talk about quality control in contributions but we love our community and want to encourage more of Clarity to come from that community. 
  4. We’re open and honest about what’s next, and about our thoughts. We’ve shared our roadmap, we post our design and technical spec before we move into implementation, and we engage the community in our decisions to make sure we’re arriving at solutions that solve real problems. 
  5. We release and plan a release every week. This has been really important in making sure we can shift priorities when needed. Solve critical bugs almost immediately, and research problems in smaller chunks of times so we’re able to provide real results quickly. 
     

We do many other things in the background as well to make sure we continue to operate efficiently and in the open but maybe that’s for another post. 
 

So far, we’ve been enjoying “product-based open source development” but we’d definitely love to hear from you. Tell us how Clarity has been working for your open source project; connect on github, follow us on medium.com, or join us on Twitter. 

Jehad Affoneh is a Staff Engineer at VMware, starting and leading Project Clarity. You can follow Jehad on Twitter @Jaffoneh.

Open Source Legal Implications, Licenses and Their Impact…

By Rupa Dachere

When you join a tech company, you usually end up signing some sort of employment agreement contract which, if you read carefully (you MUST read this thoroughly), probably states all sorts of restrictions on what you can or cannot do/own and the rights of the hiring company to own your work, whether it is during working hours or in your spare time including ideas you might have in the shower or anywhere else.  In many cases, this extends to open source contributions that you might make which might be unrelated to anything you do at work.

As we grapple with a rapidly changing software and hardware industry where the open source juggernaut slowly, but steadily, steam rolls over proprietary software, we are faced with difficult and contentious issues of what “belongs” to the company (a proprietary, closed mindset) and what “belongs” to the employee that they can contribute towards during their spare time.  Especially for contributions which are unrelated to the employee’s day job.

So, it’s imperative a developer/contributor understands the variety of open source licenses that exist, what they imply and how one should use them.

The choice of the open source license for a new project can directly or indirectly affect how it grows, how many contributors and maintainers it has, how it is used, whether people can make money using it commercially or not and, eventually, whether the project is sustainable.

Here are a few things to think about regarding licenses as you enter the world of open source:

  1. Talk to the open source office at your company and ask them for clear guidelines on what is your intellectual property and what is the company’s.  If you don’t have an open source office, locate the legal department and ask someone in that team.  If they don’t know, talk to the legal department about your concerns and see if you can get some sort of guidance from them.
  2. Read and re-read your employment agreement to understand what the implications are for your ideas and contributions to open source.  Does the company own any contribution you make regardless of whether it’s related to work or not?  Is it restricted to software, hardware, both?  What about blogs, presentations at external conferences where you are not sponsored by your company, published articles in journals or technical magazines?
  3. Read, re-read and completely understand what the different types of open source licenses are – start with this site https://opensource.org/licenses/category.  Also, look at https://creativecommons.org/licenses/.
  4. If you create open source software or hardware, ensure that you are clearly stating what license is used for contributions to that project.  If you’re unsure about what license to use, ask around, go to open source-centric meetups and talk to speakers to increase your knowledge and get help online.  Also, see https://www.gnu.org/licenses/license-recommendations.html.
  5. If you contribute to existing open source software or hardware, ensure that you understand what license is used for your contributions and if you are allowed to share it, own it and use it as you and others intend.

     

    Your choice of license (whether you started the project or are contributing to an existing one) can have a significant impact on the culture of the project.  MIT, Apache and BSD licenses are extremely open and allow contributors to share and contribute freely.  Projects that use these licenses might not, however, have stringent rules and oversight to vet the quality of the contributions since anyone can contribute – whether they have an appropriate background or not.  This could also lead to many new contributors asking for help and maintainers being overwhelmed (and, therefore, quitting).  If you go the GPL route, the various versions have varying implications and your contributor/maintainer profile and personality might be quite different. What if it’s more litigious – is this what you want for your project?  Do you want your project to be commercially distributable with acknowledgement to the origins and contributions back to the original project?

     

    I tend to use the MIT or BSD licenses for projects – CodeChix Technical Curriculums contain a series of technical curriculums spanning mobile, security, NLP and Big Data under the MIT license.  We used the Apache license for the OFconnect SDN controller project.

     

    But, that may not be the case all the time, depending on what makes sense and how I would want to structure the impact of the project and what I call the “target market” – as in what types of developers do I want to attract to the project.  I’m still thinking about this a fair bit and pondering pros/cons and what would be most beneficial for my “target market” – women engineers in industry who typically have very little “free” time and energy.

Rupa Dachere is a Sr. Member of Technical Staff working on new product development delivering virtualized SAP landscapes to the market.  Additionally, she is the Founder and Executive Director of CodeChix and DevPulseCon, which she has been voluntarily building/running in her copious free time since 2009.  CodeChix is a 501(c)3 non profit dedicated to the education, mentorship and advocacy of women engineers in industry with a primary focus on retention and a secondary focus on recruitment.

She is a tinkerer/innovator, technical speaker and intrepid homeowner as well.

Contributing Isn’t Just About the Code

By John Hawley

Over the years, I’ve grown to know a lot of different people in the open source world, they are all fighting the good fight for various reasons, in different places.  They all bring together unique skill sets, personalities, experiences and as a result we have the open source world we have.  You might be surprised though, not all of them are coders, and in fact some of them have yet to even be able to write “Hello World” in Python, and yet they are some of the most prolific contributors to open source.  Why?  Because open source isn’t just about writing the code.
Don’t get me wrong it’s kind of hard to have open source without the code, but to abuse a metaphor – it takes a village to write it.
So you now ask, how can you contribute without coding?
Submit a bug report.
It’s really that simple, and I’m sure anyone will find *SOMETHING* wrong with a piece of open source software.  This isn’t to say the submitting the bug report itself is easy, though it’s gotten a lot better in more recent years (projects should take note to try and lower this barrier to entry as much as possible).  The way you may be using the software, is quite likely, in an environment, or a use case that the developers either don’t have, or never envisioned.  Take the time to try and be as specific, and detailed, as you can be – developers loathe reports that just say “it’s broken”.  If you can screenshots, or logs are very helpful, if you can get debugging information – that’s good to include.
Just getting this information sent back to the developers is good for a numbers of reasons, so don’t believe this isn’t important, for example:
  • Lets the developers know how users are actually using their software
  • What things their own usage is missing
  • Points out issues in the code
This isn’t to say your bug report will actually change things, it’s entirely possible that your specific bug won’t get resolved, or considered a priority, but getting it reported is important all the same.  Be patient, be helpful.
Even the simplest bits of open source code needs infrastructure, and there are folks quietly lurking behind the scenes running servers, making sure that the rust keeps spinnig, keeping build machines and websites running so that projects have the pieces they need to actually be productive.  If you are ever curious, go take a look at what OSUOSL, or ISC, does as they are a great examples of infrastructure just quietly making everything work.
There’s other areas too, mentoring, outreach, community development, even straight up donations that individuals can contribute to that doesn’t involve coding.  So don’t let coding be a barrier to entry for you contributing to a project, step up and offer assistance where you feel comfortable, listen to feedback from the community, don’t be demanding or pushy, and there you go – you can contribute.  You shouldn’t let not being able to code be a deterrent or seem like you wouldn’t be valued as a contributor, because you can be a contributor without being a coder.
John Hawley is an open source software and hardware developer, because he sees fundamental value in being open, and open collaboration. He’s built multiple star ship bridges, robotic dogs, and content distribution and mirroring system. You can follow him on Twitter @warty9.

Improving Open Source Management at VMware

By Travis Finch

Open source is, by its nature, an organic, unpredictable, and opportunistic world. Which can make managing a wide reaching open source community at a large company like VMware especially challenging.  Github forms the organizational foundation for many companies – both internally and externally. Many companies with their own GitHub organization eventually face a common problem – to create consistency and ensure a positive experience for all the teams working with GitHub, manual operations are just too slow, clumsy, and error prone. At VMware, we realized that we needed to automate and streamline much of the back office operational aspects of our Open Source efforts. In this monthly series, I’ll reveal some of the challenges we faced and will walk you through our journey automating GitHub and introduce tools we built to help.

Step 1: Building a Contributor License Agreement (CLA) Portal

The first problem we tackled when it came to automating our open source management was improving the contributor license agreement (CLA) workflow. The purpose of a CLA is to provide a record that the copyright holder grants the necessary rights to contribute code to be redistributed as part of an open source project. For more details on the purpose of a CLA, see our FAQ here (Link to: https://cla.vmware.com/faq). The old process involved the developer downloading the CLA in PDF format, reviewing the agreement, printing, signing, scanning, and e-mailing it back to the VMware legal team before contributions could be accepted for the project. You can imagine the frustration met by our community contributors, along with the possibility of mistakes in this manual workflow.

We set out with the goal that making contributions to a VMware open source project should be as frictionless as possible, yet meet the requirements defined by our legal team. In 2016 we built and released the internal CLA portal, which enables a workflow for community contributors to review and digitally sign the CLA for our projects on GitHub. When a contributor opens a pull request, they will be prompted to sign the agreement if they have not already. Once signed, the VMware legal team is notified to review the agreement. If approved, the project maintainers have the necessary legal clearance to merge the change. In the rare case the agreement is rejected, the contributor has the ability to re-sign. When an agreement has already been signed and approved, an indication is made within the pull request, and future pull requests, that the contribution(s) can be accepted right away. The portal provides an administrator interface used for agreement authoring and project mapping, in addition to reviews.

tblog2

Also supported is a workflow for developer certificate of origin (DCO). When enabled on a GitHub project, all commits within a pull request will be checked for a signature that matches the commit author. The first VMware open source project to use the DCO workflow is Clarity, which is licensed under the MIT license. We are also using the widely accepted Linux Foundation DCO, which can be found here.

Since enabling the improved workflow, over 150 developers across more than 20 projects have signed our CLA or DCO. We have also released the CLA Portal as open source on GitHub (Link to: https://github.com/vmware/claportal) with the hopes that it will help others maneuver the legal hurdles necessary to properly accept contributions from the community efficiently.

Travis Finch is a Staff Engineer 2 at VMware based in Austin, Texas. For almost a decade he has developed a wide range of software for technology partner enablement. One of his current projects is focusing on automation to improve VMware’s open source operations.

Coming Back to the Code

By John Hawley

As many folks know, I’ve spent the last 3 years banging around in the open hardware world.  It’s got a lot in common, right now, with the early days of open source software: lots of interesting projects, folks trying to figure out what works and what doesn’t, and lots and lots of collaboration going on at every level of design.
It’s amazing watching where things are going in every aspect of that world, from 3D printable prosthetics, to RISC-V MCUs gaining traction.  There’s some truely revolutionary things going on there, and I’m glad I have been, and will continue to be a part of it.

So why did I decide to quit it as my day job, to come and work at VMware?

The reasoning is simple – when you are working on hardware, you are hyper focused on *THAT* hardware, particularly as complex a board as I’ve been dealing with for the past three years.  It makes it hard to see the forest for the trees, and when you’ve got a hammer, all problems tend to become nails.  So by getting to take a step back, get back to working on code that doesn’t only support one little board, and let open hardware shift back to being a hobby – I can go back to seeing the forest and get back to working on code.

I’m actually pretty stoked to join VMware – as a result, I get the opportunity to focus on upstream work, and the breadth to work on everything from Mailman, to SyncDiff, to working with Steven Rostedt on ktest.  I joined because there’s opportunity here to work with some amazing people, on a new and growing team, that has a lot of internal support, focused on working on upstream – it’s an opportunity that was genuinely hard to say no to.

John Hawley is an open source software and hardware developer, because he sees fundamental value in being open, and open collaboration. He’s built multiple star ship bridges, robotic dogs, and content distribution and mirroring system. You can follow him on Twitter @warty9.

And Then They Join You…

By Steven Rostedt

It is said that Mahatma Gandhi once said “First they ignore you. Then they laugh at you. Then they fight you. And then you win.”

Although it has been shown that Gandhi never actually said this phrase, that doesn’t take away its relevance in the world. My previous company, Red Hat, used this quote as a statement for Open Source. And Open Source software certainly followed this trend.

I was first introduced to the Free Software movement when I was working at Lockheed Martin. I was frustrated with a Microsoft product and asked rhetorically out loud, “Isn’t there a Unix for the PC?”. An intern asked me if I had ever heard of “Linux”. This was my first introduction to the Open Source operating system back in 1996. I was so impressed by the license that I started pushing it throughout my company at the time.

At first, most people just ignored it.

Later, I would read reports about Linux as a child’s operating system written by amateurs, and nobody took it seriously. People would laugh at the thought of using Linux in a production environment. But then it took hold, and we had executives calling Linux a cancer and such. There were lots of companies fighting to destroy Linux, but due to its open source nature, it could never be defeated. At least not on a technical platform.

Today, most people have accepted that Linux is the go-to operating system for servers and embedded devices alike. Basically, Linux has won. But I would like to extend that quote with “And then they join you”. As those that this quote attributes to, it was never about defeating others. It was about showing people a better way. This goes with both Gandhi as well as the Open Source methodology. That means once you win, those that once were against you will now join you.

For the last 10 years I have been employed by Red Hat. I have really enjoyed my time with them, and they have always taken open source seriously and maintained an “Upstream First” mentality, where changes should always be pushed upstream before they are incorporated into their own products.

I have left Red Hat to pursue a new adventure. I have joined a company that is known more for its proprietary software than its open source contributions. This company that I am now at, VMware, is taking open source seriously. By hiring myself and others, VMware is not just talking about open source, but wants to actively take part in the community. Actions speak much louder than words. Linux and open source has won and is here to stay. Linux is now a key part of enterprise software and companies like VMware acknowledge this, and they are making an effort to join, and become a productive member of the open source community.

Steven Rostedt has been working on the Linux kernel professionally since 2001. He’s one of the original authors of the Real Time patch and the current maintainer of the Real Time stable tree, as well as the original author and current maintainer of ftrace, the official tracer in the Linux kernel. He’s also the creator of the kernel’s “make localmodconfig”, ktest.pl, trace-cmd, kernelshark, and other little projects that help make Linux a better operating system. You can follow Steven on Twitter @srostedt.

Open Source @ VMware

By Dirk Hohndel

It’s 2017. No one needs to explain why they do open source software. Realistically, open source has become a corner stone of our industry. There are a few companies out there (not naming names, you know who you are) that no one ever expected to become major supporters of open source – or Platinum members of the Linux Foundation.

VMware have been engaged in open source for many years. We use open source components in our products (as you can tell from our open source compliance documentation). We contribute to many open source projects (from the Linux* kernel to OpenStack* to dozens of other projects). And we have launched our fair share of open source projects, some of which are highlighted at our GitHub* landing page.

The Open Source Program Office

Open Source Program OfficeIn mid-2016 we created an Open Source Program Office, which I have the honor of leading. Our goal is to move the needle on VMware’s engagement in open source even further. We’ll work a lot behind the scenes on our internal processes and tools, things that aren’t really visible to our customers and partners and friends. But we’ll also work on increasing the contributions that VMware makes to open source projects. From encouraging our teams to contribute more to the open source projects they use, to releasing more of our internal tools as open source projects, to contributing more to some key projects upstream above and beyond what we do today.

As part of this effort we are building an Open Source Technology Center in the Open Source Program Office. Right now, it’s a small team, but that means the only way is up. We’ve brought in some key developers into this team who can lead the way for VMware to have a much bigger impact in the industry. And the plan is for quite a few more to come.

The Open Source @ VMware blog

The Open Source @ VMware blog will be a place where we will talk about some of our efforts. We will give space to VMware’s key upstream contributors to talk about their projects and the things they care about. And we will strive to build a broader community around what we do. This blog will chronicle our journey and provide an inside look at our progress. In the coming weeks you’ll be introduced to a wide variety of open source topics, projects, contributors, and opinions – through the eyes and ears of our development community. Our goal is to be more open, a stronger and more collaborative participant in the thriving and vibrant open source community. Let’s see what we can do this year. And the years to come.