Open source Project Weathervane may not tell you the direction of the wind, but it is a clear indication of where the wind is blowing when it comes to open source technology. Mandy Botsko-Wilson, a consulting architect at VMware, delivered an insightful vBrownBag Tech Talk at VMworld 2017 entitled “Automating Benchmarks for Cloud Infrastructure with VMware Weathervane & vRealize Automation.”
Throughout her vBrownBag talk, Mandy addressed common questions around performance implications of moving an application to the cloud, performance trade-offs between running application services in containers vs virtual machines and hybrid approach performance by presenting Weathervane as a benchmarking tool for these needs.
Developed over five years as a project by Hal Rosenberg, a performance engineer at VMware, Project Weathervane is VMware’s open source benchmarking tool that allows you to model an enterprise application–and its workload–and then automate the repeatable execution of runs to collect performance data.
Weathervane provides application-level cloud infrastructure performance benchmarking and can help determine solutions without you having to try out a bunch of different container services in your cloud infrastructure. It’s also well poised for benchmarking hybrid apps in your cloud infrastructure and allows for the discovery management of containers.
Weathervane contains three defining components:
- Model Application. With a model application, you don’t have to use your production application and re-architect it in different ways. The first model application available as part of Weathervane’s open source code is Auction, a modern web application.
- Workload Driver. This means Weathervane can drive a realistic and repeatable load against your application. You can build in the workload driver using Auction so you don’t have to build a point-and-click system for the enterprise app you’re trying to test out.
- Run-harness. This automates the process of executing runs and collecting the results.
Once you’ve launched Weathervane, you can deploy the software in a flexible configuration based on the application. You do not have to use all of the Weathervane services, but you do need:
- A workload driver;
- A messaging server;
- An application server;
- A relational database; and
- A NoSQL database.
Underneath all of those components is the run-harness automating the runs.
Weathervane is not a typical, industry-standard benchmark. It offers flexibility in configuration without strict governing rules. As a performance benchmark, Weathervane provides:
- Model Application, which is representative of a common class of production applications.
- Workload, which only specific operations simulated users perform.
- Application-level Performance Metrics, which compares application performance across multiple runs and makes tweaks to configurations.
- Quality of Service (QoS), a measure to determine whether a given run is acceptable.
As an open source project, Weathervane will continue to grow and create more model applications that you can benchmark against in your own environments. It currently supports Docker containers and virtual machines, including vSphere Integrated Containers, the latest version of which supports native Docker Container Hosts. Support for other containerized options will be added in the future. Weathervane also offers simple setup and quick deployment, support for variable loads and application elasticity.
Taking it a step further, Weathervane can be blueprinted in VMware vRealize Automation to allow your DevOps teams to deploy, scale and configure Weathervane on varying endpoints to compare the effects on application-level performance before the enterprise application goes live for that architecture. Automating this process means you don’t have to move everything around in distribution, which can get a little tedious.
To watch a complete demo of the installation and initial run of Weathervane, Auction running a test on Weathervane, a run with Docker and blueprinting of Weathervane in vRealize Automation, check out the full recording of Mandy’s vBrownBag talk here. And remember, open source project Weathervane is available as a free download on GitHub.
Stay tuned to the VMware Open Source blog for more deep dives into vBrownBag talks from VMworld 2017, and follow us on Twitter (@VMWOpenSource).
