By Adam Schepis
In the “new world of DevOps”, how do teams test their applications and ensure quality in production? What types of testing occur on applications and systems? From configuration management to business functionality, how do organizations identify, prioritize and modernize our expectations of quality? How does automation impact the responsibilities and expectations of quality engineers and other roles?
These questions were the focus of a DevOps meetup last month here at CHT’s offices, and they made for a thoughtful discussion on best practices, tooling, and the current evolution of testing. Four panelists, under the skilled moderation of meetup organizer Laura Stone, tackled these issues:
- Gleb Bahmutov, VP of Engineering at Cypress.io
- Adam Blackwell, Engineer at Quantopian
- Paul Bruce, Performance Engineer at Neotys, DevOps Advisor and Founder at Growgistics
- Adam Schepis, Senior Team Lead of Engineering at CloudHealth
Some of Laura’s prompts, to get the ball rolling:
- When is the right time to write a test and/or perform testing?
- What types of testing are more important, and when?
- Where does the data you use to test with come from? Assuming you have that data, how do you know it’s good data?
- What is language that’s specific about testing? What is not?
- What is the difference between quality and testing?
The underlying theme was: how do we integrate the QA role with modern software development? Turns out, it is actually quite hard.
From a process perspective, it’s hard because QA has historically been manual, with a heavy amount of human verification. This doesn’t align with many newer, continuous trends, where processes are represented as code and run by computers in response to some trigger, such as a developer finishing work on a new feature. If quality is not a continuous process, then either a team slows down as they attempt to ensure product quality, or they can eschew quality to deliver new features more quickly. If quality is a continuous process, then the overhead of each new change becomes less time-consuming and less risky than it could be otherwise.
From a people perspective, it’s hard because the QA role is changing in response to being focused on code rather than repeatable steps. It now resembles that of a software developer than a manual tester. In many cases, these roles are combined, and developer/QA engineer are rolled together. As such, QA engineers are learning to code, and developers are learning how to write tests.
From a technology perspective, it’s hard because QA is not the only role affected by the move to continuous processes: the way we build software has changed. The move to continuous processes has also influenced design, so that testable units are now much smaller, yet greater in number, making the need for testing frameworks and testing patterns all the more evident. The viability of an organization’s testing framework thus directly affects not only how quickly new tests can be added, but also how high-fidelity the testing process is.
So…why does CloudHealth Technologies care about this issue? One of our mantras is Iterate in the presence of customers. That means releasing new features early, often, and constantly getting feedback.
As we continue to scale the number of people in the engineering organization (and we are hiring!) and add new technologies to our product, we need to build out a QA process that lets us deliver features quickly while minimizing risk, hire quality personnel who can fill both the role of developer and QA engineer, and take advantage of new technologies tailored to these specific problems.
These are important discussions, and we must continue to have them. For more on the next DevOps meetup, join the group. Hope to see you at the next event!
