We at VMware Tanzu recently published our first-ever summary of the current state of observability. The main goal of our research was to uncover the key trends in observability adoption by hearing directly from IT practitioners, including DevOps teams, SREs, application architects, and their managers. We also wanted to understand what’s driving the popularity of observability and what the organizational impact of deploying observability is. In this post, I will summarize some of our key findings.
Modern cloud application environments are complex
Managing and running modern cloud applications is an increasingly complex undertaking. Of the engineers we surveyed, 86 percent believe that modern cloud applications are significantly more complex than just five years ago. For example, 84 percent report using hundreds or even thousands of compute instances across a single organization, with a large number of containers and microservices in use.
Eight-three percent of IT practitioners report having thousands of containers at peak usage, while 77 percent of them use hundreds or thousands of microservices. And when it comes to larger organizations with more developers, there is an even higher rate of container utilization. This popularity of containers and microservices is prompting organizations to rethink monitoring. Indeed, some of our customers run hundreds of thousands of containers in production, which introduces potential visibility gaps and even blackouts if the traditional monitoring tools cannot keep up with the rate of container changes or updates.
The popularity of cloud native services and APIs is also skyrocketing; our survey respondents state that cloud application requests frequently touch dozens of different technologies. In environments where there are a large number of containers and microservices, more than 50 percent of survey participants say that an application request touches more than 25 different technologies. Having an accurate visual representation of the application map and communication between microservices in addition to topology discovery is essential.
We also found that many organizations push code to production frequently. More than 50 percent do it at least weekly, and some 20 percent push code multiple times a day. Pushing code to production so frequently requires having real-time visibility for developers and Ops/SRE teams so they can assess the immediate impact of those pushes and whether they need to roll anything back.
Traditional monitoring approaches are not enough when used with modern apps
For 90 percent of survey participants, highly distributed apps create monitoring challenges an order of magnitude greater than what traditional monitoring tools can handle. The graph below summarizes the top challenges.
Despite all of these challenges, there is still a lot of traditional tooling in use, such as check polling and logging. Interestingly, open source monitoring tools are more popular than traditional application monitoring (APM) tools, according to our survey.
Observability enables effective business decision-making
When it comes to observability tools, there are many capabilities that survey respondents find valuable. For 92 percent of them, such tools enable more effective business decision-making. Indeed, given the large number of metrics collected about the behavior of highly distributed application environments, real-time business insights start to emerge with the use of observability tools. And the more observability is in deployment across an organization, the more value gets created, for all stakeholders.
For instance, a ride-sharing service can understand its business better by observing its cloud service metrics, which make clear when its service is used the most, how many customers wait for rides, and how many drivers are on the streets during peak hours. In addition to observability being important to its engineering teams, the ride-sharing company provides its executives, including its GMs and VPs, observability tools to better understand how the business is performing based on both real-time and historical insights.
Find out if observability is the right approach for you
Based on our survey findings, there are some steps your organization can take to decide if observability is the right approach:
-
Evaluate the deployment of microservices, hybrid or multi‑cloud infrastructures, and Kubernetes for your cloud applications, including the complexity of managing them and the length of incidents
-
Plan for your cloud application growth from Day 1 by evaluating whether your current tools will reliably support your SLOs once you’re dealing with thousands—or more—of containers
For more detailed findings and additional recommendations around whether your organization should think about using observability, check out the State of Observability 2021 report.
