I recently had a chat with Nick Warr, Head of IT at Bossa Studios. We talked about how Bossa’s DevOps team of SREs and developers use observability and analytics to help build and optimize performance of their highly popular games. Here I recap how Wavefront helps Bossa improve their customers’ gaming experience with visibility into all system components powering their awesome games.
Introduction
Bossa Studios Ltd. is a gaming development company founded in 2010 in London, England. Some of their popular game titles are: Monstermind (BAFTA prize winner), Surgeon Simulator (BAFTA prize nomination), I am Bread, and Worlds Adrift (the latest). The company has brought much novelty into the world of games, gathering millions of gamers worldwide. Bossa strives to create games beyond genres and reach out to new audiences, which often brings new technical challenges.
Bossa’s DevOps team includes developers making games, and SREs maintaining them online, but they work together as a unit. They’re responsible for ensuring that complex back-end services enable the functioning of the game. This includes dealing with an inventory of the system, logins, the connections, the voice communications, and so on. They keep all of these services up and running, and monitor the overall availability of their games. Developers adopted a microservices approach to their software with many moving parts. Bossa has a multicloud hybrid environment: partially running workloads on-premises, partly in the public cloud (AWS). As it relates to serverless and containers, they use AWS Fargate and Kubernetes.
Wavefront is a one-stop observability shop for Bossa: you click it, do what it says and in an instant, there is a working dashboard. When it comes to microservices, Bossa developers use mostly Java, but also C# and Scala. They are running CI/CD pipelines pushing non-production code hourly with daily production code pushes.
To monitor their hybrid environment, Bossa’s DevOps teams initially used another commercial monitoring tool but they soon became dissatisfied. This was primarily due to excessive alert false positives and poor technical support from the vendor, but frustration increased with cost issues and a lack of visibility into usage pricing. They became very unhappy with that vendor’s opaque billing for their dynamic container environment.
The Wavefront Solution
Bossa engineers wanted to spend more time on fixing and improving things, instead of just trying to figure out what and where something went wrong. This is why they decided to adopt Wavefront. From the start, Wavefront’s flexibility and support made integration with their systems effortless. While it was clear Wavefront was superior for all their monitoring needs, they still needed to migrate fully to Wavefront. But Wavefront made it easy to track all the data with running job apps, especially as Wavefront is fully compatible with the OpenTracing.
Bossa developers appreciated that Wavefront offers lots of SDKs with pieces available to tie in quickly all their moving parts. This helps their small teams to be more efficient and focus on solving problems. Wavefront also provides lots of help for developers with little previous monitoring knowledge – they can quickly get into Wavefront and get observability working.
Although still in the early days, Bossa’s engineers speak highly of the office calm that’s come since Wavefront was introduced, both from its ease of implementation as well as its ease of use. Moreover, they praise the excellent support that the Wavefront customer success team has provided. It’s been a positive experience for Bossa. And with minimum effort, they are creating real-time dashboards.
Wavefront’s Distributed Tracing allows for even deeper, well-informed iterations when troubleshooting, and for faster understanding where bottlenecks or problems lie. Before Wavefront, Bossa had to plow thru application logs, which was tedious and complicated. Now they are tagging metrics and traces automatically. When they observe these after an incident, they can also see correlations, and what went wrong precisely. Wavefront is able to work alongside, or above/in-between their current monitoring solutions, which facilitates a much more relaxed deployment, as nothing remains unmonitored as they migrate to Wavefront.
“Now, if you do actually get woken up at 2:30 AM, it’s because there’s actually a problem that requires your attention.”
Nick Warr, Head of IT, Bossa Studios
With Wavefront, there are no more alert false positives from earlier on, and no more unnecessary pager calls in the wee hours of the morning. Wavefront integrates alerts with a much more flexible dashboarding too. Alerts playback allows developers to tweak things a lot better, much more quickly, instead of reacting after the fact. Seeing how alerts act on past data, they can see how the alerts come through. This newly acquired ability to run alerts on past data allows them also to try different scenarios – a significant advantage. They can discern the sequence of events that caused an incident. They can try out the effects of accumulation, or if an incident X or Y happens after an amount of time. They can “proactively tune” alerts instead of reactively. “So, if you do actually get woken up at 2:30 AM, it’s because there’s actually a problem that requires your attention”, said Nick Warr, Head of IT, Bossa Studios.
The Wavefront Impact
The impact of Wavefront that the Bossa team is looking forward to – besides reduced troubleshooting times – is undoubtedly the total cost of ownership (TCO) savings. With little cost difference, they added distributed tracing with metrics analytics, to gain much more complete and accurate overall picture of their system health. The traces display insight into issues; internal visibility detail events inside the actual code. Some of it, a developer could trace previously through direct control over the server, logging a lot of it. Then the developer starts forensics, digging through all the logs, trying to find where the traces gave the exact timing per transaction. Now Wavefront make it a lot easier. The developer can now find code bottlenecks easier and faster. “So, Wavefront enables us with for more rapid iteration, so we can make more rapid improvements to our code, basically”, said Nick Warr.
Bossa’a premier use case for tracing is visibility into third-party services and APIs that they do not control. They use Amazon ELB and managed database services in the background through the RDS. The administrative overhead was a bit much for managing it all on their own. Now they don’t have to worry about administration and backups; all their systems are highly available. Wavefront dashboards overlay metrics and traces. Thus combined, everything gets pulled into one view.
Once you get a page, you can immediately identify a long expanse, and so on. Bossa is still finishing up the deployment of the “new pieces” before sending them to the Wavefront proxy along with metrics extrapolated from logs. They plan to stack up both at the same time. Afterward, and after obtaining everything from data logs, they will redeploy with just Wavefront. The simplicity of being able to integrate things effortlessly to a single unified observability platform is a great benefit.
“You click it, do what it says and in an instant, there’s a working dashboard. Since starting with Wavefront, Bossa has been able to do a lot more, with a lot more information at our fingertips.”
Nick Warr, Head of IT, Bossa Studios
Wavefront’s correlation analytics capability is unique, and the Bossa team sees significant value in it. It enables correlations across a full domain of system entities. There are some analytics functions that Bossa has deployed and integrated directly into the code for tracing purposes. Apart from that, the Wavefront functionalities they find particularly useful include Wavefront’s Linux integrations, Kubernetes dashboards as well as its numerous AWS integrations.
For Bossa, Wavefront provides a first pane of glass and is a one-stop shop for observability: “You click it, do what it says and in an instant, there’s a working dashboard. Since starting with Wavefront, Bossa has been able to do a lot more, with a lot more information at our fingertips”, said Nick Warr.
Moreover for Bossa, Wavefront is a “big winner”, and at the same time “so easy to use.”
They were happy with all the outstanding technical support, providing all the information they needed. “No ifs, buts, or maybes.” That’s also what made it a significant benefit. As Nick Warr and the Bossa engineers asked themselves: “Am I doing this the right way? Is this the right way to go about it? And it is.”
For Bossa, the overall experience with Wavefront has been like winning a game, over and over again.
Get Started with Wavefront Follow @stela_udo Follow @WavefrontHQ
The post Wavefront Multicloud Observability Helps Bossa Studios Improve Its Gamers’ Experience appeared first on Wavefront by VMware.
