Data opportunities surround us everywhere. A data enthusiast has unlimited possible sources of data in both their personal and business lives. One of those is your car. Combine a car enthusiast with a data enthusiast, and we may well be describing you. We’re certainly describing Wavefront co-founder Durren Shen.
As a driver, you, like Durren, might be interested in optimizing your ride – or at least understanding what’s going on – literally – under the hood. But there are other possible reasons for other people. If you’re the manufacturer of the car, you might want to see how it’s performing so that you can update the software with settings that make the car more efficient. If you’re a parent, you might want to know how – and when and where – your teenager is driving.
What we’re going to do here is take you through some of what Durren did with his BMW. The goal isn’t so much to show how to get data from a car; it’s to show how easy it can be to set up Wavefront analytics to visualize dashboards and receive alerts. What we do here can be directly extrapolated to much more complex, cloud-native application environments.
Getting to the Data
Every modern car has an OBD (or OBD-II) port through which performance data can be extracted, under the dash where your mechanic plugs his or her analyzer. There’s lots of data to be had from here, although the system maintains very limited historical data. So you can’t go out for a Sunday drive and then come home and download everything that happened; you have to send the data in real time and store it.
We can process data offline after the drive, or we can create real-time alerts that store and monitor data while the car is running. Bear in mind, however, that connected cars of the future are expected to send some 25 GB of telemetry data per hour. It takes a powerful data engine like the one Wavefront provides to manage the incredible amount of data that will result from managing an entire fleet of cars.
The one catch that stops us simply plugging in is that our laptop didn’t come equipped with an OBD-II connector. So, to complete the connection, we used an adapter that takes the OBD data and sends it via Bluetooth; we used the Bluetooth feature on the laptop to do the capture. The laptop will then listen for new data.
Simple Data Ingestion and Clean-Up
Wavefront tools can pull together data from multiple sources to create sophisticated analytics and alerts. Let’s start with a single data stream and look at some basic data munging.
The first thing we want to do is see what data is available from the OBD port. That lets us know what we have to work with, and it also keeps us from trying to calculate anything that’s already on offer.
Figure 1. The data available from the OBD-II port.
You’ll notice that the metrics are all prefixed with “vehicle” – a good thing to do in your polling script. From there, we can play with basic data like the fuel level using Wavefront’s powerful proprietary data language. While we’ll see a couple of examples from that language, there are over 100 analytics functions that Wavefront makes available to comb through a data warehouse.
To start with, we’ll often find that the data must be smoothed, since some raw data can be very noisy. For instance, the actual fuel level may show the fuel sloshing as you drive. That can be cleaned up by using a moving average, which doesn’t completely eliminate the noise, but provides some smoothing.
Figure 2. Raw (noisy) fuel-level data.
Figure 3. Fuel-level data with some noise removed using a moving average.
Finding the rate of fuel usage uses a first-derivative function, and derivatives are even noisier. They literally amplify any noise in the raw data. This can be smoothed with a moving minimum function.
Figure 4. Raw, noisy data for the rate of fuel usage
Figure 5. Fuel usage rate smoothed using a moving minimum function.
From this point, you can start looking for things that you’re interested in understanding – perhaps looking for performance or other correlations that might be hidden in the data.
Creating Dashboards
The fuel rate metric, viewed as a chart, is a good example of the information you’d want to include in a dashboard. A dashboard is a data-visualization tool that presents the current status of various metrics and key performance indicators (KPIs) for your system, whether it’s your car or your cloud application.
KPIs are most often “composite metrics” – custom metrics that you create using Wavefront analytics. Systems can create boatloads of basic metrics, but they are often highly granular, not representative of overall system performance, and not good indicators of what your customers are experiencing.
One of Wavefront’s most powerful capabilities uses query-driven analytics to create custom new composite KPI metrics based on one or more basic metrics – often from different pieces of the system. Composite metrics, like fuel rate in the previous example, are usually the most important metrics you want to track and use as a basis for alerts.
Wavefront’s dashboard tool bench allows you to customize your views, consolidating and arranging both metric representations and, in some cases, performance scorecards on a single screen. You can start with one of the many prebuilt dashboards that come with the growing library of Wavefront integrations, or you can create a dashboard entirely from scratch.
Durren created the dashboard below in about 30 minutes for his BMW. It gives you an idea of the richness and flexibility that comes with Wavefront’s dashboard workbench. In his particular case, he decided to track which gear he was in at what time; his speed, engine speed, throttle position, and engine load; time spent driving and stopped; percent and amount of fuel consumed; and a dozen other metrics, all available in an instant.
Figure 6. The dashboard that Durren Shen created for his car.
Creating Alerts
It is equally easy to set up alerts. The concepts we illustrate here can be extrapolated to include multiple data sources and complex evaluations. That said, let’s start with basics, since even complex alerts start out simple and get built up step-by-step.
The first thing we’re going to do is to create a low-fuel alert. Yes, you have a gas gauge that can tell you when you’re low on fuel. But an alert can break through the highway hypnosis way out in the middle of nowhere so that you can get to a station in time.
We start with our smoothed data from before, and we simply create an alert that is triggered when the fuel is at some level – let’s say 40%, just to pick a number.
Figure 7. A low-fuel alert.
Of course, 40% is still quite a bit of fuel, but that’s a trivial number to change in the alert definition.
Now let’s try something nominally harder. Let’s say we have a teenager old enough to drive, but that he or she isn’t allowed to drive after an 11 PM curfew. We also want to know if his or her driving speed exceeds 70 mph.
Let’s start with the speed portion of the alert. We can see that the speed data from the OBD port is clean and doesn’t need to have noise removed. So now we can simply create a condition that is true if the speed is greater than 70 mph.
One catch, however: you don’t want this firing when you’re driving – only when the kid is driving. At present, cars have no way of reporting who the actual driver is, so you need some other condition as a proxy for this. If you typically use the car between 8 and 5 on a weekday, then you can exclude this timeframe from the alert.
Figure 8. High-auto-speed alert.
For the curfew aspect of the alert, you can simply test whether the car is reporting any data at all after 11 PM.
Figure 9. Curfew alert
These two conditions can be combined into a compound alert that we’ll call, “Kid is grounded.”
Figure 10. “Kid Is Grounded” alert, combining both speed and curfew alerts
Next we need to decide how we want to be paged. If a simple email is enough, then you can fill in your email address. Wavefront can also alert via notification tools like Webhooks or PagerDuty. For example, if you want this to be a truly automated system, you can alert to Webhooks that’s waiting for alerts to fire. When Webhooks gets an alert, it then issues a ‘disable all iPad/laptop devices’ request to our router and cuts all internet access from the kids for a day or two.
This Works for Any Data Generators
What works for a metrics-generating automobile can work for any other metrics-data generator, collector, or library – especially those originating from cloud-native application environments. All we need to do is to capture and normalize the data; then we have a Wavefront language available to create analytics and alerts. Once you have a chance to look at what the data is telling you, you may learn surprising – and, hopefully, profitable – things.
To find out more about how you can put Wavefront to work on your metrics data, contact us at: [email protected]. To start a 30-day free trial of the Wavefront service, just click on www.wavefront.com/get-started.
Get Started with Wavefront Follow @WavefrontHQ
The post Interrogating Your Ride: Data Analytics for the Car Enthusiast appeared first on Wavefront by VMware.
