Today’s consumers expect continuous improvement in the apps they use. As a result, cloud-based application development teams need to innovate and release new app services and updates continually. An efficient alerting mechanism that augments the Continuous Integration and Continuous Delivery (CI/CD) pipeline is crucial to support this engineering and product velocity.
Wavefront is enhancing its alerts capability by introducing Threshold Mode, a new way to define and configure alerts. With Threshold Mode, you can define multiple severities and thresholds in a single alert, and you can notify a different alert target for each severity.
Classic Alerts
In Classic Mode, each Wavefront alert can be assigned to only one severity (Figure 1.). You define a condition on a metric and associate it with a single severity. When the condition is met, Wavefront notifies the target or targets. If you wanted to notify different people depending on the severity of the condition, you’d have to create 3 different alerts:
- Alert 1: If CPU is over 95, send a SEVERE PagerDuty notification to the on-call SRE and the manager of the SRE group.
- Alert 2: If CPU is over 90, send an INFO email to all SREs and a WARN email to the on-call SRE.
- Alert 3: If CPU is over 80, send an INFO email to all SREs.
Figure 1. Classic Alerts: One Alert per Threshold
Multi-Threshold Alerts
In Threshold Mode, you can define different thresholds and assign a severity to each threshold (Figure 2.). For example:
- If CPU is over 95, send a SEVERE PagerDuty notification to the on-call SRE and the manager of the SRE group.
- If CPU is over 90, send a WARN email to the on-call SRE.
- If CPU is over 80, send an INFO email to all SREs
Figure 2: Multi-Threshold Alerts: Single Alert with Multiple Severities, Thresholds, and Targets
Reduce the Number of Alerts in Your Environment
The benefits of introducing Threshold Mode alerts are:
- Ease of Administration: Using Threshold Mode increases your overall efficiency by reducing the number of alerts that you need to set up and manage in your environment. You can have a single alert on a metric with multiple severities, as shown in Figure 2.
- More flexibility: You can notify certain people, e.g., a manager, only when the alert becomes severe. At the same time, all users who receive notifications for lower-level targets continue to be notified as long as that target threshold is met. For example, if you send a SEVERE notification to the manager if CPU is over 95, then the condition for the other thresholds is still met and those notifications continue to be sent.
Start Benefiting From Enhanced Alerting
To experience Threshold Mode for alerts, create a new alert, and choose Threshold as the Type. Then, associate each severity of interest with a threshold value and notify different people depending on the severity of the condition. This threshold values will trigger the alerts at the specified severities.
With Wavefront multi-threshold alerts, you reduce the number of alerts you need to set up and manage. Also, with more flexible alert-tailoring to your DevOps team members, you speed up the customization of your CI/CD pipeline.
If you’re not yet an existing Wavefront customer, check out these new alerting features via a Wavefront free trial. If you have any questions, or you want to share your feedback, please slack us.
Get Started with Wavefront Follow @Gaanesh_K Follow @WavefrontHQ
Co-Author: Chung Kung Wang is a is a Member of Technical Staff at VMware. He is working as a full-stack engineer in the Wavefront team. Chung Kung Wang used to be a gamer, and now he’s trying to beat this game called Life. He loves to exercise in his spare time and has been hooked on climbing recently.
The post Efficient CI/CD Pipelines with Wavefront Multi-Threshold Alerts appeared first on Wavefront by VMware.
