Cloud Automation Cloud Operations vRealize Operations vRealize Orchestrator Web/Tech

The Self-Healing Datacenter – Using Webhooks to Automate Remediation

Recently, Steven Flanders wrote a nice blog post explaining what Webhook Shims are and how they can be used.  In summary, both vRealize Log Insight and vRealize Operations support webhooks.  This is a capability that allows for integration with any other solution that has a REST API available.  Ticketing, notification, chat and other capabilities can be leveraged by vRealize Intelligent Operations.  Extending that capability can make your datacenter a self-healing datacenter!

If you are a user of vRealize Operations, you know that it can monitor your infrastructure, server OS, applications and more.  But as you know, monitoring is only part of the answer.  Wouldn’t it be much better to have vRealize Operations attempt some simple fixes before giving up and calling for human intervention?

In this blog post series, I will explain how to activate automatically remediate a problem detected by vRealize Operations using the vRealize Orchestrator Webhook shim to launch workflow based on an alert.

REST Notification Plugin

Currently, vRealize Operations alerting can trigger notifications via email, SNMP trap or REST notification.  The REST notification plugin allows you to interact with REST APIs from practically any third-party system – almost!

Setting up an Alert notification rule in vR Ops

The truth is the REST Notification Plugin is not very robust and does not provide a lot of control over format of the REST call.  You plug in the URI and some credentials and vRealize Operations will try a PUT and POST against that URI.  By the way, it will append the alert ID onto the end of the URI.

Adding a REST Notification Plugin instance in vR Ops. Note the limited options available; there is nowhere to format the request body or control the method used.

This means is that unless your REST API is expecting the request in the way vR Ops would like to send it – it will fail.

For example, Orchestrator has a robust REST API.  To launch a workflow via that API, I would send:

curl -X POST -H "Authorization: Basic dmNvYWRtaW46dmNvYWRtaW4=" -H "Accept: application/json" -d '{
  "parameters": [
    {
      "value": {},
      "name": "",
      "type": "",
      "description": "",
      "scope": "",
      "updated": false
    }
  ]
}' "https://10.140.45.13:8281/vco/api/workflows/a0b8b820-b8c9-454e-b128-c4eabb9d0015/executions"

Enter Webhook Shims!

As you can see, Orchestrator expects a body for this request that includes any inputs.  Orchestrator expects this payload even if there are no inputs.  My vRealize Operations REST call to Orchestrator will fail because it is not making a valid request.  How can we fix this?

One way is to translate the REST call from vRealize Operations into a REST call that can be consumed by Orchestrator.  Enter the Webhook Shims for Log Insight and vRealize Operations!  This diagram shows how it works:

Webhook Shims enables self-healing with vRealize Operations through an Orchestrator shim.

The Orchestrator shim takes the simplistic input from vR Ops REST notification and rebuilds the request in the format expected by Orchestrator.

In the next blog post, I will show you how to implement this solution to automatically attempt remediation based on alert notifications from vRealize Operations to create your own self-healing datacenter!