While we eagerly await the initial availability of Project Ensemble, I want to share some customer use cases and go deeper into the technology and capability this solution will introduce to our customers in a special series of blogs. If you are not familiar with Project Ensemble, please take the time to read the announcement blog. Very briefly, Project Ensemble will seek to unify and simplify cloud management services and data into a common data model and add intelligent and actionable business insights focused on application-centric views and customized for personas who support those applications.
Before I dive in, please remember that anything discussed here is subject to change and may not make it into the final service. With that I’ll share our legal disclaimer.
- “This blog may contain product features or functionality that are currently under development. This overview of new technology represents no commitment from VMware to deliver these features in any generally available product. Features are subject to change, and must not be included in contracts, purchase orders, or sales agreements of any kind. Technical feasibility and market demand will affect final delivery. Pricing and packaging for any new features/functionality/technology discussed or presented, have not been determined.”
The Challenge: One VM Many Truths
When a virtual machine is created in vCenter, it is assigned an identifier and metadata about that virtual machine is stored in the vCenter database. Now, if vRealize Automation was used to create the virtual machine it is also assigned an identifier and stored in the vRealize Automation database with metadata unique to vRealize Automation. So, now you have a single object that has two sources of information about the state and attributes of that object.
Begin to add more services, vRealize Operations for monitoring performance and capacity, vRealize Network Insight for connectivity, application discovery and flows. All this information is useful in the context of each of these services, but there may be information in vRealize Operations that would be important to know in vRealize Automation.
Practically this presents some challenges. For example, if I want to verify that an application deployment from vRealize Automation is currently healthy in vRealize Operations I need to understand and access two different APIs. Then I need to correlate the information from both services to get the information I require.
This is just for a single object. If you need information about related objects such as the host, datastore, networking and so forth you can see that the complexity scales rapidly as more services are added.
The Solution: Unified Data and a Common Object Model
Project Ensemble will deliver a unified consumption surface that meets the unique needs of the cloud administrator and SRE alike. From an architectural perspective, this means creating a platform designed for programmatic consumption and a firm “API First” approach. I think you will agree, after you read this blog, that Project Ensemble’s APIs raise the stakes for what is required for management APIs. In this blog, I am working within the mind of the SRE, who is trying to access management tools programmatically and prefers to use APIs.
As mentioned in the introduction, Project Ensemble will bring in data from multiple management services (i.e., vRealize Operations, vRealize Automation and vRealize Network Insight) and normalize the data into a single, common object model. This will allow the AI capabilities within Project Ensemble to identify, recommend and broker automated actions to improve performance and efficiency and do so proactively.
However, there is another key capability that this will deliver: the ability to reference any object from any management service from a single API call.
Below is an example of what this metadata will look like in Project Ensemble (in this case an NSX Edge, not a virtual machine, but it applies to any object). Notice the unique properties from each vRealize service are listed here.
So, every bit of information from multiple services about any object will be available. Let us now look at how we can access this information programmatically using the API.
If you are used to working with vRealize APIs, you are familiar with RESTful APIs and our Swagger documentation which makes it easy to learn and test. Instead of REST, Project Ensemble uses GraphQL for data retrieval. This allows for more powerful and easier content retrieval. If you have not been exposed to GraphQL, I think you will find it even easier to use than REST and Swagger. For more information about GraphQL visit their website. For purposes of this blog, let us dive into how GraphQL works with Project Ensemble.
First, GraphQL is a query language and there are essentially only three types of request operations:
- Queries are the essence of GraphQL, fetching information about specific objects.
- Mutations provide ability to request the server to perform an action on objects.
- Subscriptions provide the ability to track changes in the system as they occur
See, already simpler than REST! Seriously though, GraphQL is ideal for solutions like Project Ensemble where the goal is to consolidate, normalize and simplify data from multiple systems. Project Ensemble will provide a web IDE for GraphQL, based on GraphiQL, but what you really need to know is that this will be an easy tool to use and familiarize yourself with the API.
Below is a current development version of the Ensemble API GraphiQL interface. This will likely change before it is publicly available but I want to at least show some concepts using this current state so you can understand the power behind this API approach.
I will not spend a lot of time explaining GraphQL and GraphiQL so if you are interested in learning more and even trying it out for yourself, there are a lot of public GraphiQL instances out there to play with. For example, I have enjoyed learning more by using the GitHub GraphQL API but there are many more you can find from this link from APIs.guru.
Examples of API Usage
Now that I have covered the background, let me show some examples of how this will work with Project Ensemble. Remember, this could change a bit before it is available to customers, but the essentials should give you a solid understanding of the proposed API capability.
Finding and Exploring Objects
Above, I discussed how Ensemble will consolidate and organize data from other systems using a common object model, for example a virtual machine. These objects are referred to as “entities” within Ensemble.
Here I have queried the API for any virtual machine entity with the string “Dina” in the name (and limited to the first 10 occurrences that match). I also want to know the services that are contributing data to the entity using the “entitiesIn” field.
The response includes the Ensemble virtual machine entities from vRealize Automation, vRealize Operations and vRealize Network Insight. I have only asked for the most basic details at this point, but I hope you can start to see the power in this approach.
Above I can see the unique ID assign to this virtual machine in each of the vRealize services. But what is great is that I do not actually need to know those IDs to get information about the virtual machines from each service. I can do that solely with the Ensemble entity ID.
For example, assume I want to know the CPU usage from vRealize Operations for this virtual machine. I can form a query like this:
I am using the entity ID, but specifically asking for data from vRealize Operations. This returns the “latest” metric values, but I can optionally add begin and end timestamps, rollup type, and other options as I would using the vRealize Operations REST API.
Now let us look at how you can perform actions on entities with the API.
Taking Action with Mutations
In REST, you typically use methods like PUT or POST to trigger some activity via API requests. In GraphQL this is an operation type of mutation. First, we need to know what actions are available. I use this query to find the actions I can use for an entity.
Ensemble responds with the available actions, and the availability depends on the state of the entity. For example, this virtual machine is powered on, so we see there is an action for power off, but not for power on – like how the vCenter UI would show power on actions as unavailable.
You may have noticed that I also queried for “actionRequests” and that returns a list of actions taken on this entity, so I have a way to audit activity.
To start the power off action, we send a mutation request. We need to provide the entityId and actionDefinitionId from the query above. Let us also request the actionRequestId and Status so we can check to make sure the action was successful.
We see the action was started.
Now we can check a few minutes later to confirm that it completed.
Success! You may have noticed that the action is taken through the vRealize Automation service, so that actions are committed within the policy and governance set by Cloud Administrators.
VMware Project Ensemble API will allow you to simplify automation by using a single API to retrieve information about and take action on business applications and workloads.
Stay tuned and watch for my next blog that will preview more capabilities of Project Ensemble.
Want to Learn More About Project Ensemble?
If you are interested in being considered for the beta to provide VMware with input, please let us know here (https://www.vmware.com/learn/1243700_REG.html) and your eligibility will be evaluated.