Home > Blogs > VMware Cloud Management

Let’s talk about weather… in Log Insight

1950s Western Railroads poster Just imagine you are an IT guy, working hard in an air conditioned windowless (home-) office – your social life is limited to slack, skype, socialcast or some other social media plattform. Wouldn’t it be nice to pretend, that you participate in the life outside and chat about something not work related – for example about the weather.

In this small example, I will show how to retrieve world wide weather data for any city and visualise it in Log Insight without leaving the windowless office.  So in the next chat with your colleague  in Seattle you may surprise him with an accurate rain prediction.

I will gather data from the excellent and partially free openweathermap.org service – and we will do it for an array of cities. This data will be written to a log file and collected by the Log Insight agent on a  Linux system. In Log Insight we will create example dashboard for the “World wide current weather” and a second dashboard for the next day forecast. If you decide to take a shortcut and  import the provided example content pack, you can skip the field extraction and dashboard building, i.e. steps 6 and 7.

Step 1: Get your own personal api key from http://openweathermap.org/appid – just create a free account (sufficient for the example below) and get an API key.

Screenshot 2016-06-25 07.36.39

Step 2: In the next step we use the search on the openweathermap.org and write down a list of locations we will collect weather data. You may use geo coordinates or city codes, but in most cases a name the format “CityName,CountryCode” will be sufficient. My example list look like this:  “Yerevan,AM Seattle,US PaloAlto,US Sofia,BG London,UK Frankfurt,DE Sydney,AU”

Step 3: Now we will test the API key with a curl request. Please replace the api key in the command below with the key you retrieved in step 1. In the US (for Fahrenheit) you may prefer to use “units=imperial” instead of “units=metric” (for Celsius), if the units= is ommitted, Kelvin will be used. If you receive an error, check the doublequotes and check if you are using your own api key retrieved in step 1.

root@ugurke:~# curl -s "http://api.openweathermap.org/data/2.5/weather?q=Frankfurt,DE&mode=xml&units=metric&cnt=7&APPID=905730596699e6cd7d4c70a5eb8abb86"
<current><city id="2925533" name="Frankfurt am Main"><coord lon="8.68" lat="50.12"></coord><country>DE</country><sun rise="2016-06-22T03:15:43" set="2016-06-22T19:39:05"></sun></city><temperature value="19.58" min="16.67" max="22.22" unit="metric"></temperature><humidity value="83" unit="%"></humidity><pressure value="1005" unit="hPa"></pressure><wind><speed value="1.03" name="Calm"></speed><gusts value="2.57"></gusts><direction value="288" code="WNW" name="West-northwest"></direction></wind><clouds value="12" name="few clouds"></clouds><visibility></visibility><precipitation value="19.56" mode="rain" unit="1h"></precipitation><weather number="503" value="very heavy rain" icon="10d"></weather><lastupdate value="2016-06-22T06:16:02"></lastupdate></current>

Step 4: We will automate the process of data retrieving in a simple shell script. This script below is  just a very basic example. You may want to tweak it. You will need to provide your API key. You may want to use “imperial” units for Fahrenheit. This script will do a separate curl call for each city, gather data in XML  and add the output to the file /var/log/current.log and /var/log/forecast.log.

#!/bin/bash
# Add your own API key from http://openweathermap.org/appid
APIKEY=905730596699e6cd7d4c70a5eb8abb86
MODE=xml  #json or xml
UNITS=metric #metric or imperial
COUNT=1 #forecast days

# Add City,CountryCode to the list of cities
for CITY in Yerevan,AM Seattle,US PaloAlto,US Sofia,BG London,UK Frankfurt,DE Sydney,AU
do 
	CURRENT=`curl -s "http://api.openweathermap.org/data/2.5/weather?q=$CITY&mode=$MODE&units=$UNITS&cnt=7&APPID=$APIKEY"`
	FORECAST=`curl -s "http://api.openweathermap.org/data/2.5/forecast/daily?q=$CITY&mode=$MODE&units=$UNITS&cnt=$COUNT&APPID=$APIKEY"`
echo $CURRENT >> /var/log/current.log
echo $FORECAST >> /var/log/forecast.log
done

Copy the script to any linux box with log insight agent installed, make it executable (chmod +x) and add it to your crontab (crontab -e). I am running the collection every 10 minutes, so the crontab entry looks like this:

*/10 * * * * /usr/local/bin/getweatherdata.sh

 

Step 5: Now that we have a working cron job logging the weather data, we now need to configure the Log Insight server to collect this data.

On Log Insight Server: create a group or just add the following file directive to the agent configuration and check the results in interactive analytics. (You may skip this step and go directly to step 8 and import the content pack)

[filelog|weather]
directory=/var/log/
include=current.log
tags={"weather":"current"}

[filelog|weather-forecast]
directory=/var/log/
include=forecast.log
tags={"weather":"forecast"}

Step 6: In Log Insight “Interactive Analysis”  I used the semi-automated feature “field extraction” to mark, extract and name the fields used in the weather dashboards. XML is not a log format, so some manual tweaking was required. With the fields extracted and most of them containing numeric data it is easy to create colourfull and meaningfull dashboards. The following screenshot shows the automatic extraction of temperature value.  (You may skip this step and go directly to step 8 and import the content pack)

Extracting the field: temperature value

Step 7: Now, I can take any extracted data field and visualise it in the dashboard.  This example shows the extraction and visualisation of the cloudiness data field.

Extracting and visualizing cloudiness

And finally I have put this world weather station dashboard together:

Weather dashboard in presentation mode

Step 8: If you want to have a shortcut: feel free to use my example dashboard and fields included in this importable content pack. This will work with evaluation and full versions of Log Insight. It will not work with Log Insight “vCenter edition”, because in this edition the import button is missing.

tb_sc_2016-06-24_02-45-13_PM

After the import, you will just need to clone the agent group and apply it to the Linux system running the getweatherdata.sh script.

tb_sc_2016-06-24_02-46-43_PM

 

If everything works as expected, you should get two dashboards: one for the current weather and one for the next days forecast:

World Weather

World Weather dashboard

World Weather Forecast

World Weather Forecast

 

Some ideas for further enhancements:

  • You could create an alert for certain forecasted weather conditions, e.g. “Forecast: A sunny day  in Seattle” and automate your  time off requests through a webhook.
  • Enhance the forecast for up to 16 days, you may need to redesign the dashboard and re-define some fields

References:  you will find information on how to build your own content packs here: https://developercenter.vmware.com/web/loginsight  I also recommend Steve Flanders blog: http://sflanders.net/log-insight/

Download link for the unofficial World Weather Content Pack

 

 

 

Relationship Mapping Oracle-VMware in vROps

Kyle Wassink, Blue Medora

There are two key relationships that need to be identified in order to provide comprehensive Oracle monitoring in VMware vRealize Operations (vROps): relationships within the Oracle stack, and relationships between Oracle and the VMware stack. When combined, these relationships give you a complete picture of the Oracle on VMware stack for your environment. Let’s take a look at how we can identify these relationships with simple Relationship Mapping using vROps.

Relationships within the Oracle Stack

Identifying the first key relationships, those within the Oracle stack, is straightforward. The Oracle Management Repository (OMR), which is queried by the Management Pack for Oracle Enterprise Manager, contains a database view named MGMT$TARGET_ASSOCIATIONS. This view simply copies over all of the relationships that Oracle EM has already created in their platform, making Oracle relationship mapping readily available in vROps.

Screen Shot 2016-06-18 at 4.13.02 PM copy

Figure 1 –  The Oracle Host (top) and the related Oracle resources in a vROps hierarchy

Continue reading

Zebra Technologies Unifies Hybrid Cloud with vRealize Suite

by Rich BourdeauHybrid Cloud

You have seen the surveys, hybrid clouds are everywhere.  If you are not using one today you are considering using one or so the experts are telling us.  In a March 2016, Tech Pro Research Survey 88% of the respondents said that they were familiar with the concept of hybrid clouds and 68% had already deployed or were considering deploying a hybrid cloud.  Based on these numbers, it sounds like everyone is doing it.

However, when I talk with different companies the reality is that far fewer companies are actually enjoying the benefits of hybrid cloud than the surveys would indicate.  That is because everyone is using a different definition for what hybrid cloud is.  If you Google “Hybrid Cloud Definition” you will see the following description about what makes a hybrid cloud.

Continue reading

VMware Welcomes Arkin To The Family

We’re excited to announce we’ve closed our acquisition of Arkin and I want to formally welcome the Arkin employees to VMware! This is an exciting time for both of our companies, and based on market feedback, there is a tremendous amount of energy around this acquisition from customers, partners and influencers. Here’s what IDC had to say about the acquisition:

“Enterprise customers see the value of SDDC and SDN, but they often struggle with how to deploy and operate the technologies most effectively. In acquiring Arkin, VMware will provide customers with greater visibility spanning physical and virtual infrastructure while also making NSX easier to deploy for use cases involving micro-segmentation.” – Brad Casemore, director of research for datacenter networking at IDC.

This acquisition has been almost universally applauded as a move that will help us accelerate momentum for VMware NSX and the Software-Defined Data Center. The addition of Arkin to VMware vRealize Suite is expected to translate into exponential value to our customers, including:

  • unprecedented visibility and insight to the datacenter traffic flows
  • faster deployments via micro-segmentation planning; and,
  • significantly lower time to resolution post deployment across virtual and physical layers, leveraging constructs like time-machine, natural language search and contextual analytics.

VMware’s vRealize Suite will have the ability to extend across the entire SDDC, including the VMware NSX and network virtualization layer, as a result of this acquisition.

Ajay

Monitor Your Customized Applications in vRealize Operations

Alain Geenrits, Blue Medora

If you want to monitor your complete infrastructure today, vRealize Operations (vROps) is second to none. It allows you to watch your whole VMware environment automatically through a vCenter connection.

At Blue Medora we broaden your horizon even more. Our management packs connect your vRealize Operations to infrastructure and application components, allowing you to monitor all from one central point. NetApp storage, Cisco UCS servers, F5 BIG-IP load-balancers, Oracle databases, and much more now become objects – with their metrics and properties – in your vROps environment. And we perform this magic without installing a single agent – I promise!

Of course this may mean you end up with hundreds of virtual machines, datastores, hosts, and more in your environment; we need some ways to organize. It helps of course that vROps detects relationships automatically – a virtual machine is tied to a host, which in turn is linked to a cluster, and so on. Blue Medora management packs use that same functionality to detect relationships and tie, for example, a Microsoft SQL server database to the virtual machine it is running on, or a Dell server to the hosts and virtual machines running on it. If you open the inventory screen you will see inventory trees for all adapters and management packs you have installed, as seen in Figure 1.

vrops - inventory

Figure 1 – Inventory Trees for each object are displayed along the left side of the Environment Overview

If you click the F5 BIG-IP environment for example, you will get an overview of your applications services, pools and Big-IP systems. Note also that you get the famous badges calculated for these objects as well.

F5 inventory

Figure 2 – Badges for our F5 BIG-IP system

But what if you want to quickly monitor a group of objects that constitute an application in your company? Or you want to give customers access to just their environment? Well, there is a well-kept secret that you may have spotted in the screenshot already: Applications. The name is a bit unfortunate since they are not application objects imported through management packs, but constructs created from your own creativity and needs!

Add Applications and Constructs

Click Applications in the Inventory screen and the plus sign to create a new one. You will be presented with some templates to start. I have to stress here that the tiers are just a representation, you fill them with the objects you want and you can create your own tiers.

define applications

Figure 3 – The Add Application dialog box

When you click OK a selection screen will pop up with your empty tiers and selection boxes underneath. Don’t forget to change the name of your application! When you select a tier, you can drag any object to it. Open up Adapter instances to see any object imported through a management pack. Beware of the multiselect devil; each time you click in the window it adds  to the selection. This can be very confusing. To deselect, click the  the cross. I have added a Lenovo switch port and a Big-IP pool under the network tier here just to show that you can add any items.

applications tiers

Figure 4 – Adding objects to my custom application

Go ahead and add some elements of your choice. After you save you will see the application overview with our trusted badges and overview. You might notice that everything is grey, though. Since you have just created a new container in vROps, it needs to recalculate the badges. Go grab some coffee…

finnished application

Figure 5 – The overview screen for our custom application

Add and Monitor from Overview

You can add applications and monitor them from the overview screen, click to see details and related objects, and all vROps goodness. It gets even more interesting when you use application objects in your dashboards. How about a quick overview for management of the health of all critical applications? You can create that with one dashboard and the health score widget. Just select your newly created applications in the selection under Object Type, Applications. And voilà!

Monitor from applications dashboard

Figure 6 – Our new dashboard using the custom application

I hope I have shown you a new way to organize objects in vRealize Operations and derive added value from them. Let me know in the comments how you will use applications!

User Permissions: Staying Secure with vROps

Jake Martin, Blue Medora

VMware’s vRealize Operations (vROps) takes advantage of user accounts and security in a number of different ways. Secure monitoring and varied access to particular sections of information by managing user permissions are just a few of the options available that we will discuss.

The majority of vRealize Operations management packs require a service or admin account to authenticate to the internal vROps REST API. The default admin account can provide this functionality, but is not considered ideal for network security.

Manage Credentials

Figure 1 – Using the default vRealize admin account for authentication

Creating a Service Account

The more ideal situation is to create a service account, whose only purpose is to authenticate to the internal REST API. This, as well as all user configurations within vROps, can be accomplished within the Access Control tab, which can be reached by navigating to Administration then Access Control as seen below in Figure 2.

Creating a Service Account

Figure 2 – The Access Control tab of vROps

User Permissions with Groups

Another useful configuration is groups. These can be accessed by navigating to Administration -> Access Control -> User Groups. With groups you could, for example, add all of the members of the data center team to a single group, then allow that group access to all NetApp dashboards and alerts, but deny them access to SAP dashboards and alerts. In this way, when a data center user logs into vROps they are only presented with alerts pertaining to NetApp.

manage user permissions with groups

Figure 3 – User groups in vRealize Operations

User Permissions with Roles

If groups are too broad of a configuration or if you want to customize access even further, vROps Roles could be the answer. Within roles you have the ability to create or modify new and existing roles, granting a wide range of rights and privileges. It is then as simple as assigning a role to a newly created user to grant that user all rights and privileges within it.

manage user permissions with roles

Figure 4 – Roles in vRealize Operations

Depending on the desired outcome, one or a combination of these approaches could be used to lock down your vROps cluster and protect your sensitive data. With the user friendly interface, even beginner admins should have no issue customizing the system to suit their needs while still keeping their security on lockdown.

Maintenance Mode in vRealize Operations

Greg Hohertz, Blue Medora

Maintenance mode is a feature of vRealize Operations Manager (vROps), which prevents a resource from presenting misleading data or status during planned changes or unscheduled outages to the system’s software or hardware. In this blog, we’ll take a look at a couple ways to place resources in maintenance.

Manually Placing a Resource in Maintenance Mode

The simplest way to place a resource into maintenance is manually, via the Administration interface of vROps. Log into vRealize Operations and navigate to Administration, then Inventory Explorer (in 6.0.x it is called Environment Explorer). From here, you can search for the resource you would like to put into maintenance. Note that the first column contains the name of the resource, the second column the adapter type, and the third is object type.

Figure 1 - Manually Starting Maintenance Mode - Finding the Resource-border

Figure 1 – Manually Starting Maintenance Mode – Finding the Resource

Once you’ve found the resource you’d like to place into maintenance, left-click on the row. Next, click on the Start Maintenance button in the menu bar. You’ll be presented with a dialog box which will ask you if you’d like to end maintenance on the resource yourself (manually), or if you’d like to end it automatically, either after x minutes or on a specific date.

Figure 2 - Manually Starting Maintenance Mode - Starting Maintenance-border

Figure 2 – Manually Starting Maintenance Mode – Starting Maintenance

To manually end maintenance on the resource, you would navigate to this same location, find the resource, then click on the End Maintenance button.

Using Maintenance Schedules to Schedule Recurring Maintenance

Maintenance Schedules can be used to schedule maintenance for a resource which undergoes maintenance at a fixed interval. For example, a database might perform maintenance every Friday from 8pm to 9pm. A maintenance schedule can be created so that vROps does not show misleading data or health status during the scheduled outage.

To create a maintenance schedule, navigate to Administration and then Maintenance Schedules, then click on the Add (green plus sign) icon in the toolbar. Enter a friendly name for this maintenance schedule. You may want to come up with a naming standard so that the schedule can be easily identified in the future. Enter the maintenance window by indicating a start time and an end time. Note that you can also use days to indicate maintenance that extends a day or more. Next, specify the recurrence pattern. In our example, I’m creating a database maintenance window from 20:00 to 21:00, weekly, on Friday.

Figure 3 - Creating a Maintenance Scheduled-border

Figure 3 – Creating a Maintenance Schedule

Next, this newly created maintenance schedule will need to be assigned to my database resource. To accomplish this, I’ll need to either create a new database policy which contains the database(s) I wish to apply maintenance to, or use a pre-existing policy which contains the databases in question. To create a new policy, navigate to Administration then Policies and select the Policy Library tab. Click on the green plus sign to create a new policy and give it a friendly name such as “Production Databases”. Next, select Analysis Settings then filter the resources you’d like to apply the maintenance schedule to. In this example, we’ll select MS SQL Server.

Figure 4 - Applying the Maintenance Schedule to the Policy-border

Figure 4 – Applying the Maintenance Schedule to the Policy

Expand the Microsoft SQL Server until you find Time Range. Click on the lock to unlock this section for this policy. Use the drop down next to Maintenance Schedule to select your newly created maintenance schedule, then click Save.

With maintenance schedules in place, you’ll no longer receive those false alerts during the Friday maintenance schedules on these databases. You may consider additionally adding the virtual machine and/or Endpoint Operations Agent to this same schedule, so that the database, operating system, and virtual machine all have monitoring suppressed during this scheduled maintenance.

Try a hosted evaluation of vRealize Operations in a Hands On Lab.

 

Seamless Integration of vROps Using Third-Party Management Packs

Chuck Petrie, Blue Medora

Now more than ever, IT departments are faced with an overwhelming number of avenues to manage and analyze their IT environment. These avenues have a wide range of management options created in-house to vendor-created management interfaces. With so many options to choose from, it’s hard to know the best choice. One common theme in making a decision is efficiency of the solution. For example, it’s more efficient to monitor all of your systems from a single console with comprehensive management packs than it is to use a half dozen tailored monitoring platforms for each layer of your stack.

Today we will talk about how third-party management packs seamlessly integrate into vRealize Operations (vROps), turning vROps into a point of data aggregation and simplifying your monitoring needs. Once you’ve learned how to use a single third-party management pack, you’ll be able to expand your knowledge and use vROps as your single management console.

Continue reading

New Forrester Study Says Virtualization Continues To Grow

VMware commissioned Forrester Consulting to evaluate current levels of virtualization maturity and the challenges customers have at each level of maturity. In conducting in-depth online surveys with 210 US and European IT decision makers, Forrester found that the journey to greater virtualization and cloud is accelerated when companies acquire the right management tools, automate the IT operations life-cycle, balance cost savings with agility, prioritize speed, establishing governance and making developers as productive as possible.

We’ve developed a new Infographic to summarize the results.

78% of IT organizations plan to grow their virtualization environment

Most organizations require a higher level of virtualization than they have today. In fact, 73% of respondents said that they want to virtualize more than half of their workloads, but only 29% have crossed that threshold today. 78% of IT organizations plan to grow their virtualized infrastructure in the next year to deliver on-demand cloud-style responsiveness and agility. Automation, enabled by powerful tools, is the fastest path to expanding virtualization and delivering resources at cloud speed.

virtualization

New Forrester Study Says Automation is Key in Customer Virtualization Journey

Key Findings: Only 24% of companies can provision in hours

1. As companies virtualize more, lack of visibility and tools creep become the top challenge
As companies get more virtualized, they encounter more challenges with visibility, in part because of the number of tools needed to manage their infrastructure.
2. Only 24% of companies can provision infrastructure in hours – the rest take days or weeks
Regardless of how virtualized an organization is, to move forward they must automate. Automation, enabled by powerful tools is the fastest path to expanding virtualization and delivering resources at cloud speed.
3. Companies need unified, simple and cost-efficient tools to deliver resources at cloud speed
To move towards greater DevOps efficiency, organizations need automation solutions that help them maintain vision and control while helping them accelerate the development process.

To learn more about how you can accelerate your virtualization and cloud journey, check out the Forester Cloud Maturity Study and the Forrester infographic.

Have questions about the results? Post a comment below.

How to Get the Perfect Graph in vROps

Kyle Wassink, Blue Medora

Graphs in vROps

Graphs can be found throughout VMware vRealize Operations (vROps), from the All Metrics page of a resource to the Metric Charts widget in a dashboard. While simple to use, these graphs are extremely powerful tools for displaying historical data exactly how you want to view it. Let’s take a look at some of the key display settings that allow you to transform your historical data.

Note: The toolbar with the graphical settings, as well as the graphs themselves, vary slightly from vROps 6.0 to 6.1. The settings covered below are available in both versions unless otherwise noted. All screenshots are from a 6.1 system.

Displaying Graphs from Multiple Resources

One simple yet powerful feature of graphs is that they can display multiple resources, multiple metrics, or both. To see what this looks like, use the navigation tree to find a resource, then go to the Troubleshooting > All Metrics page.

Screen Shot 2016-05-17 at 2.59.47 PM

Continue reading