This blog post is authored by Bill Erdman and Sudhir Bhatti.
The use of unstructured data, for better understanding operational issues within the data center infrastructure, is fast becoming a mainstream technology. And where this data was once relegated to the propeller heads, who had intimate tribal knowledge of “machine generated data,” given the ability to collect this data at scale, slice and dice the data into meaningful trends, graphs, and events, there is a broader applicability of this unstructured data, for today’s operation management teams.
The key enabler is the management application itself, and its ability to collect multi-terabits of data, filter for significant activities, structure for human intelligence, and to report up into other systems in the form of events, alerts, and key trend indicators. vRealize Log Insight, as one of the core vRealize Operations products from VMware, is a key product here.
Network Log Management – NSX for vSphere Overview Dashboard
Networking devices generate ton of machine data, better known in the networking world as syslog. As a majority of networking devices use some form of Unix and Linux operating systems for running switching, routing, firewall, application delivering and a whole host of other unique data inspection and forwarding functions, log data is a standard operational data source for any type of administration, event, change, or status updating for networking devices.
Cisco, Arista, Brocade, Juniper, f5, Palo Alto Networks, Infoblox, Lenovo, NSX for vSphere, Netflow Logic all generate volumes of log data from their devices, irrespective of whether the networking services are virtual or physical. Each one of these respective networking vendors offer “handbooks” on what each one of their log messages means, with several of these vendors offering 1000 plus pages Syslog user guides where they describe each message type. Only the level 3, highly paid, certified network engineers, understand the meaning of these messages, and know what to look for when troubleshooting through log messages.
And while there have been purpose-built networking management applications, specifically in the area of security and change/configuration management, that leverage log messages for triggering security and change management events, log messages as whole have not been a mainstream source of information for more generalized operations management. Typically, a system administrator needs a degree in a particular vendor’s technology to make use of these messages. In summary while log messages offer a great source of near real time operations information, the tribal nature of these do not scale well for enterprise organizations building out cloud infrastructure operations team, with cross technology responsibilities. Networking operations management remains a challenge for them.
Log Insight is a game changer here for SDDC inspired organizations. As mentioned above, Log Insight collects volumes of log data, structures the log data and generates meaningful visualizations by summarizing into events and alerts, and archives for anyone to drill in for deeper analytics. Different than many of the purpose built networking tools, in which log data is leveraged, Log Insight is more general purpose for looking at the health, activity, change, event, and operations severity levels. Rather than having numerous purpose built networking tools, on a per vendor, per function basis, Log Insight offers a broader approach across many vendors and many networking functions. Content Packs uniquely customize Log Insight to each networking vendor. These Content Packs are readily available to download from the VMware Solutions Exchange, as well as available within the Log Insight product application catalog. With no more than a few mouse clicks, IT admins can download Content Packs developed specifically for the various technologies they have deployed within their data centers.
Over the past six months, VMware has been aggressively working with the leading vendors aforementioned above regarding the development of Content Packs. These Content Packs capture the key log messages, from huge volumes of unstructured data (where only the gurus know what they mean), to highly-structured data in the form of GUI representation within pre-defined Log Insight display widgets.
The below table summarizes different kinds of the network specific operations data contained with in these networking Log Insight Content Packs. As a disclaimer, the below table is a compilation of features across many of these Content Packs. Each Content Pack is a subset of these features.
Links to Networking Content Packs on VMware Solution Exchange
