In our last post, we looked at the IPMI SEL and how you can use that to gather events recorded by the hardware. In this installment, we'll take a look at a different sub-system – storage. In particular, we'll look at internal RAID card support so you can see the health of your RAID volumes and detect when disk drives fail.
At present, there are a few different RAID cards that are instrumented via the CIM code available from VMware and our partners, and that list will continue to grow over time. Starting in version 3.5, VMware developed a rudimentary CIM provider for LSI MegaRAID and IR cards that implements a small portion of the SNIA SMI-S Host Hardware RAID Controller Profile. The intention of this implementation was simply to report the health of the RAID volumes on the system and very basic configuration information, so users can take preventative action should a drive or component fail, before actual data loss occurs. Subsequent to 3.5, LSI developed their own implementation of the HHRC profile, which expands on the capabilities to include the ability to create RAID volumes, and perform other operations on the card. HP has also implemented an HHRC provider for their Smart Array internal RAID controllers. In this post, we'll focus on the basic read-only aspects for now, and we'll come back to some of the advanced capabilities of the LSI provider in a later post.
Before we dive into code, and look at the results on a few different systems, we need to talk a little about another CIM concept called "namespace." In CIM, the classes and instances you access exist within a namespace. There are two "well known" namespaces called "root/interop" and "root/cimv2" where the former is used to advertise profiles (we'll touch on that in more detail in a later post) along with other core infrastructure information, and the latter is the default implementation for the system. In VMware's case, our primary implementations exist in root/cimv2, and if you look back at the prior examples, they all hard-coded that namespace. When another vendor adds new functionality to the system, they typically place that implementation into a different namespace so there isn't any collision with the core system instrumentation. There's an easy way to discover all the namespaces on the system. Just query for the CIM_Namespace class in the root/interop namespace. Here's a short excerpt of code that does that:
list = client.EnumerateInstances('CIM_Namespace', PropertyList=['Name'])
return set(map(lambda x: x['Name'], list))
One little trick we're using in the above code is an optional parameter to EnumerateInstances called "PropertyList" which allows you to filter the properties in the results to speed things up. Since we know we only care about the Name property, we can skip all the other properties so they don't have to be sent over the network. Now if you run this on a ESXi 4.0 system, you might see something like the following: (output will vary depending on what add-on VIBs you've installed on your system.)
The primary class that we're going to look at which carries the most important RAID volume information is CIM_StorageVolume. The VMware developed LSI MR/IR implementation exists within the root/cimv2 namespace, while the LSI developed provider exists in an LSI specific, versioned namespace (lsi/lsimr12 in the example system above) and the HP implementation exists in root/hpq. Lets look at example output, first on an LSI based system using the VMware implementation:
StatusDescriptions = [u'Logical Disk is degraded', u'Logical Disk Extended Status']
StripeSize = 131072
SystemCreationClassName = SMX_SAArraySystem
SystemName = QT79MU0151
TransitioningToState = 12
You'll notice that the HealthState of both shows 10 (degraded/warning), which is the best field to monitor if you wanted to script something on top of this.
The complete script used for this post is attached here.
