Home > Blogs > VMware vSphere Blog > Monthly Archives: March 2012

Monthly Archives: March 2012

NFS and the vCD Appliance

Tom Stephens
Senior Technical
Marketing Architect

I’m sure many of you have heard about the vCloud Director Appliance by now.   It’s one of the fastest ways you can get vCloud Director up and running to evaluate it in your environment. 

The key here is that it is only supported for evaluation environments.  As a result, certain design decisions were made with this in mind.  For example, even though it is based on CentOS, only packages that were critical to its use are included.  This helps to keep the size of the vCloud Director Appliance down, making it quicker to download and install.

The configuration provided should be more than sufficient for most evaluation environments.   Every once in a while though, someone has a need for something that surpasses what the vCloud Director Appliance was designed for.

Recently I was made aware of someone who had such a requirement.  What they needed was an ability to mount a NFS share from within the vCloud Director Appliance.

In a production environment, customers often use a NFS share on the vCloud Director cells to provide adequate space for the transfer service operations.   When this person tried to do this with the vCloud Director Appliance, they quickly realized that they couldn’t. 

Screen shot 2012-03-29 at 11.12.26 PM

The reason for this is because not all the NFS related packages are provided with the vCloud Director Appliance.  Normally, this is not a concern as the amount of transfer service space provided with the appliance is adequate for evaluations.  However, if you find yourself in a situation like this person was in where you need it, I’m going to show you how to get NFS working on the vCloud Director Appliance.

Before I do though, I’d like to place a special note here that this is UNSUPPORTED.  As you will have to install packages that are not included with the vCloud Director Appliance, that means that we do not perform any functional testing with these packages installed.    So if you do this, you are doing so at your own risk. 

Now the process to get NFS functionality on the vCloud Director Appliance is pretty simple.  First, make sure that your vCloud Director Appliance is connected to the Internet.  This is required, as it will be downloading the correct packages to install.  Next, use yum to install the portmap and nfs-utils packages.  You can do this by logging into the vCloud Director Appliance as the user root (where the default password is Default0) and entering the following commands:

# yum install portmap

# yum install nfs-utils

After you do this, you simply need to start the portmap service with the following command:

# service portmap start

You should be able to mount a NFS share now, using a command similar to:

# mount –t nfs4 mynfsserver:/nfs_share /nfs_mountpoint

for NFS v4, or if using NFS v3 or v2…

# mount –t nfs mynfsserver:/nfs_share /nfs_mountpoint

That’s all there is to it!

Debunking Storage I/O Control Myths

I had the pleasure of presenting to a number of our customers and storage partners last week. Both sessions were very interesting as it gave VMware technical marketing a chance to elicit some good feedback about what's working well and what's not working well from a vSphere perspective. One of the things which surprised me was how some people are still not using the Storage I/O Control (SIOC) feature, which I see as one of the most helpful storage performance management features that we have in vSphere. So I started to ask why customers weren't using it and these are the responses I got.

1. Some customers don't actually know about this feature and what it does.

Read all about the cool features of SIOC here in Duncan Epping's great blog post.

2. Some customers find that it is too difficult to configure.

Well, it is quite straight forward to do. First you have to enable it on the datastores. Only if you want to prioritize a certain VM's I/Os do you need to do additional configuration steps such as setting shares on a per VM basis. Yes, this can be a bit tedious if you have very many VMs that you want to change from the default shares value. But this only needs to be done once, and after that SIOC is up and running without any additional tweaking needed.

3. The feature is not granular enough.

I have heard that some customers want IOPS to be managed via resource pools in much the same way as CPU and Memory resources can be configured. Using this method, a group of VMs residing on a datastore can be given a maximum number of IOPS to use, and a different group of VMs placed in a separate resource group, but on the same datastore, can be given another maximum amount of IOPS. This would allow a more granular approach to controlling I/O, as well as easing the configuration steps, and give it the same look and feel as we have for resource pools right now. Right now SIOC works off of all the VMs on the whole datastore, but we may look at this at a later date.

4. Customers who require all VMs to get equal access to the datastore don't believe that they need SIOC.

The thing is, without SIOC, you could definitely hit this noisy neighbour problem where one VM could use more than its fair share of resources and impact other VMs residing on the same datastore. So by simply enabling SIOC on that datastore, the algorithms will ensure fairness across all VMs sharing the same datastore as they will all have the same number of shares by default. This is a great reason for admins to use this feature when it is available to them. And another cool feature is that once SIOC is enabled, there are additional performance counters available to you which you typically don’t have.

5. Some customers are finding it hard to identify the correct latency threshold to set on the datastore.

I admit that this can be difficult to determine, and that we currently set a 30ms threshold by default, without ever profiling the underlying datastore to see if this is an ideal threshold. However 30ms is an appropriate threshold for most applications. All I can say at this point is that we understand that this is a concern, and we are actively working on a mechanism to automatically determine an appropriate latency value for datastores. In the meantime, you may want to have a discussion with your storage array vendor, as they often make recommendations around latency threshold values for SIOC.

6. Some customers mentioned that they are seeing 'external workloads' causing issues for SIOC.

One reason that this can occur is when the back-end disks/spindles have other LUNs built on them, and these LUNs are presented to non ESXi hosts. Check out KB 1020651 for details on how to address this.

7. Certain customers couldn't use it because they do not have correct licensing edition.

SIOC does indeed require Enterprise Plus.

8. Lastly, some customers are using a version of vSphere that doesn’t have SIOC.

For block storage you would need to use vSphere version 4.1 or later. For NAS storage, you would need to use vSphere version 5.0 or later.  In this case, these customers will need to update to a newer version of vSphere to use SIOC. Its a great reason to upgrade.

If you are not yet using Storage I/O Control, I would be interested in hearing why. If there is some other concern which is preventing you from using this feature, I would also like to hear from you. Please feel free to leave a comment.

Get notification of these blogs postings and more VMware Storage information by following me on Twitter: Twitter @VMwareStorage

Free ESXi Hypervisor – Auto Start Breaks with 5.0 Update 1

Kyle Gleed, Sr. Technical Marketing Manager, VMware

If you are running the free version of ESXi (aka ESXi Hypervisor) then you'll want to be aware of a critical issue that surfaces after upgrading to 5.0 Update 1.  I want to stress this only applies to people runing the free ESXi version.  If you're ESXi hosts are licensed this issue does not affect you.

There were some changes made in the way the ESXi APIs are accessed in Update 1 that unfortunately breaks the VM Auto Start feature in the free ESXi version.  Please note that this issue only affects the free ESXi version and it was not intentional.  I've heard some people making claims that VMware did this on purpose as a way to further limit the free ESXi version, this is not true.  I'm not aware of a workaround, but many customers have been able to leverage the dual-image architecture of ESXi in order to roll-back to the pre-upgrade state.   

To roll-back to the pre-upgrade image:

Using the DCUI is the supported and recommended way to rollback your ESXi image.  From the ESXi host console reboot the host, at the very beginning of the boot, when you see the boot options prompt (<ENTER: Boot>), press SHIFT+R to enter recovery mode, and when prompted answer "y" to confirm that you want to revert back to the previous image profile.


ESXi Shell
If you don't have access to the host console you can still revert back to the pre-upgrade image profile by editing the "boot.cfg" files located in the /bootbank and /altbootbank directories (yes, you have to edit both files).  Note that manually editing these configuration files is not supported by VMware, so you never want to do this on a production ESXi hosts.  However, because this issue only affects the free ESXi version, and because I've had a couple folks ask how this can be done I wanted to go ahead and show you.  Assuming the last line in your /bootbank/boot.cfg shows "updated=2" and the last line in your /altbootbank/boot.cfg shows "updated=1", simply reverse the numbers so that /bootbank/boot.cfg is "updated=1" and /altbootbank/boot.cfg is "updated=2".  For example:

/bootbank/boot.cfg = "updated=2"
/altbootbank/boot.cfg = "updated=1"

/bootbank/boot.cfg = "updated=1"
/altbootbank/boot.cfg = "updated=2"

After editing the boot.cfg file in both the /bootbank and /altbootbank reboot your host for the change to take affect.  Having reversed the order of the bootbanks in the boot.cfg file the host will now reboot off the pre-upgrade image. 

Keep in mind that it is always a good idea to take a backup of your host (vicfg-cfgbackup) before attempting to revert your image profile.  Again, please note that the supported/recommend method is using the DCUI.  Only resort to editing the boot.cfg files for noncritical hosts such as those used in home labs and other test and development environments as any mistakes will very likely render your host unable to boot.

Many thanks to William Lam (@lamw) for his help on this blog…

Follow me on twitter @VMwareESXi

Technical Marketing Update 2012 – Week 12

By Duncan Epping, Principal Architect.

Technical Marketing Update 2012 – Week 12

This week I have a very cool white paper for those interested in Disaster Recovery / Distaster Avoidance. Ken Werneburg wrote an excellent comparison of the two solutions and what the advantages of each are. A highly recommended read! 

  • Stretched Clusters and VMware vCenter Site Recovery Manager - http://bit.ly/GHhW7G
    This paper is intended to clarify concepts involved with choosing solutions for vSphere site availability, and to help understand the use cases for availability solutions for the virtualized infrastructure.

Blog posts: 


VASA, Profile Driven Storage & RDMs – Heads Up.

I had a discussion earlier this week with Cody Hosterman from EMC. Cody was working on VASA & Profile Driven Storage, and spotted some anomalies in the way the feature works with RDMs, VMware's Raw Device Mappings.

RDMs, as the name implies, using a mapping file to maintain details about a physical LUN that is presented directly to a Virtual Machine. This mapping file sits on a VMFS filesystem. Now, even though the underlying LUN/Raw Device may have a unique set of capabilities, it seems that Profile Driven Storage is using the capabilities of the VMFS volume on which the mapping file resides (and not the physical device) to determine if the RDM is compliant or non-compliant with the capabilities defined in the profile.

It has since transpired that RDMs are not supported by Profile Driven Storage feature at this point in time. Just a heads up in case you come across something similar with your implementation. Good catch Cody.

We'll work on getting a solution to this into a future release, but no timeframes yet I'm afraid.

Get notification of these blogs postings and more VMware Storage information by following me on Twitter: Twitter @VMwareStorage

Setting up a Shared VMware Tools Directory

Kyle Gleed, Sr. Technical Marketing Manager, VMware

I often get asked why VMware provides two different image profiles as part of the ESXi offline bundle. For those who are not familiar with what an offline bundle is think of it as a copy of the installation ISO in a zip format.  The offline bundle is meant to be used with the ESXi Image Builder CLI to create custom installation images. These custom images, referred to as image profiles, can then be used to install ESXi hosts by exporting them as an ISO image or uploading them to an Auto Deploy server.

The offline bundle, sometimes referred to as an offline depot, can be downloaded from the same place where you download the ISO image (look for the .zip file).  The offline bundle comes with two pre-configured image profiles that you can use right out of the box, one with VMware Tools (standard) and one without (no-tools):


The reason for providing two separate image profiles has to do with using vSphere Auto Deploy to PXE boot your hosts. The full ESXi 5.0 image is approximately ~300MB and of this the VMware Tools accounts for roughly half (~147MB). While in many respects a 300MB installation image is quite small, it’s important to understand that when PXE booting the full image gets copied over the network each time a host boots. When working with a large number of hosts, and especially when many hosts are likely to boot concurrently (i.e. boot storm), the summation of many 300MB images going over the network can quickly add up, and as such it helps to use as small an image as possible. Because the VMware tools aren’t required to boot an ESXi hosts, and because it accounts for almost half of the image size, one easy way to reduce the size of the image and improve PXE boot performance is to exclude the tools package. Hence why we provide the two separate image profiles.

Continue reading

Customers and Partners Weigh in on Why vSphere 5 is The Best Platform for Cloud Infrastructures

It’s been a little more than seven months since we announced the general availability of VMware vSphere 5. With almost 200 new and enhanced capabilities, vSphere 5 simplifies the lives of our customers and delivers quick and tangible value to their organizations. Simply put, with vSphere 5 and the VMware Cloud Infrastructure Suite, we are enabling our customers to take advantage of the next era of IT.

Since we launched vSphere 5, the feedback we’ve been getting from our customers and partners has been really positive. I’d like to share some interesting insights from a recent TechValidate survey of more than 1,200 VMware beta customers and partners:

  • 70% of respondents surveyed have downloaded and deployed vSphere 5 and are taking advantage of the great new features and capabilities that vSphere 5 has to offer.


  • 82% of those respondents are using vSphere 5 to virtualize their business critical workloads and applications. vSphere 5 is designed to enable customers to run their business critical applications with confidence and this is made possible in part by the great availability features in vSphere 5. Customers and partners are excited about vSphere’s availability features and in particular are using vMotion (92%), High Availability (87%) and Storage vMotion (68%).


  • One of the significant advancements in vSphere 5 is all about automation, no more manual processes, it’s about automating everything. With vSphere 5, customers and partners have the ability to automate datacenter resource management to respond to the needs of their business faster and reduce operating expenses. More than 70% of respondents are achieving OpEx savings with vSphere 5.


  • Other key advancements of vSphere 5 are the storage enhancements we made to simplify the management of storage and automate load balancing capabilities. We found that 80% percent of respondents have upgraded to VMFS 5, and are able take advantage of the increased scalability and performance of this filesystem. Additionally, we found 52% of respondents are using Storage DRS and are able to take advantage of how this feature automatically manages the placement and balancing of a VM across storage resources. We are also seeing great uptick of new storage features including Storage I/O Control, which allows customers to configure rules and polices to specify the business priority of each VM and Profile-Driven Storage, which streamlines storage provisioning and ensures application services match the available storage.

We want to share some commentary from customers on their experiences, lets hear it in their own words:

“vSphere 5 is a game changing product. Much better product than anything else on the market. vSphere is 2 years ahead of the competition.”

“I have deployed vSphere 5 globally, upgrading from version 4. Being able to deploy Linux and Windows servers on ESXi reduces our cost of purchasing hardware globally. We also benefit from High Availability (HA), and other products, such as vCenter Site Recovery Manager (SRM).”

“I believe we will be able to accommodate larger workloads with more confidence using vSphere 5. The Storage Distributed Resource Scheduler (Storage DRS) feature is fantastic as we migrate VMs for efficiency and refresh.”

“We’ve dramatically reduced timelines from first deployment to seeing relevant returns on investment, greater agility and better consolidation ratios including business critical workloads with vSphere 5.”

You can see the full results of the survey by visiting the TechValidate results page here.

-Michael Adams, Group Product Marketing Manager, vSphere

Ye olde Controller, Target, Device (ctd) Numbering

I was having a discussion today about something that had not come up in a long, long time. It was about how controller numbers, target numbers and device numbers are assigned on an ESXi host.

The scenario is a Microsoft Exchange implementation where the requirements are to have separate LUN for each DB & Log. The net result is a requirement for 170 LUNs. Our configuration maximums guide states that an ESXi host will support a maximum of 1024 paths which, with 4 paths per LUN, allows for 256 LUNs (which is also the maximum number of LUNs per ESXi host too). There was some confusion with our Virtual Machine Storage Maximums, especially around the number of targets, so I put together this scenario to explain the host maximums.

Let's say that there are two HBAs in the ESXi host and all 170 LUNs were presented from a single array via two storage processors. You would theoretically see paths like this (I'll come back to why I use the word theoretically shortly):

– from the first HBA to first storage processor: c0t0d0 … c0t0d170
– from the first HBA to second storage processor: c0t1d0 … c0t1d170
– from the second HBA to first storage processor: c1t0d0 … c1t0d170
– from the second HBA to second storage processor: c1t1d0 … c1t1d170

Here c0 & c1 represent the HBAs (controllers), t0 & t1 represent the storage processors. I should point out that in later versions of ESXi, the controller number was changed to include the actual HBA name (e.g. vmhba1) and that the 'c' reference now relates to channel, but for the purposes of this discussion we'll stick with the older nomenclature. In the list above, d0 thru to d170 represent the devices/LUNs. These are the same set of LUNs, but discovered on different paths/HBAs/targets.

If you had two arrays, with 85 LUNs presented from each, again via two storage processors, you may see something like this:

– from the first HBA to first storage processor on array 1: c0t0d0 … c0t0d85
– from the first HBA to second storage processor on array 1: c0t1d0 … c0t1d85
– from the first HBA to first storage processor on array 2: c0t2d0 … c0t2d85
– from the first HBA to second storage processor on array 2: c0t3d0 … c0t3d85

– from the second HBA to first storage processor on array 1: c1t0d0 … c1t0d85
– from the second HBA to second storage processor on array 1: c1t1d0 … c1t1d85
– from the second HBA to first storage processor on array 2: c1t2d0 … c1t2d85
– from the second HBA to second storage processor on array 2: c1t3d0 … c1t3d85

Now, t0, t1, t2 & t3 represent the storage processors, two from the first array and two from the second array. Each target number will represent a discovered storage processor from the HBA/controller.

In this case, LUN 0 on the first array would have 4 paths:


And LUN 0 on the second array would also have 4 paths:


You can have different storage processors represented by the same target number, but they would be on different controllers/HBAs. For example, if HBA c0 was only mapped to the first array and HBA c1 was only mapped to the second array, you may see paths like this:

– from the first HBA to first storage processor on array 1:        c0t0d0 … c0t0d85
– from the first HBA to second storage processor on array 1:     c0t1d0 … c0t1d85

– from the second HBA to first storage processor on array 2:     c1t0d0 … c1t0d85
– from the second HBA to second storage processor on array 2: c1t1d0 … c1t1d85

Now we have two instance of t0, and two instance of t1, but the target numbers now represent completely different storage processors. And this is allowable since they are on unique controller ids, c0 & c1.

Lastly, I say theoretical above as there is no way of guaranteeing which storage processor gets assigned a target number. My understanding is that it is based on order of discovery, and if changes are made to the fabric/network, the discovery order and thus target numbers could change on the next reboot. This doesn't matter as we no longer rely on target numbers for accessing LUNs (like we did in the old days). I'm guessing this is why this doesn't come up in conversation too much these days.

Get notification of these blogs postings and more VMware Storage information by following me on Twitter: Twitter VMwareStorage

VMware Update Manager Survey

As VMware looks to improve or updates its solutions we continually look to understand our customer use cases and capture their feedback. VMware Update Manager is seen as a critical component for maintaining virtual infratstructures and we ask if you could spend a couple of minutes providing us feedback on how you use VMware Update Manager today by filling out this short survey