VMware vSphere Blog

You have 2 days until our deadline. Get those entries in for your chance at $100.

VMware vShield Zones, a new security service for vSphere, ensures strict compliance with security policies and industry regulations for user data as customers adopt cloud computing with virtual environments for increased efficiency and flexibility. Previously, compliance required diverting network traffic to external physical appliances resulting in disconnected ‘islands’ of infrastructure.  With VMware vShield Zones, customers can now create logical zones in the virtual datacenter that span all of the shared physical resources, and each zone represents a distinct level of trust and confidentiality.  This allows businesses to comply with corporate security policies and regulations on data privacy while still running applications efficiently on shared computing resource pools.

Traditional security products, such as firewall appliances, require that all network activity pass through a handful of fixed physical locations in order to be monitored.   Virtualized applications, in contrast can be  migrated between physical hosts for higher resource efficiency and improved uptime. Therefore, companies virtualizing security sensitive applications faced the choice of either leveraging virtualization capabilities such as live migration for optimal load balancing and availability, or enforcing strict security compliance. To solve that dilemma, most customers ended up dividing their virtual environments into smaller, less efficient clusters for areas such as their Internet-facing demilitarized zones (DMZ’s) or consumer credit data processing systems subject to Payment Card Industry regulations.  VMware vShield Zones enables customers to create security zones within enterprises or in multi-tenant cloud infrastructures, where security policies are enforced even as virtual machines dynamically migrate between hardware devices. Deployed as a virtual appliance and integrated into vCenter Server, vShield Zones makes it easy to centrally manage and enforce compliance with security policies across large pools of servers and virtual machines. Built-in auditing capabilities make compliance straightforward and verifiable.

As an example, today you send network traffic to an external Network IDS/IPS box which becomes a chokepoint.  With this feature all that traffic can be handled internal to the virtual infrastructure.  Similarly, there is also the capability for packet/protocol monitoring to be on the alert for SQL insertion or other data oriented attacks.  By combining multiple layers of the security “onion” within the virtual infrastructure you can more easily pass security and compliance audits will eliminating much of the costs associated with these activities.

Learn more about vShield Zoners at  http://www.vmware.com/products/vshield-zones/overview.html

Congratulations to Scott Sauer. He is the winner of our fourth cycle of the vSphere blog contest.

His entry is shown below and can also be found at:

http://www.virtualinsanity.com/index.php/2009/10/12/get-thin-provisioning-working-for-you-in-vsphere/

Get Thin Provisioning working for you in vSphere

October 12th, 2009 | Author: Scott Sauer 

Goto comments Leave a comment 

Going Thin and not looking back.

Yes, I am slowly losing my hair like 
many other aging men out there, but
it wouldn’t be virtual insanity if I were 
blogging about my personal male
pattern baldness issues. With the latest release of 
VMware vSphere comes a lot of new features and functionality that can be leveraged to make our lives easier. One of these features, that I personally have been looking forward to for a while, is Thin Provisioning. If you aren’t familiar with this technology, jump over to

Gestalt IT for a great explanation of what it is and how it works.

One of the exciting promises of thin provisioning, is getting more “bang for your buck” out of the expensive enterprise storage you have been investing in for your ESX environment. But, as Bret Michael’s once said, “Every rose has its thorn” and there are some things to look out for and considerations to make, before implementing thin disk technologies.

Efficiencies are great if they work right and don’t over 

complicate the environment.

Do your homework and make sure you understand the characteristics of the virtual machine that you are considering migrating into a thin disk configuration. The last thing you want to do is convert every VM to thin disk, and four months down the road all of your data stores are filling up and you’re scrambling for a storage CAPEX. Some people are of the opinion to do thin provisioning either on the host side (VMware) or on the storage array side, but not both. Take a gander at Chad Sakac’s blog that discusses thin on thin and some thoughts around each of these approaches. I’m not going to go into all of the pluses and minuses of thin provisioning but rather focus on how to make it work for you.

Coffee Talk

So now that we have some of the basics out of the way, I wanted to share my thoughts on thin provisioning. Like many organizations, we get requests from our customers that err on the side of caution. They want to plan for the worse case and ensure that their project and/or application isn’t setup for failure. I don’t blame them really, I do it myself all the time when I make coffee at home. I always end up making more coffee than I typically drink, just in case I might need that extra charge. The best way to do that is pad it, request more than what you might really need, just in case something comes up down the road. Virtual machine disk storage in some cases fits this same profile. If my coffee maker granted me access to hot coffee on demand, I would stop making extra coffee. Thin disks can give your end users that capacity on demand so you can gain control of the padding effect that typically takes place in most corporate organizations.

Take it back…

So now you have done your research, you’re starting to get a feel for what this thin stuff is and how it might play out in your shop. It’s go time. If you’re a smaller VMware customer, you probably already have an idea of what are good target disks to convert. If you’re a larger environment, it might be a little more difficult to gauge where the bloated pigs are hiding.

I worked at GE for a couple of years and was exposed to some of the Six Sigma methodologies they preach as well as practice. Sounds boring, right? Not really. You can really leverage DMAIC for a lot of IT related problems/issues/projects. You don’t have to take it to the extreme, use the framework to help guide you on your quest:

DMAIC

The DMAIC project methodology has five phases:

Define high-level project goals and the current process. 

Measure key aspects of the current process and collect relevant data. 

Analyze the data to verify cause-and-effect relationships. Determine what the relationships are and attempt to ensure that all factors have been considered. 

Improve or optimize the process based upon data analysis using techniques like Design of experiments. 

Control to ensure that any deviations from target are corrected before they result in defects. Set up pilot runs to establish process capability, move on to production, set up control mechanisms and continuously monitor the process. 

We have already defined our project goals and what we are trying to accomplish. We need a good “Measure” tool to really find where we might benefit from thin provisioning. Powershell is a great tool that most VMware administrators use, or have at least heard of. So this was the first place I turned to for assistance.

Alan Renouf of “Virtu-AL” http://www.virtu-al.net/ gave me a hand in writing the powershell script needed. (Thanks again, Alan!). Alan already had a one liner script to produce a list of vm’s, their disks assigned, and how much data each disk was consuming. I needed the ability to see this data outside a powershell window and be able to analyze it in a better format. We have a decent-sized VMware environment and exporting this out to a .csv for analysis is extremely helpful. Here is the script!

************************************************************************

# Set the Filename for the exported data
$Filename = "C:\VMDisks.csv"

Connect-VIServer MYVIServer

$AllVMs = Get-View -ViewType VirtualMachine
$SortedVMs = $AllVMs | Select *, @{N="NumDisks";E={@($_.Guest.Disk.Length)}} | Sort NumDisks -Descending

$VMDisks = @()
ForEach ($VM in $SortedVMs){
$Details = New-object PSObject
$Details | Add-Member -Name Name -Value $VM.name -Membertype NoteProperty
$DiskNum = 0
Foreach ($disk in $VM.Guest.Disk){
$Details | Add-Member -Name "Disk$($DiskNum)path" -MemberType NoteProperty -Value $Disk.DiskPath
$Details | Add-Member -Name "Disk$($DiskNum)Capacity(MB)" -MemberType NoteProperty -Value ([math]::Round($disk.Capacity/ 1MB))
$Details | Add-Member -Name "Disk$($DiskNum)FreeSpace(MB)" -MemberType NoteProperty -Value ([math]::Round($disk.FreeSpace / 1MB))
$DiskNum++
}
$VMDisks += $Details
Remove-Variable Details
}
$VMDisks | Export-Csv -NoTypeInformation $Filename

***********************************************************************

So now that you have this great spreadsheet, you can do all sorts of crazy sorting and reporting, within Excel. Take some time on phase 3, “Analyze” what you’re seeing. Talk to your VM stakeholders to see how things might be changing from their perspective. Try to plan for the surprises and position yourself accordingly.

Next is the “Improve” phase of DMAIC (see it’s easy!). This is the part where you actually do the work. It’s time to start leveraging the storage VMotion API’s, and reclaim some of that unused disk.

1. Select the target VM in the VC client. 
2. Right click on the VM and select the option “Migrate”. 
3. Select the option “Change Datastore”. 
4. Select the destination, or click advanced if you are targeting one particular disk. 
5. Select “Thin provisioned format”. 
6. Select Finish. 

Rinse and Repeat for the rest of that spreadsheet you have worked so hard on.

The last phase of DMAIC is “Control”. This is one of the most important pieces to thin provisioning in my opinion. At the minimum you need to setup Virtual Center alerts to monitor when your datastores are approaching critical levels. You can’t implement thin disks in your vSphere environment and walk away. The smart people over at VMware have given us the ability to monitor datastore disk space usage and over-allocation with the latest release of Virtual Center. Setup your monitors so you are e-mailed when some of these thin disks begin to grow and you need to take some action.

Eric Gray of VMware takes this to the next level, check out his blog post on utilizing powershell to prevent datastore emergencies. My personal approach to this concept is to setup a “hotspare” datastore for your environment. A good practice to implement here would be to try reclaiming enough storage from your migrations to thin disks to free-up a “hot spare datastore”. Implementing an automated recovery solution like Eric’s will help you sleep easier at night. Worried about what might happen if your script doesn’t work or you do hit the perfect storm and end up with a full VMFS volume? Intelligence has been built into vSphere to automatically pause the virtual machines, impressive. Check out Eric’s video:

Wrapping it all up

Thin disk provisioning is a great feature that you should consider leveraging in your environment. With some forward thinking and best practices you can achieve higher ROI for your ESX storage. VMware vSphere offers the ability for you to migrate from thick to think with no downtime, so you can begin reclaiming storage on the fly. Keep it simple, start out with a high level analysis of your infrastructure. Identify the candidates that are a good fit and worth focusing on. Setup your alerts on the datastores as soon as you migrate your first virtual machine so you are protecting yourself from problems down the road. Consider taking automated actions if your datastores are reaching critical thresholds.

I hope you found this article helpful, good luck!

Scott Sauer

Cycle 6 of the contest (starting 11/30 and ending 12/11) will now focus on the Nexus 1000V product for use with vSphere rather than ESXi. 

This week starts the 5th cycle of our vSphere Blog Contest. Over the next two weeks we will be highlighting VMware vShield Zones (http://www.vmware.com/products/vshield-zones/). vShield Zones simplifies application security by enforcing corporate security policies at the application level in a shared environment, while still maintaining trust and network segmentation of users and sensitive data.

Get those blog entries going and enter by next Friday (11/20) for your chance to win!

Thanks for your blog entries so far on vStorage Thin Provisioning. For those of you yet to post, you have until 11:59 this Friday to get your entry in for this cycle. Below are our entries to date (nice work folks!):

1. Rich Bramley

http://vmetc.com/2009/11/03/provision-a-thin-provisioned-standby-lun-for-vsphere-thin-provisioning/
Rich brings together a few posts on the topic and relays a number of the key areas of focus for this features (using this in conjunction with storage layer thin provisioning, use cases, etc.).

2.Dwayne Lessner

http://www.itbloodpressure.com/index.php?option=com_content&view=article&id=52:living-thin-in-a-fat-world&catid=1:latest-news&Itemid=50
Dwayne discusses how to get started and then weighs the pros and cons of the feature.

3. Vaughn Stewart (Entry 1)

http://blogs.netapp.com/virtualstorageguy/2009/10/vce-101-thin-provisioning-part-1-the-basics.html
Vaughn's two part entry (see below) lays out the basics and then gets into the tie with Network Appliance hardware. A must read for users that have VMware running on Network Appliance gear. 

4. Vaughn Stewart (Entry 2)

http://blogs.netapp.com/virtualstorageguy/2009/10/vce-101-thin-provisioning-part-2-going-beyond.html

5. Scott Sauer

http://bit.ly/88EQg
Scott uses a coffee analogy to discuss Thin Provisioning and layers in a further discussion using principles from Six Sigma.

Don't miss your chance at our $100 prize!

Let's start week 2 of Thin Provisioning off with a bang! 

Did you know the new Thin Provisioning feature is available with all vSphere 4.0 editions (standard, advanced, enterprise, and enterprise plus)?

See what our engineers have to say about Thin Provisioning.

Increase Storage Utilization

Eliminate the need to dedicate full capacity upfront while still providing application users with the capacity they need for future growth. VMware vStorage Thin Provisioning lets you subscribe more capacity to virtual machines than they actually have, eliminating the waste of resources and space caused by unused over-allocated storage. With VMware vStorage Thin Provisioning storage administrators can increase their storage utilization by letting them dedicate more storage than the actual capacity.

Enhance Application Uptime for Improved Business Continuity

Eliminate application downtime by simplifying storage provisioning. Managing storage allocations to support dynamic environments can be a time-consuming process that requires extensive coordination between application owners, virtual machine owners and storage administrators, often resulting in downtime for critical applications.

Furthermore, delay during the process of storage allocation at any layer, storage to application can result in prolonged application downtime. By eliminating the need to periodically provision more capacity, VMware vStorage Thin Provisioning eliminates application downtime.

Simplify Storage Capacity Management

Let your application users proactively manage storage capacity transparent to the storage administrators and eliminate the manual processes requiring careful planning and coordination by IT management, storage administrators, system administrators, and application administrators.  In addition, VMware vSphere provides a single management point to set alarms and alerts required to safely thin provision storage to virtual machines.

Get a single unified tool for multiple storage or non-intelligent storage to thinly provision and eliminate the need to provisioning storage frequently. vStorage Thin Provisioning is a powerful storage enabling technology which streamlines capacity management for the storage and server teams.

One piece that may not be all that clear for VMware users is that Storage VMotion (now available as a feature within our GUI and not just in the command line with vSphere 4.0) allows for an easy transition from previously thick provisioned virtual disks to new thin provisioning virtual disks. So, any user that upgrades to vSphere can now use this function to save up to 50% in terms of storage allocated in a virtual disk. Another product of this technology is the move to a thin provisioned virtual disk will also defrag the disk.

We are looking for beta participants to test out some new vSphere features currently in development. Interested? If so, contact your VMware account team for further details.