VMware Studio 2.1 GA was announced on Tuesday, July 13, 2010 in conjunction with the vSphere 4.1 release. The 2.1 release extends support for various OS and also brings enhanced flexibility, predictability and security to the Virtual Appliance development and update process.

• We listened to our customers and unlike previous versions, VMware Studio 2.1 can now create virtual appliances from virtual machines that were not originally created with VMware Studio, based on a discovery phase. In addition, while building an appliance, EULA(s) can now be retrieved from an input file in additional to being embedded in the build profile.

• Studio now enables users to create their own build profile templates. The CLI offers the --newos option using which you could generate a build profile for any RPM-based or DEB-based Linux OS.

• We also took a stride towards securing the appliance deployment and update process. We added support for digitally signed OVF files so that VMware vSphere 4.1 can verify the signed certificate during OVF import. In addition, using signed manifests during the update process, you could ensure that patches are coming from a trusted source.

• We also added a feature around localization. You can translate your EULA(s) into multiple languages, and vSphere Client as well as the VM first boot script will display it in the language of the prevailing locale. Also, you can now enforce EULA acceptance during the appliance update process.

• On the performance and optimization side, with Studio 2.1, you could now run concurrent builds. After a configurable limit is reached, builds wait in a queue. In addition, VMware Studio 2.1 optionally analyzes the list of RPM and DEB packages to locate unused items and generate a small-footprint virtual machine. It can also reduce the footprint post-installation with a file removal list.

• With Studio 2.1, during the appliance provisioning process, the generated virtual machine always reboots before your application installs. Some applications expect a full installed working system before they themselves will install. Rebooting after OS installation provides the real environment in which to install the application.

• A new utility program “vamisupport” is shipped with the output appliances for retrieving useful data for troubleshooting VAMI related issues.

• The new Linux OSs support we added are SLES 11, CentOS 5.4, RHEL 5.4, and Ubuntu 8.04.3 and 8.04.4.

In addition to the new features mentioned above, we fixed some of the major issues our customers faced with Studio 2.0. The issues around packages installation order (as specified in the Application Package Repositories) not always being honored has been fixed. Disk capacity set in build profile is now properly displayed in the web interface. VAMI is now fully functioning in SELinux enabled systems. Finally, OVF Tool used for transporting VM during provisioning has been upgraded to address known issues.

Download VMware Studio 2.1 GA release from here and give it a spin. We look forward to your feedback.

Updated: To work with the latest version of the Zimbra virtual appliance (Sep. 6). 

The Zimbra folks have been so nice to create a virtual appliance that contains the entire Zimbra Collaboration Suite. The Zimbra virtual  appliance is packaged up in the OVF format and is using several advanced vSphere features (such as OVF properties) to  simplify the entire installation. In fact, if you have vSphere 4, it is a  really simple process to get Zimbra up and running.

But what if you don't have access to vSphere 4? (or just want to try it out on your laptop while eating lunch?)

In this post, I will show how you can use the freely available tools, OVF Tool 2.0.1 and  vApprun 1.0 to deploy, configure, and run the Zimbra appliance on VMware Fusion or VMware Workstation. Basically, we are tricking Workstation/Fusion to think it has all the nice OVF capabilities from the full-fledged vSphere product.

So before we start the pre-requisites are:

• An installed version of VMware Fusion 3 or VMware Wokstation 7
• VMware OVF Tool 2.0.1 (download)   
• VMware vApprun 1.0 (download)      

Make sure that both OVF Tool and the vApprun command is in your path.

Step 1: Create a vApprun workspace

The vApprun tool implements a complete OVF / vApp environment for Workstation and Fusion. To use vApprun, we need to setup a vApprun workspace:

Make sure to substitute the values shown for DNS, gateway, netmask, and range with values that work in your environment.

Note 1: The above command should be executed as a single line. It has only been broken into multipe lines, so it is easier to read. The quotes ("") are only needed on Windows.

Note 2: If you are lazy and are on a network with a DHCP server, you can skip this step! (see step 5)

Step 3: Download the Zimbra Virtual Appliance

To download the Zimbra Appliance, open a web browser and go to: http://www.zimbra.com/products/download_appliance.html

Step 4: Use OVF Tool to deploy to vApprun workspace

Now it is time to download and deploy the ZImbra Appliance into our vApprun workspace.

>ovftool "--name=zimbra" <<Paste URL from above here>> .
Opening OVF source: http://files2.zimbra.com/.../ovf/zimbra.ovf
Warning: No manifest file
Opening vapprun target: .
Info:    End User License Agreement
License:
     VMWARE END USER LICENSE AGREEMENT

     IMPORTANT-READ CAREFULLY: BY DOWNLOADING, INSTALLING, OR USING THE
     SOFTWARE, YOU (THE INDIVIDUAL OR LEGAL ENTITY) AGREE TO BE BOUND BY THE
     TERMS OF THIS END USER LICENSE AGREEMENT (EULA). IF YOU DO NOT AGREE
     TO THE TERMS OF THIS EULA, YOU MUST NOT DOWNLOAD, INSTALL, OR USE THE
     SOFTWARE, AND YOU MUST DELETE OR RETURN THE UNUSED SOFTWARE TO THE
     VENDOR FROM WHICH YOU ACQUIRED IT WITHIN THIRTY (30) DAYS AND REQUEST A
     REFUND OF THE LICENSE FEE, IF ANY, THAT YOU PAID FOR THE SOFTWARE.

     ... abbreviated ...
Accept end-user license agreement?
Write 'yes' or 'no' (write 'read' to reread the EULA): yes
Deploying to vapprun workspace at: c:\Temp\zimbra\zimbra
Disk Transfer Completed
Completed successfully

Make sure to replace the URL in the above command with the one you got in Step 3. 

Notice that we just use a dot (.) as the target location. We can do that since the current directory is the vapprun workspace directory. 

Let's check that we got a Zimbra VM in our vApprun inventory:

>vapprun list zimbra
Name.......: zimbra
Type.......: VM
VMX........: .../zimbra/vmx/vm.vmx
Tag........: Zimbra_Collaboration_Suite_Appliance
AppUrl.....: http://${vami.ip0.Zimbra_Collaboration_Suite_Appliance}/
Transport..: com.vmware.guestInfo
IP Policy..: fixed

Property Definitions:
 Key            Type      Value           UserConfig
 -------------------------------------------------------------------------------
 Zimbra_CONFIG_PASS    string(6..20)               True
 Zimbra_HOSTNAME      string(1..65535)              True
 Zimbra_ADMIN_EMAIL    string(1..65535)              True
 Zimbra_ADMIN_PASS     string(6..20)               True
 Zimbra_DISABLE_FW     boolean     False           False
 Zimbra_DISABLE_NETWORK_TAB boolean    False           False
 Zimbra_SERVICES      string     ALL            False
 Zimbra_LDAP_MASTER_IP   string     @@LDAP_MASTER_IP@@     False
 Zimbra_LDAP_MASTER_URL  string     @@LDAP_MASTER_URL@@    False
 Zimbra_LDAP_ROOT_PASSWORD string     @@LDAP_ROOT_PASSWORD@@   False
 Zimbra_ZIMBRA_LDAP_PASSWORD string    @@ZIMBRA_LDAP_PASSWORD@@  False
 Zimbra_VAMI_USER     string     @@ZIMBRA_VAMI_USER@@    False
 Zimbra_VAMI_PASS     string     @@ZIMBRA_VAMI_PASS@@    False
 Zimbra_MTA_AUTH_HOST   string     @@MTA_AUTH_HOST@@     False
 Zimbra_HYPERIC_HQ_HOST  string     @@HYPERIC_HQ_HOST@@    False
 Zimbra_HYPERIC_HQ_USER  string     @@HYPERIC_HQ_USER@@    False
 Zimbra_HYPERIC_HQ_PASS  string     @@HYPERIC_HQ_PASS@@    False
 vami.DNS0.Zimbra_Collaboration_Suite_Appliance expression   ${dns:Network}    False
 vami.gateway0.Zimbra_Collaboration_Suite_Appliance expression ${gateway:Network}  False
 vami.netmask0.Zimbra_Collaboration_Suite_Appliance expression ${netmask:Network}  False
 vami.ip0.Zimbra_Collaboration_Suite_Appliance ip:Network              True
 vm.vmname         string     Zimbra_Collaboration_Suite_Appliance   False

Property Settings:
 Key            Value
 ------------------------------------------------
 Zimbra_HOSTNAME      CHANGE-ME
 vami.ip0.Zimbra_Collaboration_Suite_Appliance
 Zimbra_ADMIN_PASS     CHANGE-ME
 Zimbra_ADMIN_EMAIL    CHANGE-ME
 Zimbra_CONFIG_PASS    CHANGE-ME

... abbreviated ... 

Once you have done the above steps a few times, it actually becomes quite simple. Finally, all of the above works equally well on Mac OS X, Windows, and Linux.

We are happy to announce that OVF Tool 2.0.1 was released yesterday. It can be downloaded from here, and is available for Windows, Linux, and Mac OS X (new!). 

This is the first major release since the initial release of OVF Tool 1.0 and a minor upgrade to the OVF Tool 2.0 version that is bundled with Fusion 3.1 and Workstation 7.1. I have included an excerpt from the README below which describes the changes:

Changes since 2.0.0
-------------------
. Fixed bug that caused OVF Tool not to work with the "Import from vSphere"
feature on the VMware Virtual Appliance Marketplace.
. Fixed launch issue on Linux/Mac if installed in a location where the path
includes spaces.
. Fixed issue with properties when importing to vApprun workspace.

Changes since 1.0.1
-------------------
. Support for the OVF 1.1 specification.
. Support for OVF packages that includes ISO and floppy images.
. Support for vApprun as target and source.
  (see http://labs.vmware.com/flings/vapprun)
. Support for OVF Tool on the Mac OS X operating system (Intel-only).
. A lax mode (--lax) that enables a best-effort conversion of OVF
packages which do not fully conform to the OVF standard or include
virtual hardware that is not supported by the destination. For example,
VirtualBox (version 3.0 and 3.1) produced OVF packages can be converted in lax mode.
. Bug fix for handling vmxnet3 targets.

Please use the forum on the download page to post any comments or issues with the tool. We would like to hear to hear from you.

/Rene

1. An absolute path (on VMware Studio) to a version 1.0 OVF or an OVA containing a version 1.0 OVF,
2. A URL to a version 1.0 OVF or an OVA containing a version 1.0 OVF. In this URL, the keyword [VADK.localIP] may be used, and will be substituted with the actual IP address of the VMware Studio machine.

In has been a while since the last update and a lot of exciting things has happens with regards to both OVFs and vApps in the meantime.

In the real code department:

I am very happy to annouce that  we just released our latest OVF-related fling on labs.vmware.com: vApprun 1.0. This is a free, command-line tool that implements vApps and, OVF properties, and OVF environments for both Workstation and Fusion users. Thus, vApps are no longer just for vSphere 4. From the vApprun release notes:

A vApp is a container for a distributed, multi-VM software solution. It has power-on operations just as a virtual machine, and can be imported or exported as an OVF package. The OVF properties and OVF environment provides a flexible mechanism for parameterizing guest software inside a VM upon deployment. Using the OVF properties, it is possible to both configure OS level properties, such as IP settings, and application level parameters, such as IP addresses of external servers. The features of vApprun include:

• vApps that contain multiple VMs and nested vApps.
  
• For a vApp, the start/stop sequence of the child entities can be configured.
  
• Start/stop/shutdown of vApps.
  
• Create OVF properties and access the OVF environment from guest software.
  
• Supports ISO and guestInfo OVF environment transports.
  
• Supports fixed, transient, and DHCP IP allocation modes and IP pools.
  
• Import OVF packages and run it with vApprun (using OVF Tool 2.0).
  
• Export vApps and VMs created with vApprun to OVF packages (using OVF Tool 2.0).
  
• Command-line, workspace-oriented user interaction.
  
• Cross-platform - available for Windows/Mac/Linux.
  
• VMware vSphere 4 compatible.
  

As a teaser, here is how to create a vApp with 2 VMs and a property:

>vapprun init
>vapprun create-vapp MyVApp
>vapprun create-vm VM1
>vapprun edit VM1 parent=MyVApp
>vapprun create-vm VM2
>vapprun edit VM2 parent=MyVApp
>vapprun def-property MyVApp key=ip type=ip:Network
>vapprun list MyVApp
Name.......: MyVApp
Type.......: vApp
IP Policy..: fixed

Children:
 Name    Order StartWait WaitForTools StopWait
 ------------------------------------------------------------
 VM2    30  30   True   30
 VM1    30  30   True   30

Property Definitions:
 Key      Type   Value      UserConfig
 -------------------------------------------------------------------------------
 ip      ip:Network         True

Property Settings:
 Key      Value
 ------------------------------------------------
 ip

You can download it here: vApprun 1.0 and start playing with vApps today. We will soon follow up with an more in-depth look at vApprun on this blog.

In the specification department:

The OVF  1.1 specification was released in January. This is a minor, fully backwards compatible update to the 1.0 specification. From the DMTF newsletter:

There are several new components that differentiate the OVF 1.1 specification from the original version, including:

• Capability for file system-based images to increase flexibility at deployment time
  
• A property attribute to hide password values at the user interface
  
• Joliet extensions for ISO transport image
  
• Clarification of how the OVF manifest contain entries for individual file chunks
  
• Clarification of the referencing of manifest and certificate files from descriptor
  

In even more exciting news, the OVF 1.1 specification has been submitted to ICITS to become an ANSI standard, which is the first step to become an ISO standard. From the DMTF site:

DMTF has been busy extending and refining the Open Virtualization Format (OVF) specification. The organization recently completed version 1.1 of the spec and has submitted it for consideration as an ANSI and ISO standard.

This is an important milestone for DMTF. The specification will be put through a number of tests to ensure it meets a complex system of checks and balances of reviews before it is adopted as an American and International standard.

That is all the news for this time. Take care,

/Rene

Introduction to appliances and updates

VMware Studio builds VMs. As a part of VM building, VMware Studio 2.0 embeds an in-guest agent also known as VAMI (Virtual Appliance Management Infrastructure) for Linux VMs, this in-guest agent allows users to configure system parameters, example: timezone, configure networking parameters, example: specifying ip addresses, and optionally update the VM.

A VM with update agent installed/enabled is considered to be a virtual appliance, and the user of the appliance can update the appliance with a single-click. The update model provided by VMware Studio for appliances is a pseudo-firmware model, where a single update will update all the components (application packages) in the virtual appliance. The update includes both OS updates, and updates for the application, this ensures that an update applied to the appliance doesn’t break the application.

Appliance builder can publish updates easily using Studio 2.0, updates are published to an update repository as a part of an appliance build. An update consists of a set of packages that are in native package formats (i.e. either rpms or debs based on the Linux flavor), as a result updates can be tracked using existing package managers, and as an appliance builder updates for non-application packages (OS updates) are easier to find.

Publishing Updates for your Appliance

Updates are published during build

During the creation of a build profile to build a VM, the appliance author specifies the list of application packages and OS packages that are to be installed for the appliance.

When Studio builds the VM, it can determine all the packages that were installed on the appliance, Studio uses this information to create an update repository as a part of the build. The update repository contains a manifest file which lists all the packages that were installed and a pool of packages. This update repository can then be used for testing updates and once tested it can be published externally, so that the appliances in the field can get their updates.

Enabling updates in Studio UI

While creating the build profile for your appliance you can add the update management service to enable updates for your appliance, once this service is enabled the build profile wizard will ask you to enter additional details.

Note: To publish to the same update repository the appliance version needs to be incremented.

a) Repository Access Methods

The appliance will use either a CD-ROM or will connect to the specified URL to obtain updates. Later on, the appliance user can also point to a local (alternate) repository to obtain updates, this can be configured by the appliance user using VAMI web console.

b) Repository Server Settings

As a part of the build, Studio will create an update repository at the specified location using the SCP information entered. Repository server is usually the staging server (see next section for more information) for the appliance updates.

c) Repository Export Settings

If you’d like to save a copy of the update repository in either zip format, or create a CD-ROM based repository that can be used to update the appliance it can be specified here.

Please refer to Studio Developer's Guide documentation for more information.

Staging Server

A staging server is usually a location within the organization. This location is used by Studio to publish the updates during appliance build. This repository can also be used to test the update before releasing the update.

Advanced Option - Update scripts

Studio provides ability for the author of the appliance to specify scripts that can be run before or after installation of packages (i.e. rpms or debs). A good example of such a script would be: deletion of a package which already is present in the VM and would cause conflict with a package in the update.

To edit these scripts, open the build profile using an XML editor (located at /opt/vmware/var/lib/build/profiles directory in the Studio VM), and change the default content of <vadk:PreInstallShellScript> and/or  <vadk:PostInstallShellScript> under section “vadk:UpdateSection_Type”.

Note: Scripts added will have to be XML encoded to ensure that it is valid XML.

Updating your Appliance

The appliance can be updated by the appliance user using either the VAMI Web Console, CLI on the appliance, or using vSphere Client/vCenter.

Web Console

Using the VAMI web console, the appliance user can check for updates, update the appliance, schedule automatic updates, and specify alternate methods of obtaining updates.

If updates are being received from an external repository then the user simply clicks on the Check button to query if any updates are present, and if there are any updates present user clicks on the Install button to update the appliance.

If an update is received as a CD-ROM ISO file, then the user can attach the ISO file to the CD-ROM device of the appliance, and run check/install operations as a user would for an external repository.

CLI

Studio in addition provides vamicli, a command line tool on the appliance, using which the appliance can be updated.

Please refer to Studio User’s Guide for more information.

Testing Updates

It is recommended that the author of the appliance, test updates before releasing the update to the appliance users. In order to test the appliance which is built with an external repository URL, the author can deploy the appliance inside the organization, and change the repository URL to the staging server URL or use a CD-ROM based update using the VAMI web console.

Integration with vCenter for centralized update management

VMware vCenter Update Manager a vCenter plugin allows users of appliances (built with VMware Studio) to update their appliances. The remediation process during which the appliance is update can either be started manually or as a scheduled task. With Update Manager multiple appliances can be updated at the same time, when an appliance boots up VMware Update Manager automatically discovers that the VM has VMware Studio’s update agent in the VM and allows users to scan and remediate appliances. 

Users can also define baselines. Baselines are rules that the administrator of an appliance can define using Update manager, an example would be: an appliance must always be at the latest version. Using baselines the administrator can check appliances for compliance.

More information on how to use Update Manager can be found in the Studio User’s Guide.

Have fun publishing updates for your appliances!

There are many steps in developing, building and testing VMs and vApps using VMware Studio.  To make it easier Studio 2.0 adds a new tool to help, the VMware Studio Eclipse plugin. The plugin makes life simpler to many users by adding simple ways to package applications and build the application along with the VM or vApp, all in a single click.  To demonstrate the simplicity, in this blog entry, I will walk through an example of how to develop a simple web-based VM or vApp.

To follow along you will already need to have VMware Studio running and will need to set up an Eclipse development environment.

To get started I downloaded the Eclipse IDE distributed by Aptanu at http://www.aptana.org/ and installed the optional Aptana Aflax feature by going to Help->Install Aptana Features then selecting Ajax Libraries->Aflax and then clicking install.  You can also select the editing tools that you want to use for your app.

I then installed the VMware Studio Eclipse Plugin by following the guide at http://www.vmware.com/support/developer/studio/studio20/eclipse_plugin.html.



Now the normal application development tasks take place, I will just use a sample project that is provided by Aptana as my application.  If the Samples view is not open in your environment, then you can open it by selecting Window->Show View->Other...->Aptana Views->Samples then clicking OK.  Once the view is open, I will find the sample named spirograph, right click on it and select Import Sample as Project...



Once the project is opened I used the Project Explorer view to create a new directory named html and moved all the web files into it.  I then created a VMware Studio Package for Linux (.vsp) in the project and called it mypackage.  To created the vsp file, right-click on the project name and select New->Other...->VMware Studio->VMware Studio Package for Linux .

You can use the Eclipse tools to further test and develop this web app.  Open the index.html file and modify the source or preview the changes.

The 'item' Elements

Virtual hardware is modeled as a set of devices, such as ethernet card or disk controllers, memory. Each of those devices are described by an element. The  DMTF RASD specifies a set of fields that can be set. We only a subset of those fields in an OVF descriptor. The fields in a RASD must be ordered alphabetically. The following table lists the RASD fields that are used:

Here is a conceptual figure of the virtual disks in a delta disk compressed OVF package and how it looks when it is deployed:

The OVF package contains two VMs: A Web server and a database. They run the same operating system and delta disk compression has factored out the common parts in a separate parent disk shared by the two VMs. When the OVF package is deployed the tree gets flattened and the Web server and database each get their own copy of the operating system in the parent disk.

In this blog post you will learn all about how to design your OVF package to take advantage of delta disks and how to apply this type of compression to your package using OVF Tool.

• We start out by looking at a brief example of how a typical OVF package with multiple disks could look like and how delta disk compression would reduce the size of it.
• Next we look at what delta disk hierarchies are and how they are expressed in the OVF descriptor. This is important to understand what they are to make delta disk compression work.
• Then we give some advice on how you should construct your OVF package to utilize delta disk compression.
  • Here we also give some tips on how to shrink your virtual disks to reduce the space of your OVF package.
• Finally, we show how OVF Tool can delta disk compress your OVF package.

Example

Let us assume the two VMs run the same Linux OS (for example Ubuntu Server 9). Then much of the data on the two disks would be identical and only the bits concerning the Apache HTTP Server, PHP software, and MySQL would be different. Here is a rough estimate of how much space each component will need when stored on a compressed virtual disk:

• Ubuntu Server 9: 500 MB.
• Apache HTTP Server and PHP: 50 MB
• MySQL: 50 MB

SizeOf(Web server) + SizeOf(Database) = (500 MB + 50 MB) + (500 MB + 50 MB) = 1,100 MB

In this blog post we will explain how this space can be reduced using the delta disk feature supported by the OVF specification and OVF Tool. Using delta disk compression we can extract all the components that are equal in the two VMs (the Linux OS part), only keeping one copy of them. This leaves us with an OVF package that only take up about 600 MB of space.

Technical Details of Delta Disks

In the figure parentRefs annotate the arrows that tie the disks together. This is also what the attribute is called in the OVF descriptor which link Disk elements together and it is used on Disk elements in the DiskSection of the OVF descriptor. This is what the disk section with the three disks could look like:

<DiskSection>
 <Info>Meta-information about the virtual disks</Info>
 <Disk ovf:capacity="1073741824" 
       ovf:diskId="disk1" 
       ovf:fileRef="diskFile1" 
       ovf:format="http://www.vmware.com/interfaces/specifications/vmdk.html#streamOptimized" />
 <Disk ovf:capacity="1073741824" 
       ovf:diskId="disk2" 
       ovf:fileRef="diskFile2" 
      ovf:parentRef="disk1"
       ovf:format="http://www.vmware.com/interfaces/specifications/vmdk.html#streamOptimized"/>
 <Disk ovf:capacity="1073741824" 
       ovf:diskId="disk3" 
       ovf:fileRef="diskFile3" 
       ovf:parentRef="disk2"
       ovf:format="http://www.vmware.com/interfaces/specifications/vmdk.html#streamOptimized" />
</DiskSection>

The LAMP example can be described as this delta disk hierarchy:

For this setup the disk section could look like this:

<DiskSection>
 <Info>Meta-information about the virtual disks</Info>
 <Disk ovf:capacity="1073741824" 
       ovf:diskId="parentDisk" 
       ovf:fileRef="parentDiskFile" 
       ovf:format="http://www.vmware.com/interfaces/specifications/vmdk.html#streamOptimized" />
 <Disk ovf:capacity="1073741824" 
       ovf:diskId="WebServerDisk" 
       ovf:fileRef="WebServerDiskFile" 
      ovf:parentRef="parentDisk"
       ovf:format="http://www.vmware.com/interfaces/specifications/vmdk.html#streamOptimized"/>
 <Disk ovf:capacity="1073741824" 
       ovf:diskId="DataBaseDisk" 
       ovf:fileRef="DatabaseDiskFile" 
       ovf:parentRef="parentDisk"
       ovf:format="http://www.vmware.com/interfaces/specifications/vmdk.html#streamOptimized" />
</DiskSection>

Preparing an OVF Package for Delta Disk Compression

There are some restrictions of delta disk compression that are important to understand to get the most out of the feature. Firstly, the disks in a delta disk hierarchy must have the same capacity, so if you have two disks in your OVF package with different capacity (for example, one is 4 GB and the other 8 GB) you will not be able to use delta disk compression on the two disks. Secondly, delta disk compression only compares disk content on the same part of the disk at the disk address level. For example, even though the same file is on two different disks but not on the same part of the disk it will not be reduced by delta disk compression. If the file on the first disk is at address 0x00670000 and on the other disk at address 0x02D10000 it will not be detected as a shared block and put in a parent disk – only if it is at the same address (for example, 0x00670000). In other words, for delta disk compression to work there should be a substantial overlap between disks at the disk address level.

The second requirement can be difficult to satisfy if you are not careful in how you construct the OVF package, but there are ways to do it. To explain how, let us first look at the LAMP stack example that we looked at in the beginning of the blog post, to see how we can prepare it for delta disk compression. This LAMP stack had a Linux VM running Apache HTTP Server and PHP and another Linux VM running MySQL. Each VM had a single disk.

To achieve optimal disks for delta disk compression we first create a plain Linux VM with one disk. We clone the VM so we now have two plain Linux VMs. On one of them we install Apache HTTP Server and PHP and on the other we install MySQL. By doing this we satisfy the first criteria that the disks have same capacity (Web server VM’s disk and database VM’s disk come from the same original plain Linux VM) and second criteria that a significant part of the disks overlap each other. The files from the OS part of the plain Linux install are at the same position on both disks, since they were not changed when we installed the Apache HTTP Server, PHP, and MySQL (or at least, the majority of them have not changed).

The above example is rather canonical in how you achieve the best results from delta disk compression when having multiple VMs using the same operating system, so to summarize:

1. Install a plain operating system in a VM;
   
2. Clone the plain VM the number of times you need for your solution;
   
3. Install the remaining software specific to each VM.
   

The reason why we first install a plain operating system and then clone it to the number of VMs we need, rather than installing the same operating system multiple times on each of the VMs we need, is that we cannot in general be sure that the files are put at the exact same location on the disks, even though it is the same operating system.

If cloning is not an option when making the OVF package then perhaps VMware Studio is. It can create VMs well suited for delta disk compression, since it builds the VMs operating system and other software components in a scripted manner that that can be replayed to produce almost identical VMs.

Shrinking the Disks

When you export your VMs in your OVF package you want to make sure that all unused space is zeroed out, since this compresses really well in the VMDK disk format. However, space used by swap disks and deleted files often take up space on disk, since they are not eagerly zeroed out by default by most operating systems. This means that even though your VM says it only uses about 500 MB it may actually take up a lot more space. Even worse, you may have confidential information on, e.g., your swap drive or old deleted files that you do not want to distribute with the OVF package. There are several ways to solve this problem. On most Linux distributions it is possible to do the following things to clean up a disk before you export the VM: 1) Un-mount the swap drive; 2) Write a single file to disk containing only zeroes as large as possible; 3) Delete the file immediately after you created it. On the command line you can do these three steps by invoking these commands:

1. /sbin/swapoff -a (this will un-mount all swap disks)
2. dd if=/dev/zero of=zeroFile.tmp
3. rm zeroFile.tmp

On a Windows system it can be done in various ways. We will consider Windows Server 2008, but it can be applied with modifications on other types of Windows systems.

We start out by installing VMware tools on the Windows Server 2008 VM and when it is installed, open VMware tools and choose “Shrink…”. This will zero out the disk. To zero out the swap disk you need to set an option under Administrative Tools. Go to Administrative Tools -> Local Security Policy -> Security Settings -> Security Options and enable the policy “Shutdown: Clear virtual memory pagefile”. When you shutdown the VM the swap disk will then be zeroed out. Please note, however, that enabling this option will increase the shutdown time significantly for large swap disks. One way of working around this problem could be to first delete the swap disk, reboot the VM and disabling the option again (and hopefully no data is written to the swap disk), and then shutting the VM before putting it in an OVF package.

Creating an OVF Package with Delta Disk Compression using OVF Tool

Up until now we have not explained how to actually construct an OVF package with delta disk compression, only what parts go into it. Even though the ideas behind delta disks can seem a bit complicated it is quite easy to use delta disk compression in your OVF package by using OVF Tool. Basically, you use the option –makeDeltaDisks. Source may be an OVF descriptor, a VMX descriptor, or a VIM source (for example, a VM or vApp in vSphere). Target must be a directory. For example, we can use delta disk compression on our LAMP OVF package by invoking this command:

ovftool --makeDeltaDisks LAMP.ovf output-dir/

This will create a new OVF package in the output directory with delta disk compression. That is, both the disk and the OVF descriptor are updated, which means you can take the OVF package written in the output directory and deploy it immediately without any manual post processing. There are no restrictions to the type of input you give OVF Tool in terms of disks. It will try to create delta disk trees of all the disks in the input OVF package and output the optimal OVF package in terms of delta disk compression.

The disks that OVF Tool generates are compressed in the VMDK virtual disk format, but it is possible to apply a second layer of compression which may yield even smaller disks by using the –compress option. Use –compress=9 for the best compression. On a package the size of the LAMP OVF package (about 600 MB) it would yield about 30-40 MB less disk space (in our experience). Delta disk compressing our LAMP OVF package with this extra option would then simply be done by invoking:

ovftool --makeDeltaDisks -compress=9 LAMP.ovf output-dir/

Learn more about what OVF Tool can do by going to the OVF forum at VMware: http://communities.vmware.com/community/developer/forums/ovf. Here you can ask questions about OVF Tool and related products.