VMware Hands-on Labs VMworld Hands-on Labs

Hands-on Labs: Importing vPods

This is the fourth in my series on managing content for VMware Hands-on Labs. You can find the previous posts in the following locations:

The next phase of our multi-cloud vPod management process is sometimes a data replication, but there are times when we have multiple clouds at the same physical location. In that case, the next phase is simply an import to the target cloud. Doing this by hand is tedious and error-prone, so we employ some PowerCLI to keep us sane.

We use the built-in Import-CIVAppTemplate cmdlet for importing vApp templates. It usually does the job pretty well, but I have noticed a couple of things about the cmdlet that are worth sharing. First off, in my experience, it does not do very well with high latency links. At high latency, its bandwidth utilization is terrible when compared to even the Java-based uploader that is part of the vCD web UI. The capability to script the import process can be a worthwhile tradeoff in many cases. If the source and target clouds are geographically distributed, we usually replicate the export files to a “library” host that is LAN-connected to the target cloud and then perform the import locally anyway. I will go into more on that process in a future post in this series. Beyond that, I find that Import-CIVAppTemplate can use a little help in the resiliency department and I’ve created a little wrapper to help it out:

  1. Check to see if a template with the same name currently exists in the target catalog and, if so, do not try to import the template again
  2. Validate that all of the VMDK files referred to in the OVF are present and the correct size — if you have an incomplete export and do not do this, Import-CIVAppTemplate will not fail until it hits the missing/corrupt files. That could be hours into the process, and that’s no fun.
  3. Resume the upload if it gets interrupted. Network funniness, or system maintenance can wreak havoc with long-running tasks like uploading. When we’re preparing for a large event like VMworld, it seems like I am constantly importing pods to our various clouds. There is a -ResumeUpload switch in the cmdlet, but it requires a binding to the “in progress” template and separate call once a failure/disconnect has occurred. I put this into my wrapper along with a loop, a delay between attempts and a maximum retries counter.

My Import-Vpod wrapper function:

Function Import-Vpod {
<#
  Takes a vPod Name, Catalog, orgVDC, and a path to the vPod Library
  Imports the OVF located at <library>\vPodName\vPodName.ovf
  Will attempt to resume until successful completion or $maxRetries is exceeded
#>
  PARAM (
    $VPodName=$(throw "need -VPodName"), 
    $Catalog=$(throw "need -Catalog"),
    $OrgVDC=$(throw "need -OrgVDC"),
    $LibPath=$(throw "need -LibPath")
  )
  PROCESS {
    $ovfPath = Join-Path $LibPath $($VPodName + "\" + $VPodName + ".ovf")

    #ensure OVF exists, bail if not found
    if( !(Test-Path $ovfPath) ) {
      Write-Host -fore Red "!!! OVF file not found: $ovfPath"
      Return
    }

    ## Import-CIVAppTemplate does not check for all VMDKs, but will DIE
    ## We check to see if all the VMDK files referenced in the OVF exist
    if ( !(Test-Ovf -Ovf $ovfPath) ) {
      Write-Host -fore Red "!!! OVF export incomplete"
      Return
    }

    $maxRetries = 20
    $retryCount = 0
    $vPodExists = $null

    # If a vApp with this name already exists, don't try to import it
    try { 
      $vPodExists = Get-CIVAppTemplate -Name "$VPodName" -Catalog $Catalog -ErrorAction Stop
    }
    catch {
      Write-Host -fore Green "Importing vPod $vPodName"
      $va = Import-CIVAppTemplate -catalog $Catalog -name "$VPodName" -orgvdc $OrgVDC -sourcepath $ovfPath
      Start-Sleep -Seconds 120
      while (($va.Status -ne "Resolved") -and ($retryCount -lt $maxRetries)) {
        $retryCount += 1
        Write-Host -fore Yellow "!!! Transfer interrupted. Retry $retryCount of $maxRetries in 5 minutes..."
        Start-Sleep -Seconds 300
        Import-CIVAppTemplate -sourcepath $ovfPath -VAppTemplate $va -ResumeUpload
        #refresh $va here... 
        $va = Get-CIVAppTemplate -Name "$VPodName" -Catalog $Catalog
      }
    }
    If ($vPodExists -ne $null) { 
      Write-Host -fore Red "!!! vPod $vPodName already exists. Not importing." 
    }
  }
} #Import-Vpod

Using this function is pretty simple: connect to your cloud using the Connect-CIServer cmdlet, get the target vCD Catalog and OrgVdc where the template will be stored, specify the path to the template library and the name of the vPod, then let PowerCLI do the rest:

Import-Vpod

In the above example, I am importing a vPod called HOL-SDC-1307-v6 from the template library at E:\HOL-Library\s0\ into the HOL-Staging catalog into the only OrgVdc available in this cloud Org. Based on our naming conventions for a “template library,” the full path to the OVF in this example is E:\HOL-Library\s0\HOL-SDC-1307-v6\HOL-SDC-1307-v6.ovf 

When you’re finished, don’t forget to disconnect from the cloud using Disconnect-CIServer.

Since the Import-Vpod function uses my Test-Ovf function, it makes sense to provide that as well. This provides a good example of how you can open an OVF as an XML file by casting the result from Get-Content as [xml] then treating the variable as a series of nested arrays that follow the XML file’s structure.

Function Test-Ovf {
<#
  Tests whether all VMDKs referenced in an OVF exist on disk
  Takes full path to OVF, assumes VMDK files are in same
  Returns True or False
#>
  PARAM(
    $OVF = $(throw "need -OVF")
  )
  PROCESS {
    #Read the OVF
    [xml]$new = Get-Content $OVF

    #Read the filenames and sizes from the OVF into a hash table
    $ovfVmdks = @{}
    foreach ($disk in $new.Envelope.References.File) {
      $ovfVmdks.Add($disk.href,$disk.size)
    }
    #Get properties of the files on disk
    $vmdkFiles = Get-ChildItem $(Split-Path $ovf) -Filter "*.vmdk" | select Name, Length
    #compare disk to what is in the OVF
    foreach ($vmdk in ($vmdkFiles)) {
      if ( $ovfVmdks.ContainsKey($vmdk.Name) ) { 
        if ( $($ovfVmdks[$vmdk.Name]) -eq $($vmdk.Length) ) {
          $ovfVmdks.Remove($vmdk.Name) 
        }
      }
    }
    #if there is nothing left in the $ovfVmdks, that's good...
    if( $ovfVmdks.Count -eq 0 ) { Return $true }
    else { 
      foreach ( $vmdk in ($ovfVmdks.Keys) ) {
        Write-Host -Fore Red "Missing VMDK: $vmdk with length $($ovfVmdks[$vmdk])"
      }
      Return $false
    }
  }
} #Test-Ovf

This function does not show much unless there is something wrong, but that is the point since it is a “Test” cmdlet and usually used to feed a conditional statement. In the following example, I executed Test-Ovf twice. The first time, all of the files were present and the function returned “True.” For the second run, I removed one of the VMDK files and, as expected, the function identified the missing VMDK and returned “False.”

Test-Ovf

As usual, thank you for reading and I hope this is useful to at least some of you out there. A future post will dig into the details of the data replication mechanism we use today.

Doug Baer

Doug Baer worked on the Hands-on Labs program for many years and is now a Solution Architect on VMware's Cloud Incubation team. He is passionate about sharing knowledge, writing, developing…

Related Articles