This post is part 2 in a series on the backend processes we use to manage the Hands-on Labs. The first part is available here.
Before I get into the guts of handling export, replication and import, I think it is important to understand how to identify which templates we need to work with. Sure, starting from scratch is simple: you have a bunch of templates in cloud A that you need to replicate to cloud B. Typically, that is going to take quite a bit of time. The problems involved with initial transfer to the cloud are similar to those encountered when standing up a new DR facility. Namely, how do you “seed” the DR site with the data from production. Shipping a box of tapes and performing a restore is often the least expensive option, but may not provide what you expect if your data change rate is high. That discussion is beyond the scope of this post, but I can come back around to it if there is enough interest.
“Version Control”
Back to the topic at hand. Once we have a set of clouds that host our content, we need to ensure that the templates available in each cloud are the same. As we revise our content, we check in new versions with new names. Each of our labs have a “Lab Code” or “SKU” that we use to identify them internally. For example, our wildly popular NSX lab has a SKU of “HOL-SDC-1303,” which is easier to track than the full name, “VMware NSX Network Virtualization Platform.” The first template for this lab might be called HOL-SDC-1303-v1. When we make a change to that lab, we check it in as HOL-SDC-1303-v2, then HOL-SDC-1303-v3, and so on. This is a rather simplistic practice, but it serves its purpose.
Which versions are where?
If you maintain tight control over your inventory, and you are the only one managing it, you can probably handle keeping things in sync manually. You also probably don’t get a lot of sleep. With multiple developers, a compressed development cycle, and a distributed team, I turned to PowerCLI to help compare our cloud catalogs. I am not aware of any built-in cmdlets for comparing catalogs across cloud instances, so I wrote a very basic one that met my needs. Using the version of the lab as a suffix on the template name creates a unique name and allows comparison of catalog items across clouds by vApp template name.
The basic logic is to connect to each vCD instance and enumerate the contents of the HOL-Masters catalog from each instance — that is what we call the catalog that holds our lab templates. We have defined one of the cloud instances as the authoritative source to keep the comparison logic simple:
- Collect a list of templates in the catalog on the source/authoritative cloud
- Walk the items in the target cloud’s catalog. For each item in the target cloud’s catalog that matches one on the list, remove from the list
- Once we reach the end, anything remaining on the list exists in the source and not the target.
There is no magic here, and I wasn’t looking for a complicated two-way sync or multi-site comparison. Simple is good.
In case you’re curious, the reason I use names is that date stamps and UUIDs on templates are usually different between clouds, even for the same version. There isn’t much else I have to work with besides the name, which we control.
A bit of PowerShell
I wrote a function called Compare-Catalogs to compare vCD vApp Templates in two specified catalogs, possibly across clouds. One catalog is identified as the source, or “master,” if you prefer. The function assumes that you have logged in to the “source” and the “target” clouds with appropriate credentials using the Connect-CIServer cmdlet. The call looks something like this (all on one line):
Compare-Catalogs -CatalogSrcName "HOL-Masters" -CloudSrcName dev-us01.hol.vmware.com -OrgSrcName "labs" -CatalogNewName "HOL-Masters" -CloudNewName prod-us01.hol.vmware.com -OrgNewName "labs"
The previous call would compare the HOL-Masters catalogs in the labs orgs of our DEV and PROD clouds, using the DEV version as the source. It outputs a list of templates to the console for reporting and on the pipeline for further processing.
The main “guts” of the function look something like this:
#Build the list of templates in the source catalog
$vAppList = @{}
foreach ($vApp in (Get-CIVAppTemplate -Catalog $catSrc)) {
$vAppList.Add($($vApp.name), $($vApp.Created))
}
#Walk the target catalog, remove matches from source list
foreach ($vApp in (Get-CIVappTemplate -Catalog $catNew)) {
if ( $vAppList.ContainsKey($vApp.Name) ) { $vAppList.Remove($vApp.Name) }
}
Output is a simple matter of walking what remains of the list.
#The vAppList contains only templates that are not in the target
if( $vAppList.Count -ne 0 ) {
Write-Host "==> vApp Templates present in $CatalogSrcName on $CloudSrcName"
Write-Host "==> and not in $CatalogNewName on $CloudNewName"
foreach ($name in ($vAppList.Keys)) { Write-Host $name }
Write-Host ""
#Return the vPod names on the pipeline for further processing
Return $($vAppList.Keys | Sort)
} else {
Write-Host "==>No vApp Templates exist in $CatalogSrcName on $CloudSrcName"
Write-Host "==> and not in $CatalogNewName on $CloudNewName"
Return
}
Note that this simple compare does not account for extra templates that exist in the target catalog, but we have times when there might be other templates in the target catalog, so this is not a concern for me. There is nothing stopping you from running the function in the opposite direction to check this…
I have the function output an array of names of the “missing” templates rather than objects because the objects in one vCD instance have no meaning in the other instance. If you wonder why the function displays the names in twice in the console, once is from the pipeline and the other is due to the Write-Host. I use this function in two ways: ad hoc comparisons of catalogs and to feed another function that performs the export and import processing. In the second case, the function consumes the values from the pipeline, so I don’t see how much work I have to do. By printing out the list, I can see how out of sync I am before the sync processing starts.
Conclusion
I hope someone finds this valuable, or at least interesting. Comparing lists of things is a common need, and this is just one way to do it. Rather than go crazy with features, my goal is to keep things simple so that anyone can understand and maintain it.
If you would like the complete function, you can download it here.
Up next, I’ll talk about getting templates out of one cloud and into another…
