One of the coolest feature in my opinion is Tagging in the new vSphere Web Client. Unlike Custom Attributes which was limited to an ESXi host and Virtual Machine object, the new Tagging capability allows you to create custom labels and metadata on ANY vSphere inventory object. In addition, you can have multiple tags per object and you can search based on tags to help you quickly find what you are looking and making this feature even more powerful. Just like with anything new, it takes time to get used to. To help you use the new Tagging feature, there is a built in Custom Attributes to Tags migration tool in the vSphere Web Client as Tagging will be the future going forward.
Tag Archives: migration
A short while back, I wrote a post on RDMs, and their behaviour when involved in migrations. I had someone reach out to me about this post, and highlight another behaviour which I missed in the previous blog posting. First, one needs to understand the settings for a pRDM when it is attached to the VM. Looking at the entries in the VM's vmx file, this is what we see:
scsi0:0.present = "true"
scsi0:0.fileName = "COR-XP-PRO-2_1.vmdk"
scsi0:0.mode = "independent-persistent"
scsi0:0.deviceType = "scsi-hardDisk"
The interesting part here is that the pRDM is independent persistent. This means that the RDM is excluded from Virtual Machine snapshots. Now when this RDM is converted to a VMDK (via a Cold Migration as mentioned in the previous post), the VMDK also inherits these independent-persistent characteristics, as shown here:
I'm highlighting this as an interesting fact to keep in mind after the migration. This VMDK may not behave as one might expect when snapshots are taken of this Virtual Machine, which could also have ramifications for backup products which snapshot virtual machines before backing them up (it could be that this VMDK is excluded from the backup).
Get notification of these blogs postings and more VMware Storage information by following me on Twitter: @VMwareStorage
This is more accurately referred to as migration since we are moving from a 32-bit host operating system, to a 64-bit operating system. The steps below will help you move from a SRM 4.0 / VC 4.0 environment where Site Recovery Manager (SRM) and VC are co-located (although that doesn't impact this process much if they are not) and SQL remote. I will try to point out useful information along the way to help in other migration scenarios.
I should point out that I am doing this with RTM bits, and not necessarily GA bits. That should not make a difference, but it may. And I am posting this a little early so some of my links are not available yet in this text, but will be updated after GA. As well, this is a very long blog. Sorry about that. Here is a PDF copy of this article ==>
Download Upgrading to SRM 4_c
. I will update this blog (and PDF) as I learn more.
I recommend you read carefully this document and its references completely, and understand them as well, and than plan an appropriate outage and work all the way through. You do want to minimize the outage window of both VC and SRM!
Some interesting background.
- VirtualCenter ISO build – 4.1 build 259021
- ESXi – 4.1 build 260247
- ESX – 4.1 build 260247
- SRM – 4.1 build 267817
- You must use a 64-bit DSN for VC and remember to make it using SQL Native. You can find SQLncli_x64.msi near the bottom of the page at click here.
- You will need to use a 32-bit DSN for VMware Update Manager (VUM) to see the KB article (1010401) for help making a 32-bit DSN in a 64-bit OS.
Things to get ready
- Make sure you have a good backup of everything that is going to change – which means your VC server, and database.
- Your new host that is 64-bit will need to have the same name and IP as the old host. This is important. So you will need to build it when it is not on the network. Avoid conflict with the existing VC. You need to preserve the VC and SRM FQDN name through the migration.
- Make sure you have access to your service account information for VC and SRM.
- Remove your vSphere Client plug-ins. This is not always necessary but it has helped me out in the past.
- If you have a spreadsheet that details the VM to LUN relationship that may be useful.
- An outage – you will have no VC and SRM for a number of hours. With preparation, and understanding of what is needed you should be able to keep your outage to around 3 – 4 hours but this will vary widely! SRM should be available approximately 1 hour after your VC is available again. But that will vary depending on your prep work.
- We need the VirtualCenter ISO – as we need the ISO as it comes with a folder we need, that the normal .zip doesn't. The folder is called datamigration. You can extract the ISO to a location that you will have access to when working on either the old or new VC.
Datamigration folder that is only present when you use the ISO file.
Migration – VC
- Your database for VC is remote, but you should still make sure to have a backup of it.
- You should be logged into the current VC (or current VC / SRM host). Than, either by using / mounting the ISO, or if you have extracted the files from it, click on autorun so that you get the main screen of the install. Near the bottom of it, under Utility, select Agent Pre-Upgrade Check. If it finds issues, solve them before continuing on. The issues generally have KB articles attached to help.
Autorun screen with the Agent Pre-Upgrade Check highlighted
- On the existing VC / SRM machine, copy the datamigration folder to the local hard drive and expand it.
- Make sure that the VMware Update Manager (VUM), VMware VirtualCenter, and VMware VirtualCenter Management Web Services are not running. If SRM is on the same host, it too should not be running.
- Now, you need to use the backup.bat file that is in your datamigration folder. It will do a backup of your environment in terms of the configuration that is not stored in the VC database. Notice the log folder? You can check out the backup.log file to see how the backup went and what it touched. The data folder has the actual backup. This backup contains things like port settings, SSL certificates, and licensing information.
- The backup.bat file has only a few questions.
- You should now copy the entire datamigration folder to a location where it can be copied to the new VC host when it is available.
- Turn off your existing host. Disconnect the network from it to make sure it is not accidentally turned on.
- You need to turn on your new host, and make sure it has the same FQDN / IP information, join it to the domain, patch, etc. Remember to install the 64-bit SQL Native client, and create a 64-bit DSN, as well as a 32-bit DSN. The URLs earlier in the document can help.
- You need to copy the entire datamigration folder to the new host.
- On the new host, you need to have access to the install media so map a drive.
- Use the install.bat file from the datamigration folder to start the install process.
- You will be asked for the path to VC, and than later VUM. If you are using the ISO, or extracted the files from it, the path will be the same for both.
The start of the install.bat process. The install.bat is in the datamigration folder.
- You will see the normal install prompts for VC.
- Use the same DSN information.
- Notice how you have a choice at some point to do an automatic or manual update of the VC agents on the hosts? I used automatic, but it is nice to have an option!
A nice improvement!
- Select the same path as you had used previously on the old VC.
- Use the same ports.
- The next prompt is about the size of the JVM memory. Use the default or make a more appropriate choice.
- After the VC install is finished the install will return you to the install.bat file and start the VUM install process.
- While VUM is being installed, make sure to use the appropriate VC service credentials, and the 32-bit DSN.
Use the same path for the VUM install process if using the ISO or extracted files, otherwise, use the correct path!
- There is a restore.log file available in the logs folder of the datamigration folder. It can show you what was done.
- It is important to understand that the install.bat file is smart. If, like me, you didn't have a 32-bit DSN for VUM, you could exit, and create it. When you start the install.bat process again, it will take off where it finished and not waste your time!
- You should now confirm that VC and VUM are running. In my upgrades there were NOT. They were using the Local Security instead of my VC service account. I changed the credentials and everything was fine.
- Install the vSphere Client from the autorun screen.
- Connect to the VC.
- Install the VUM plug-in. You can do this on your own desktop, or on the server.
- Now check your VUM configuration, scheduled tasks and browse around to make sure your configuration is there!
You have now upgraded one of your VC servers. You need to do the other one now!
Note1: In all of the work I did, we always had the VC / VUM services not start, and we had to assign the proper credentials instead of Local Service. No big deal, but don't forget.
Note2: Be careful with the 64-bit and 32-bit DSN as it can get confusing. If you make a mistake you can cancel the VC or VUM install process, fix the DSN, and restart the install.bat file. It will not redo an unnecessary install but rather start where you last finished successfully. A very nicely done install.bat file.
Upgrade to SRM 4.1
This too is more of a migration, but this time we don't have the lovely datamigration script to help us! But the steps below will help. The process below is for migrating to 4.1 when you are using a 32-bit host OS. If you are in fact using already a 64-bit OS, you can do an in-place upgrade, but while we do not step through that process below, the information below can help.
A very useful reference is the release notes (click here).
Some Interesting background
- SRM 4.1 build 267817
Things to get ready
- You will need to have your VC infrastructure already upgraded!
- Make sure your SRM was able to do a test failover or two before you started the upgrade process!
- You will need to confirm that your SRA has been upgraded or certified for 4.1. Most have been, but confirm it. Check the VMware download site for SRM and the current version of the SRAs, and the SRA download for a readme that talks about what it is certified for.
- SRM 4.1 bits
- SRA bits
- Be aware that SRM requires a 32-bit DSN, which is NOT the default on a 64-bit OS. If you need help with this check out the URL links in the VC section above.
- Remove your SRM plug-in.
- You need an SRM backup, but it needs to be taken at the same time – as if it was in a consistency group. BTW, it needs to be restore like that too. Never restore just one or the either side as it will not work. You should also have history reports as hardcopy as well.
- You should copy your vmware-dr.xml file from both the protected and recovery side to a central location. This file is by default, in c:\program files\VMware\VMware Site Recovery Manager\config .
Migration – SRM
- Remember your new SRM host must have the same name / FQDN / IP so you will need to turn off your old SRM host after you have your backups and .xml file so you can deploy the new host.
- Backup the SRM database on each site.
- Turn off the old SRM host.
- Turn on the new SRM host.
- Make sure you have a 32-bit DSN.
- Install SRM 4.1.
- If you are re-using the SRM 4.0 database, make sure you have a backup as errors during the install, or a cancellation of the install, could introduce corruption into the database.
- Since you are using the old database, you will be prompted about there being an SRM extension installed already. This is not an issue, and correct, so select Yes.
Select Yes at this prompt.
- You will need to remember the 32-bit DSN during the install.
- SRM will likely not start, same as VC, and so you should just change the credentials to the SRM service account and hit the retry button.
- You now have SRM running, but not quite finished.
- Start the client, and install the SRM plug-in. Again, you can do this on the server or your admin workstation.
- Install the SRA.
- Now get the other side done.
- Important – if you have made any advanced setting changes, you will need to work through the section below. If you have not made any changes you can continue. Some sample changes are SanProviderCommandTimeout or San.Provider.hostRescanRepeatCnt .
- Now you will need re-create the site pairing, and reconfigure the array manager credentials. The passwords didn't come through. After entering the correct credentials, you will need to select the array!
When you re-enter your credentials to the Array Manager, make sure to select your array!
- Now do a test failover and make sure it works!
You are now complete. If you have any issues, please do not hesitate to contact our support organization but also leave me a comment!
Migrating Changes to Advanced Settings
If you have not made any changes to Advanced Settings, you do not need to do this section – which should be true for very much most of our customers. Changes would be things like SanProvider.CommandTimeout or San.Provider.hostRescanRepeatCnt. See below for a screen-shot of the Advanced Settings categories. If you know the changes you made you can just add them to your new install. But if you are not sure, you will need to work through the process below.
Start by loading your vmware-dr.xml file. Load the one from the Recovery Site when editing the Advanced Settings for the Recovery Site and do the same for the Protected Site.
- When you are in the Advanced Settings windows, you will need to work through each category.
- One example is localSiteStatus. Search your vmware-dr.xml file for that phrase.
- In the section of vmware-dr.xml file you find the category in our example of localSiteStatus, look for variables that match in the Advanced Settings category, and change the value to match what is in your VMware-dr.xml file. See below for an example.
This is a sample of the vmware-dr.xml file.
- After we see what is in the vmware-dr.xml file, we record it in the Advanced Settings. See below for that.
LocalSiteStatus of the Advanced Settings.
Remember you will need to work through this process for each category.
Some issues I found
I have mentioned these issues elsewhere but thought I would mention them here again.
- Forgot to update the credentials for the arrays – so test failover failed.
- Used a 64-bit DSN for SRM. It didn't work until I created and used a 32-bit DSN.
- Didn't know to install the 64-bit SQL Native client on Win2K8R2 SRM server. So I did and the problem left.
- Did not see the SRM plug-in, but saw one called vcDR and it worked. Restarted the vSphere Client also cleared this up. Should have removed the SRM plug-in before starting.
- None of the VC services started when I thought they should. But of course, the VC service account settings were not migrated. So after VC is finished installing, change the account credentials and you will be fine. For SRM as well.
- Didn't know that VUM needed a 32-bit DSN.
- It is likely not connected to the migration, however, I was able to do only three successful test failovers after the migration was complete. After those three, I experienced only failures. The error was Error:Error occurred: failed to prepare ShadowVM for recovery. One VM was successfully recovered but not the others. I removed the protection group, made sure the ShadowVM lcoation had no folders for the VM's that were in that removed PG, and in fact there was several folders that should not have been there. So I removed them. Than created the PG again and everything has worked fine since then.
Thats it! As if it wasn't enough! Let me know in the comments how it goes for you! I will update things as I learn more.
If you wish to undo this migration, it is almost easy. You would turn off the new hosts, and you would turn on the old ones. They would not be happy since the databases that would still be in use would be the new ones. You would need to stop the VC and SRM services and restore the back up copy of the databases I mentioned you needed to have. Than start the services and you should be good to go!
7/17/10 – Update – I made it more clear that you need to preserve the VC and SRM server names. As well I added the undo option!
8/5/10 – Update – I added the URL for the DataMigration tool KB article, and corrected some spelling!