posted

3 Comments

This is a fairly lengthy post outlining the process for upgrading from SRM 4.x to 5.0 including the steps needed to upgrade both the vCenter servers and the SRM instances.  

Michael White and I spent a lot of time in our labs testing this process out and ensuring that it would be thorough and repeatable, so hopefully this will be of value to you and augment the upgrade section of the admin guide when SRM 5 is released.  Please ensure you read both this article and examine the admin guide thoroughly before beginning, to ensure you have all the necessary information and understand all the steps!

We will cover only one site upgrade in this process – you will need to repeat the same steps at the other site.  We recommend you start by upgrading the protected site first.  This will ensure you can still failover in case of outages during the upgrade process.

Please note we do not cover the ESXi upgrade process in this post.  ESXi 5.0 is only required for SRM 5.0 if you intend to use vSphere Replication.  If using array-based replication as is presumed for this upgrade, you can use the current shipping update of 3.5, 4.0, 4.1 or 5.0 of ESX or ESXi.

 

Overview

This document will help you upgrade from vCenter 4.1.x to vCenter 5.0, and SRM 4.1 to SRM 5.0. 

The process below begins by walking through a vCenter upgrade, then proceeds to an SRM upgrade.  This only will cover the upgrade of the vCenter Server and SRM instances at the protected site.  This will not cover an upgrade of the recovery site vCenter or SRM: that must be completed before SRM 5.0 will work.

Materials

You will need the following materials:

  • vCenter  5.0 software and licenses
  • SRM 5.0 software and license
  • A certified SRA for SRM 5.0
  • Current SRM and vCenter ODBC information
  • DSN credentials
  • Username and credentials for any service accounts in use by the current version.

Other Resources

You can find additional help in the following:

Be Aware

  • This process will impact your ability to failover in a crisis.
  • You should be sure to use FQDN or IP addresses consistently.  We recommend the use of FQDN.  Make sure whatever choice you make, however, is used throughout the upgrade and the pairing process!
  • Once started with this migration make sure you finish as rapidly as possible without impacting reliability to return to your protected status.  Ensure you give yourself enough time to complete the upgrade in one session.
  • If possible, you should practice this migration or consider using outside help from consultants that have experience.  It is not a complex upgrade, but there are multiple steps that may not seem immediately logical. To minimize the outage you want to ensure success the first time.

Objectives

Specific objectives include:

  1. Upgrade vCenter to 5.0.
  2. Upgrade SRM to 5.0.
  3. Preserve recovery plans and history
  4. Preserve protection groups
  5. Successfully perform a test failover after the upgrade.

Activities

The steps for the upgrade include:

Preparation

  1. Confirm first that vCenter and SRM work properly before any upgrades.  Do several test failovers to confirm that SRM is working and remember which ones you used. After the upgrade we will use the same RP to confirm success.  It may be of interest to note the time needed to complete the recovery plan test both before and after the upgrade.  Also please take note of any IP customization or scripts that are in use to confirm they are working at the end.
  2. Gather your materials (the new admin guide, this posting, documentation about your environment, etc.)
  3. Confirm you have good backups of the machines to be upgraded.  Make sure you can restore the SRM databases from both sites as if they were a consistency group – meaning you can restore both of them from a "point in time" backup.
  4. Ensure you have a backup of both the vCenter and Update Manager databases..
  5. Make sure you are familiar with the new release notes for vSphere and SRM.  Please read these notes carefully as there are a handful of statements that may prove important to you!.
  6. Make sure you are very familiar with the release notes for your new SRA.  Please note that previous version Storage Replication Adapters will not be compatible with SRM 5.0.  You will need to ensure you have a current copy of the SRAs appropriate for your arrays, compatible and certified for SRM 5.0 before beginning.
  7. Set aside 3 hours where you can work uninterrupted.  For some of that vCenter will not be running, and during much of that 3 hour period SRM will not be functional.  Once the vCenter Server upgrade has started, SRM 4.1 will not be accessible for that environment until the entire upgrade is complete and the vCenter Server, vSphere Client, SRM instance, and SRM plugin within the client are all upgraded.

vCenter Upgrade

  1. Stop the vCenter Update Manager service from running.
  2. Stop the vCenter service from running. 
  3. It may help your install performance to copy the vCenter 5 zip or ISO to local storage.
  4. Double click on the autorun.exe.  After you confirm that this is the file you wish to execute you should see a blue screen labeled VMware vSphere 5.0.

 

Autorun
Figure 1 – Main install screen from autorun 

  1. Notice how the right side will display information on what you have selected on the left side.  The Prerequisites that are show will be automatically installed as necessary
  2. To begin, select the first option: vCenter Server.
  3. Once the installation begins, it will alert you that there is an earlier version of vCenter Server already installed and it will be upgraded.  This is expected behaviour:  If you do not see it please cancel the installation and ensure you are running the upgrade on the vCenter Server!
  4. You will be notified that the old vCenter licenses will not work and you should have your new ones ready.  You may choose either to enter the license here or ignore this warning and instead to enter the license in the License Applet in vCenter.  In many ways the latter choice is better as you may wish to alter your licensing beyond the addition of the new vCenter/SRM license.
  5. Note that some of the prompts will already have the right information – such as the path for the install, due to this being an upgrade.
  6. After connecting to the vCenter database you will be prompted with a warning that this vCenter Server is being used by an extension, such as for example VMware Update Manager.  You may not see this, or you may see a different application extension mentioned.  It is important to understand that any extensions listed will NOT work after this upgrade and you should make sure you have upgraded versions of them.

 

Registeredextensionwarning

Figure 2 – Registered extension warning during install

  1. You will now see a prompt about upgrading your vCenter Server database.  Since this is an in-place upgrade you will want to choose to accept the upgrade.
  2. The next prompt is about an automatic or manual upgrade of the vCenter agent on the hosts.  We suggest you leave the default, on automatic, unless you have a lot of hosts and a reason to use manual.
  3. Next you will be prompted about the vCenter Server Service and what account to use.  Many customers will have a service account to use, but often the SYSTEM account is fine.  I am using the SYSTEM account, as I do not have a reason not too.  Be aware, however, that if you are executing scripts in SRM you will likely have to have a service type account to use as this account is what is used to execute scripts in SRM.  Likely what you were using with the previous version should be carried forward at this point.
  4. Confirm the path to the existing folder structure.  It should be selected correctly in accordance with the previous installation, but confirm it is correct.
  5. The Configure Ports screen should normally be left as it is.
  6. The Configure Ports for Inventory should also normally be left as it is.
  7. The next screen is called vCenter Server JVM Memory and it will help configure your JVM environment.  Pick the appropriate choice for your environment based on size.
  8. The final screen will ask if you wish to increase the ephemeral port value.  You can usually ignore this under normal circumstances..
  9. The file copy will then begin and can sometimes take a while (though usually not as long as it indicates.)

 

After this is complete, you should still see the blue "vCenter Installer" on-screen.  In accordance with your environment you may choose to install the vSphere Update Manager or other tools, but in terms of basic requirements for SRM we are now done with upgrading the vCenter Server.  Be aware that at this point SRM will not function on the site you are upgrading, nor will any other extensions until they are upgraded to be compatible.

Note that if you are using an older operating system like Windows XP to host your vSphere Client it is a good idea to use the vSphere Client option in the vCenter Server Installer screen to update your client.

Certain prerequisites such as .net will not be automatically installed or upgraded on this platform by connecting directly to the vCenter server.  If you are using Win7 or later you can use the traditional method of starting the vSphere Client and connecting to the new vCenter and being updated. 

Using the official installer from the blue installer screen will always ensure all pre-requisites are installed.

Ensure your hosts and vCenter are healthy and everything is running.  If so, you do not need to do any further upgrades of vCenter or the hosts before continuing.

If you did not add the vCenter 5 licenses during the upgrade, when you connect to the new vCenter your hosts will show as disconnected.  Add your new vCenter 5 license in the license applet and apply it to the vCenter instance, and then re-connect to the hosts.  If using a vCenter 60-day evaluation license that has not expired you will not likely experience this error.

Be aware that if you were using Linked Mode before the upgrade, at this point it will not be working as your remote site is incompatible.  You will need to re-enable linked mode after upgrading the other vCenter at the end of the upgrade process.

SRM Upgrade

Preparation

1. Ensure again that you have a backup of both SRM databases, both protected and recovery, and ensure they are from the same point-in-time.  If errors occur you will need to restore them together: You cannot restore just one.

2. Ensure you have SRM 5.0 licenses.

3. You will need an updated SRA for SRM 5 from your storage vendor appropriate to your storage, so make sure you have one.

4. Copy the SRM installation file locally to the SRM server you are upgrading.  Again, we recommend upgrading one site in its entirety before upgrading any components on the other site.  If you have upgraded your protected site's vCenter, please ensure you are now upgrading the protected site's SRM instance.

5. Use FQDN, if possible, everywhere!  When pairing sites, connecting SRM to the vCenter instance, choosing usernames for pairing or for deploying instances, or any other hostname/username instances, where possible fully specify the domain name, full DNS name, or so forth.  Most important to this process however is consistency.  If using fully qualified domain names make sure you do so consistently throughout the process.

6. Your install performance may be better if you copy the install file locally rather than reading the file across a network share.

7. Remove the SRM plug-in before the upgrade.  This is not strictly speaking necessary as it can be upgraded, but this will ensure a clean environment and help avoid potential issues.

Migration

  1. Start the install and watch for the new description field as you install.

Copyandinstall
Figure 3 – File copy during installation includes extra information

  1. You will get questions that you have seen before during the initial SRM installation (location, etc.) and you can answer them as you answered them during the previous install.
  2. Once the install is complete, you will be shown your storage information.  Confirm what you see – this is the info for your SRA configuration.  See below for an example. Either leave this window open with the information to refer to later, or copy-and-paste the data into a text file.  It is not necessary to do so, but this will save time later as we will need to once again pair the sites and enter array-manager information.  Having this data handy makes it easier to do this.

Storageinfopostinstall
Figure 4 – Storage information for SRAs after install has completed 

  1. You will now need to install the new version of your SRA that works with SRM 5.  You may need to uninstall the previous version, or install the new SRA to a specific location.  This information can be found in the SRA release notes.  Please read the SRA notes carefully to review the requirements.  You do not need to, nor should you, restart the SRM service to load the newly installed SRAs.  Instead you will need to use the refresh button in the user interface within SRM after installing the new SRA.  This is different from previous versions of SRM.
  2. After launching the vSphere client you will need to install and enable the new SRM plugin.  If you have forgotten to uninstall your SRM plug-in, and you try to use the old plug-in you will encounter an error.  Uninstall the plug-in via the Add Remove panel and then install the new plug-in.
  3. Remember to install your SRM license.  If you install the license on both sites it will help make a failback operation smoother – this doesn’t however mean you can have twice as many protected machines!  If you will be using Linked Mode, you do not have to install the SRM license on both sites.

Even though SRM is now updated, you are not yet able to use it until you install the SRM 5 plug-in in your vCenter Client.  If you have already installed it manually, you may need to exit and restart your vSphere client.   Once the plugin is successfully installed and the SRM icon shows in the main screen of your vSphere client, you have completed the installation of the upgrade for the primary site.

At this point, only the installation and configuration of vCenter and the installation of SRM are completed at one site.  Before continuing to configure the upgraded SRM you will need to upgrade the other site, both vCenter and SRM.  Please follow the same steps at the remote site to bring the versions into alignment of both vCenter 5.0 and SRM 5.0.  Once this is done, the configuration of the SRM pairing may be continued. 

Click on the SRM icon in your vSphere Client to continue on the vCenter instance at your primary protected site.

  1. You will now need to pair the protected site with the recovery site.  Please remember to use a consistent naming convention for both username authentication and FQDN for your sites.
  2. You will need to ensure the SRAs are installed and configured at both sides, and then enable them.  To do this, click on the Refresh button on the SRA tab for both sites.  Next, add an Array Manager for your storage arrays under both sites, and remember to click on the enable button to make the array pair active.  Since SRM 5.0 has a single GUI from which you may manage the entire SRM environment, you will not need to open two vSphere Clients as in the past.  You may do the entire management, site pairing, array management and the rest from the one single SRM interface you are currently using.
  3. At this point SRM will be paired and have the storage configured.  You may notice at this point that your recovery plans and protection groups are not visible.  This is normal, expected behaviour, do not panic!  You will need to work at the command line to import the rest of the configuration.
  4. Open a command prompt (cmd.exe) and change directory to the SRM bin folder.  By default this is found at:

 

C:Program FilesVMwarevCenter Site Recovery Managerbin 

 

  1. Once in this folder, type the following command to import the missing protection groups, recovery plans, and history reports.  This data was automatically generated by the upgrade installation, but needs to be imported by the command line listed:

 

Srm-migration.exe –cmd importConfig –cfg ..configvmware-dr.xml –lcl-usr administrator –rem-usr administrator

 

  1. The lcl-usr command is for an admin equivalent user on the local side, and –rem-usr is the same on the remote side. Specify a user that has appropriate administrator privileges!
  2. This will re-populate your "missing" recovery plans, history, IP Customization, protection groups, and script information back into the upgraded SRM.
  3. It should be noted that in the past scripts didn’t have names, so the scripts in SRM 5 will not have names either.  They will still work, but make sure you edit the recovery plan steps that call the scripts to provide a name.  This will ensure that the script name is captured in the improved history reports.
  4. At this point the upgrade is complete.  If everything looks as it should,test your configuration now by running a test recovery plan failover!
  5. You will need to do additional configuration in order to prepare for failback operations.  This entails setting up Inventory Resource mapping, and IP Customization for both the protected site and recovery site so they are mapped correctly between sites for both failover and failback..

You should now be able to use your old PG and RP to do new functions of SRM 5.0 such as planned migrations and reprotect.  Moreover, your scripts and IP customization should be correctly preserved.

Adaptations

This guide covers the upgrade of the environment in a scenario where there are separate vCenter / SRM and SQL hosts.  You can likely use the information in this guide to help in different but similar circumstances when you have, for example, co-located SRM / vCenter instances, but these scenarios are not immediately handled in this document.

Evaluation

Once the upgrade is finished it is very important to test to ensure functionality.  You should be able to do the following to confirm the upgrade was successful.

  1. Test that failover works as it did before the upgrade.  Test the same PG / RP as you did before this all started.  If you noted the recovery time in your testing prior to the upgrade, please compare the recovery time following the upgrade.  It should be the same or even considerably better.  If the recovery time is slower it is possible that there is a timeout occurring during recovery, for example during script execution.
  2. To confirm the new SRA is working appropriately, you should run a planned migration (or DR failover), a reprotect, and a failover (or planned migration) to fail back to the primary site as quickly as can be scheduled.  It doesn’t need to be a production RP or virtual machines to do this, but should be a genuine failover instead of a test.  This will give more authoritative testing of the upgraded functionality.

Supplemental

Confirm as well that any scripts you have configured in the recovery plans continue to work, and consider if they might benefit from being migrated from server-based scripts to execution as in-guest scripts. 

You will need to redo any dr-ip-customizer work so that each VM contains both protected site and recovery site IP configuration.  Please read pp.81-84 of the Site Recovery ManagerAdministration Guide for full details on the changes and usage of dr-ip-customizer.  You can import your old CSV file but it may be better to start again if possible, as the formatting of the command line and CSV file for IP customization has changed.  Samples of the new commands for generation and application of IP CSV files for batch loading are as follows:

Dr-ip-customizer –cfg ..configvmware-dr.xml –o c:example.csv –cmd generate –vc vcenter-recovery

Dr-ip-customizer –cfg ..configvmware-dr.xml –csv c:example.csv –cmd apply –vc vcenter-recovery

vSphere Replication

This guide does not cover the deployment or configuration of vSphere Replication.  For deployment guidance on vSphere Replication please see the Site Recovery Manager Administration Guide found at https://www.vmware.com/pdf/srm_admin_5_0.pdf

Before starting configuration of vSphere Replication, don’t forget to edit the Runtime Settings on both sites’ vCenter Servers before starting.  There is a wizard that includes a number of the vSphere Replication configuration steps within SRM that you may not see if you have disabled the Show Getting Started tabs.  It contains links for the necessary steps, but moreover will also display incomplete items.  

RuntimeSettings
Figure 5 – Runtime settings in vCenter Server – Managed IP address information must be correct

vSphere Host Upgrades

We recommend using vSphere Upgrade Manager to do host upgrades.  You will need to download the ESXi 5.0 Offline Bundle or ESXi ISO and use it to create a host upgrade baseline.  It will update ESX or ESXi 4 to ESXi 5.

You may see some plug-in errors in the plug-in manager before you upgrade your hosts, but they should disappear after the final upgrade of the hosts.

VMware Tools Upgrades

We recommend you use VUM to upgrade the tools and virtual hardware. If using vSphere Replication the virtual hardware must be at version 7 or later.  We always recommend using the latest VMware Tools unless there is a reason not to such as failover to older vSphere versions.

vSphere Web Client

This is the future direction VMware is taking the vSphere Client, and is included for install with vSphere 5.0 in parallel with the full vSphere Client.  It is worthwhile to install and configure the web client services as part of your vCenter Server upgrade, as it is very powerful and offers a more flexible means of accessing the vCenter Server.  With the current release there is some functionality difference between the products, so be aware not all standard vCenter capabilities are accessible through the Web Client.  After deploying it to your vCenter Server, use the info below to configure it.

Initial configure of Web Client – http://vc_fqdn:9090/admin-app

Use of Web Client – http://vc_fqdn:9443/vsphere_client

Upgrade Flow Sequence

The outline below may help you to think about or picture the upgrade process.

i) Site A

  1. Preparation – software and info
  2. Upgrade vCenter
  3. License
  4. Upgrade SRM
  5. Install or upgrade SRA

ii) Site B

  1. Preparation – software and info
  2. Upgrade vCenter
  3. License
  4. Upgrade SRM
  5. Install or upgrade SRA

iii) Site A

  1. Pair SRM
  2. Import configuration
  3. Test failover
  4. Complete Resource Mapping for protected side
  5. Complete IP Customization for protected side (if necessary)
  6. Test planned migration
  7. Test reprotect
  8. Check out the history reports

iv)  (Follow up activities for Site A)

  1. Consider requirement for new or improved SRM features such as VM dependencies and vSphere Replication.
  2. Upgrade remaining
    1. Extensions and plugins like VUM
    2. vDR 2.0 upgrade and plugins
    3. etc.
    4. Add new Extensions like Auto Deploy
    5. Upgrade hosts
    6. Implement, if necessary, vSphere Replication (protected site components like vSphere Replication Management Server)

v) (Follow up activities for Site B)

  1. Upgrade remaining
    1. Extensions and plugins like VUM
    2. vDR 2.0 upgrade and plugins
    3. etc.
    4. Add new extensions like Auto Deploy
    5. Upgrade hosts
    6. Deploy remaining vSphere Replication components (recovery site components like vSphere Replication Management Server, vSphere Replication Server)