Home > Blogs > VMware vSphere Blog


Migrating RDMs, and a question for RDM Users.

Cormac_Hogan
Posted by Cormac Hogan
Technical Marketing Manager (Storage)

On a number of occasions recently, I had to investigate what happened to a Raw Device Mapping (RDM) when:

  • The VM to which the RDM was attached was Storage vMotion'ed (VM Powered On)

 

  • The VM to which the RDM was attached was Cold Migrated (VM Powered Off)

 

Some of you may even have been following along the comments in some of my previous postings. Well, this is what I observed, testing with both pRDMs and vRDMs.

 

VM with Physical (Pass-Thru) RDMs (Powered On – Storage vMotion):

  • If I try to change the format to thin or thick, then no Storage vMotion allowed.
  • If I chose not to do any conversion, only the pRDM mapping file is moved from the source VMFS datastore to the destination VMFS datastore – the data stays on the original LUN.

 

VM with Virtual (non Pass-Thru) RDMs (Power On – Storage vMotion):

  • On a migrate, if I chose to covert the format in the advanced view, the vRDM is converted to a VMDK on the destination VMFS datastore.
  • If I chose not to do any conversion, only the vRDM mapping file is moved from the source VMFS datastore to the destination VMFS datastore – the data stays on the original LUN (same behaviour as pRDM)

 

VM with Physical (Pass-Thru) RDMs (Powered Off – Cold Migration):

  • On a migrate, if I chose to change the format (via the advanced view), the pRDM is converted to a VMDK on the destination VMFS datastore.
  • If I chose not to do any conversion, only the pRDM mapping file is moved from the source VMFS datastore to the destination VMFS datastore – the data stays on the original LUN

 

VM with Virtual (non Pass-Thru) RDMs (Power Off – Cold Migration):

  • On a migrate, if I chose to covert the format in the advanced view, the vRDM is converted to a VMDK on the destination VMFS datastore.
  • If I chose not to do any conversion, only the vRDM mapping file is moved from the source VMFS datastore to the destination VMFS datastore – the data stays on the original LUN (same behaviour as pRDM).

 

As you can see, there are 3 occasions when an RDM could be converted to a VMDK. Perhaps the most surprising is the fact that a pRDM could be converted to a VMDK, when a cold migration of the VM is attempted, and the format is changed.

I've since asked our engineering team to put a warning into the migration wizard in vSphere to highlight that this is what's going to happen. Right now, you don't get any warning about this.

I wanted to finish this post with a question to the community. How useful would you find an RDM -> RDM migration tool, i.e. the ability to move data from one LUN to another LUN via the vSphere migration wizard? Please leave me a note in the comments if you think you would use this?

Get notification of these blogs postings and more VMware Storage information by following me on Twitter: Twitter @VMwareStorage

58 thoughts on “Migrating RDMs, and a question for RDM Users.

  1. Andrea Casini

    I think it would be pretty useful.
    The new features available with vsphere 4 and even more with vsphere 5 make less and less sense to use RDM or at least they reduce a lot the reasons and the scenarios why they should be used.
    Such a tool would be useful for those customers who used RDMs in the past and could use a migration to a newer platform for say goodbye to RDMs and improve managebility, so in my opinion it would be a great thing!

  2. Steffen Oezcan

    … and now add VAAI-support to your RDM-to-RDM-tool and we are ready to go. I think such a tool would be a nice-to-have for some(!) customers. If its easy to develop, why not.

  3. d_

    Wouldn’t use it. We’ve already converted our vRDM’s to VMDK’s.Largest was 1.5TB. No downtime, no issues. Wish the rest of the organisation could understand the magic that gets done behind the scenes.

  4. Dan Stace

    Great article Cormac, thanks for providing this info.
    I look forward to retiring RDM’s with VMFS5 and the info here.

  5. Alex G

    You are talking about pRDM to pRDM migration, if I am reading this correctly.
    In my opinion this would be incredibly useful, but only if done on a live VM. It would allow a migration from one array to another, or from a slower LUN to a faster one within the same array. Or from a RAID10 LUN to a RAID5 one… You get the picture. Even more useful if the migration tool display the VASA information somehow.
    It would only have limited uses if the migration cannot be done live, as this kind of LUN to LUN migration could be done live using OS features. But doing it from inside VMware could be simpler, easier and neater.

  6. Chogan

    Thanks for all the comments so far. Right now, I’m just trying to figure out if there is a demand for such a feature, so keep the comments coming (doesn’t matter if you’re in the yeah or nay camp).

  7. james

    Anything to make any process easier is worth doing. I didn’t realize that doing some of these storage migrations hot or cold would possibly result in changing it from a RDM to a vmdk format so at the very minimum warnings in the svmotion wizard would be a good thing. A migration wizard to do these things would be a little clearer possibly when it detects a RDM attached to a VM vs. a svmotion wizard ? Its slightly different but still important to be able to do the migration in some cases which it sounds like is possible.
    Thanks!

  8. Chogan

    Hi James, thanks for that. The request to get a warnings added to the migration wizard if a conversion from RDM to VMDK is already gone in.

  9. David Francis

    Any tool to help assist with data migrations is a must have.
    Would love to see this feature added in an up coming release.
    Great post keep them coming.
    David

    1. Shabana

      Is there a way to migrate Microsoft Windows 2003 cluster with RDMs from one storage to another. Keeping the disks as RDMs?

  10. Scott Plater

    This feature would be incredibly useful. I work closely with a healthcare partner who exclusively uses pRDMs for backup purposes. The need to migrate to newer storage arrays every 3-5 years has always been a challenge for smaller customers (expensive array replication software, technical know-how, etc). Enabling pRDM storage migrations would solve this problem, and incent customers to remain loyal to vSphere in the wake of Hyper-V 3…

    1. Douglas Pratt

      Hi Scott – We’re working on it. Depends on whether or not the backup software vendors can determine what LUN and copy are associated for each host’s E: volume. For now we’re stuck doing a SCSI scan so we can tell the backup software what’s what.

  11. Michael

    I would like to have this feature.
    Regarding migration of powered off VMs with pRDM or vRDM, THEY WILL ALWAYS BE CONVERTED TO VMDK. At least in vSphere 4.0 / 4.1.
    That’s what I have seen myself and is also confirmed in official VMware KB:
    http://kb.vmware.com/kb/1005241 (under the Cold Migration section)
    Very annoying (and dangerous) behavior.

  12. John

    RDM to RDM migration isn’t that big a deal when your Array’s already support migration, or storage virtualization (VSP).
    The biggest advantage on this is migrating at this layer you have better visibility and I feel like people are less likely to migrate the wrong direction :)

  13. Chogan

    Thanks for the comment John.
    I’d agree to some extent, but as a previous poster noted, if you are migrating between arrays from different vendors, or between arrays from the same vendor (but you do not have remote replication features), then I think the feature is still an enabler.
    And yes, syncing datastores in the wrong direction isn’t a pleasnt experience. I definitely agree with you there.

  14. Chris

    It would be grateful to have such a tool which can migrate data from RDM to VMFS. If such a tool exist , please let me know ….

  15. B

    I’ve read from several sources, including this explanation, that Storage vMotion with an RDM in virtual compatibility mode moves the mapping file when you choose “Same as Source.”
    I was wondering about the wording in the vSphere Virtual Machine Administration Guide (Chapter 11, page 228) as it says:
    (Use migration with Storage vMotion to relocate a virtual machine’s configuration and virtual disks while the virtual machine is powered on.)
    Same as Source: If you select this option for an RDM disk in virtual compatibility mode, the RDM is converted to a virtual disk.
    I’m currently studying for VCP5 and would appreciate clarification from VMware as to the document wording.

  16. Adr_Ych

    I am quite new to VMware. And i might also have made the mistake of relying on storage snapshots therefore the usage of RDMs and also lumping the various OSe into 1 datastore.
    The bad thing about this is that Database servers have issue with backups. I cannot backup Whole VMs with SQL on RDMs.
    The RDM to VMDK tool will help us move VMs into their own datastore with the data.

  17. Adr_Ych

    I would also like to add…if would be great if we do not have to rely on vMotion or the Standalone Converter tool to migrate VM Server with RDM to VM Server with HDD

  18. Ray S

    Hi Cormac,
    Can you please confirm that the scenarios you cited above are true for ESX 4 only, is it not possible to storage vmotion a vRDM completely to a VMDK
    Thanks,
    Ray

  19. RAMESH GEDDAM

    Hi Cormac,

    Nice post! All the above operations you mentioned is applicable for RDM’s where when a clone operation is done, the destination vmdk max size allowed is below 2TB – 512 bytes in vSphere 5.
    I have a requirement, where I want to move 2TB pRDMs to vmdk’s. What do you suggest? Are there any tools.

    Appreciate your support!

  20. Matt Eynon

    THis would be very useful to move WIndows Server 2003 MSCS or Windows Server 2008 PRDMs between old and new storage.

    1. Jessy Ybarra

      Matt this is exactly what I’m trying to do! Yes, this would be a fantastic tool to have!

  21. Rene Rodriguez

    Not only useful, but a tool that would migrate LUN to LUN and would also leverage VAAI would be extremely useful!

    Im running into an issue right now where an HP EVA Auto Assigned Different LUN ID’s for the same vDisk throughout a vSphere environment.

    1. Jason T

      There’s a step in the vdisk presentation where you can set the LUNID to be the same for each LUN.

      Or you can just create the fdisk then use SSSU to add the lun and specify the desired LUN ID on the command line.

      The base commands are (Linux syntax):

      You can use this to find the names of the Vdisks
      sssu “SELECT MANAGER 10.2.28.2 Username=Administrator Password=${PASS}” “select system ${EVA_NAME}” “ls VDISK”

      You can use this to find the name of the hosts
      sssu “SELECT MANAGER 10.2.28.2 Username=Administrator Password=${PASS}” “select system ${EVA_NAME}” “ls HOST”

      You can use this to find existing LUNs – the LUN ID is the last number in the path that is output.
      sssu “SELECT MANAGER 10.2.28.2 Username=Administrator Password=${PASS}” “select system ${EVA_NAME}” “ls LUN”

      Then use this to bind a present a vdisk to a host, using a lunid that does not appear in the lun listing:
      sssu SELECT MANAGER $COMMAND_VIEW_HOST} Username=Administrator Password=${PASS}” “select system ${EVA_NAME}” “add lun $LUNID host=\”\Hosts\${HOST}\” vdisk=\”${VDISK}\” ”

      Obviously you can put these in a script with looping etc.

  22. Dharshan

    This will be really help full, and even if you introduce a new feature like live storage vmotion of MSCS cluster virtual machine with PRDM would be very usefull, during stroage migrations.

  23. rommel

    Will this feature be implemented? I’m currently facing a pRDM migration from an old SAN to a new SAN and just found out about this limitation.

    1. Cormac

      I am afraid that we cannot make any comments about future development plans. Right now we are simply trying to gauge interest in such a feature were we do implement it sometime in the future.

  24. Jason T

    Yeah pRDM to pRDM migration would rock.
    I found this article, while googling for exactly that.
    We need to migrate our RDMs from one vendor’s array to another vendors – I’m not really sure how,to go about it without downtime and creating new RDMs on the target array.

  25. Mike M.

    An RDM to RDM migration tool would be extraordinarily useful. We frequently run into customers who use RDM’s so they can do snaps/clones on their storage array and they cannot have interference from an added layer such as VMFS. I agree that RDM’s are being used less frequently, but they are not going away any time soon.

  26. Jon K.

    The ability to migrate the data from one RDM LUN to another would be extremely valuable. We are in a constant state of data migration, which takes immense planning preparation, and scripted execution. It would free us from a very arduous process.
    1 – initiate data copy between arrays
    2 – shut down VMs
    3 – resync data
    4 – unmap rdms from virtual machines
    5 – unpresent original luns from hosts
    6 – present new luns to hosts
    7 – map rdms to new luns
    8 – bring vms back online

  27. Deepu

    This is a excellent article. What you have mentioed is what happend to me today. we lost a MSCS 2008 cluster today ( It was a POC cluster, hence not business impact). We powerd off the VM’s and then proceed with the storage migration and happen to see that the mapping files were converetd to VMDK’s. The VM’s were not able to come online after this.

    First of all, Vmware should definetly put a warning message on what is going to happen. it will be really useful if we can have a tool to migrate pRDM’s.

    I’m wonder, how will the people who are having production class MSCS clusters using pRDM’s migrate to a new array if there is a need?

    Are there any documents to migrate a pRDM to a new array when configured with MSCS cluster. Any help in this regard will be appreciated.

  28. Paul Chilton

    Cormac,

    Thanks for writing this post. This was very helpful. I’m planning a fairly large pRDM to pRDM migration currently (470+ pRDM’s). This is from NetApp 7-Mode to NetApp 7-Mode storage. From what I’ve researched, sVmotion will move the VM configuration and pointer files and I’ll have to do the pRDM’s with snapmirror. Having an additional feature in sVmotion that could migrate the pRDM’s hot either at the host layer or by passing the task to the storage layer, would be huge. Thanks.

    Paul

  29. Joe Elliott

    Cormac,

    Is there any new information available concerning whether or not such a feature will be implemented? Thanks!

  30. Rama

    Lot of SAP customers who is moving to virtual from physical workls use mix of VMDKs (for OS) RDMs (for database Data, Log volumes) and this enterprise usecase is going to be around for at least few more years. Currently when the SAP systems are cloned using vCenter or SAP LVM (Landscape Virtualization Management) tool then RDMs get copied/migrated to VMDK files and what is really needed for SAP usecase (SAP System Copy, Clone) is to clone the RDM at storage level to another RDM (using VAAI if supported by storage) and use it in the cloned system. It will be greatly help SAP customers to move from Physical to Virtual if you can implement this feature in the vSphere API as well as command line tool

  31. Justin Applebee

    I would find this tool extremely useful! I just stumbled across this blog yesterday after starting to plan out an HCIS migration from an EMC CX4-120 to a VNX5300. The healthcare organization I work for going through a combination storage migration/expansion of 7 virtual servers running Server 2008 32bit. Each server is currently configured with two 64GB RDM LUN’s that mirror via the proprietary HCIS method. The HCIS vendor insists that these LUN’s remain RDM’s and they will not support VMDK storage. If I could simply storage vMotion these RDM’s from the CX4 to the VNX, it would save me a world of work and planning. We have current support through VMware on our Enterprise Plus licenses.

  32. Bill Oyler

    I agree with everyone else – Storage vMotion of pRDM to pRDM would be a fantastic new feature. I have wanted this capability for many years. As other posters have noted, this would be incredibly useful for MSCS migrations as well as other mission critical apps that are still using pRDMs due to vendor support requirements or other reasons. Typically VMs with pRDMs are some of the most mission-critical VMs in a cluster, so having zero-downtime migration options would be incredibly useful.

  33. Josh

    Really any RDM migration would be fantastic. pRDM to pRDM is great, but vmdk to vRDM would be wonderful as well

  34. seb

    Migrating from Equallogic vRDM to Dell 3600i based vmdk on cold migration fails with:

    Incompatible device backing specified for device ’0′

    which makes no sense (at least to me)

    None of the “solutions” in posts with same error made any difference

  35. Tours News

    It’s really a great and helpful piece of info. I’m satisfied
    that you shared this helpful information with us.
    Please keep us up to date like this. Thank you for sharing.

  36. Travis Williams

    I would be using this feature currently if it were available! I found this post searching for my options as this is exactly what I need currently.

  37. Ashok

    Hi Cormac, Is it possible to re-map an existing pRDM to different controller..? let’s say from SCSI 0:1 to SCSI 1:1..? Would the drive & data stay intact inside OS after remapping..?

    Basically couple of RDMs were mapped to the same controller as VMDKs by mistake which is causing issues with vmotion!

    Thanks

  38. frank

    I would like that, but I want more. I had a clustered SQL Server with mapped RDM’s. We were going to try to convert to vmdk’s. We shutdown 1 side of the cluster and started the migration. Everything seemed to be going great.

    At the end of the Storage Vmotion the drives were converted but we could not see the data on the drives. Windows Cluster manager locked the drives and could not access the data.

    We were able to build a new server, and create new vmdk’s and re-establish the databases.

Comments are closed.