Posted by Cormac Hogan
Technical Marketing Manager (Storage)
On a number of occasions recently, I had to investigate what happened to a Raw Device Mapping (RDM) when:
- The VM to which the RDM was attached was Storage vMotion'ed (VM Powered On)
- The VM to which the RDM was attached was Cold Migrated (VM Powered Off)
Some of you may even have been following along the comments in some of my previous postings. Well, this is what I observed, testing with both pRDMs and vRDMs.
VM with Physical (Pass-Thru) RDMs (Powered On – Storage vMotion):
- If I try to change the format to thin or thick, then no Storage vMotion allowed.
- If I chose not to do any conversion, only the pRDM mapping file is moved from the source VMFS datastore to the destination VMFS datastore – the data stays on the original LUN.
VM with Virtual (non Pass-Thru) RDMs (Power On – Storage vMotion):
- On a migrate, if I chose to covert the format in the advanced view, the vRDM is converted to a VMDK on the destination VMFS datastore.
- If I chose not to do any conversion, only the vRDM mapping file is moved from the source VMFS datastore to the destination VMFS datastore – the data stays on the original LUN (same behaviour as pRDM)
VM with Physical (Pass-Thru) RDMs (Powered Off – Cold Migration):
- On a migrate, if I chose to change the format (via the advanced view), the pRDM is converted to a VMDK on the destination VMFS datastore.
- If I chose not to do any conversion, only the pRDM mapping file is moved from the source VMFS datastore to the destination VMFS datastore – the data stays on the original LUN
VM with Virtual (non Pass-Thru) RDMs (Power Off – Cold Migration):
- On a migrate, if I chose to covert the format in the advanced view, the vRDM is converted to a VMDK on the destination VMFS datastore.
- If I chose not to do any conversion, only the vRDM mapping file is moved from the source VMFS datastore to the destination VMFS datastore – the data stays on the original LUN (same behaviour as pRDM).
As you can see, there are 3 occasions when an RDM could be converted to a VMDK. Perhaps the most surprising is the fact that a pRDM could be converted to a VMDK, when a cold migration of the VM is attempted, and the format is changed.
I've since asked our engineering team to put a warning into the migration wizard in vSphere to highlight that this is what's going to happen. Right now, you don't get any warning about this.
I wanted to finish this post with a question to the community. How useful would you find an RDM -> RDM migration tool, i.e. the ability to move data from one LUN to another LUN via the vSphere migration wizard? Please leave me a note in the comments if you think you would use this?
Get notification of these blogs postings and more VMware Storage information by following me on Twitter: @VMwareStorage
I think it would be pretty useful.
The new features available with vsphere 4 and even more with vsphere 5 make less and less sense to use RDM or at least they reduce a lot the reasons and the scenarios why they should be used.
Such a tool would be useful for those customers who used RDMs in the past and could use a migration to a newer platform for say goodbye to RDMs and improve managebility, so in my opinion it would be a great thing!
As I requested this pRDM migration feature, I want it.
… and now add VAAI-support to your RDM-to-RDM-tool and we are ready to go. I think such a tool would be a nice-to-have for some(!) customers. If its easy to develop, why not.
Wouldn’t use it. We’ve already converted our vRDM’s to VMDK’s.Largest was 1.5TB. No downtime, no issues. Wish the rest of the organisation could understand the magic that gets done behind the scenes.
Great article Cormac, thanks for providing this info.
I look forward to retiring RDM’s with VMFS5 and the info here.
You are talking about pRDM to pRDM migration, if I am reading this correctly.
In my opinion this would be incredibly useful, but only if done on a live VM. It would allow a migration from one array to another, or from a slower LUN to a faster one within the same array. Or from a RAID10 LUN to a RAID5 one… You get the picture. Even more useful if the migration tool display the VASA information somehow.
It would only have limited uses if the migration cannot be done live, as this kind of LUN to LUN migration could be done live using OS features. But doing it from inside VMware could be simpler, easier and neater.
Thanks for all the comments so far. Right now, I’m just trying to figure out if there is a demand for such a feature, so keep the comments coming (doesn’t matter if you’re in the yeah or nay camp).
Anything to make any process easier is worth doing. I didn’t realize that doing some of these storage migrations hot or cold would possibly result in changing it from a RDM to a vmdk format so at the very minimum warnings in the svmotion wizard would be a good thing. A migration wizard to do these things would be a little clearer possibly when it detects a RDM attached to a VM vs. a svmotion wizard ? Its slightly different but still important to be able to do the migration in some cases which it sounds like is possible.
Thanks!
Hi James, thanks for that. The request to get a warnings added to the migration wizard if a conversion from RDM to VMDK is already gone in.
Any tool to help assist with data migrations is a must have.
Would love to see this feature added in an up coming release.
Great post keep them coming.
David
Of course it would be super useful to have RDM-RDM SVMotion!
Is there a way to migrate Microsoft Windows 2003 cluster with RDMs from one storage to another. Keeping the disks as RDMs?
Not at this time. You will have to do a backup and restore.
This feature would be incredibly useful. I work closely with a healthcare partner who exclusively uses pRDMs for backup purposes. The need to migrate to newer storage arrays every 3-5 years has always been a challenge for smaller customers (expensive array replication software, technical know-how, etc). Enabling pRDM storage migrations would solve this problem, and incent customers to remain loyal to vSphere in the wake of Hyper-V 3…
Hi Scott – We’re working on it. Depends on whether or not the backup software vendors can determine what LUN and copy are associated for each host’s E: volume. For now we’re stuck doing a SCSI scan so we can tell the backup software what’s what.
Great use case – thank you Scott.
I would like to have this feature.
Regarding migration of powered off VMs with pRDM or vRDM, THEY WILL ALWAYS BE CONVERTED TO VMDK. At least in vSphere 4.0 / 4.1.
That’s what I have seen myself and is also confirmed in official VMware KB:
http://kb.vmware.com/kb/1005241 (under the Cold Migration section)
Very annoying (and dangerous) behavior.
RDM to RDM migration isn’t that big a deal when your Array’s already support migration, or storage virtualization (VSP).
The biggest advantage on this is migrating at this layer you have better visibility and I feel like people are less likely to migrate the wrong direction
Thanks for the comment John.
I’d agree to some extent, but as a previous poster noted, if you are migrating between arrays from different vendors, or between arrays from the same vendor (but you do not have remote replication features), then I think the feature is still an enabler.
And yes, syncing datastores in the wrong direction isn’t a pleasnt experience. I definitely agree with you there.
It would be grateful to have such a tool which can migrate data from RDM to VMFS. If such a tool exist , please let me know ….
I’ve read from several sources, including this explanation, that Storage vMotion with an RDM in virtual compatibility mode moves the mapping file when you choose “Same as Source.”
I was wondering about the wording in the vSphere Virtual Machine Administration Guide (Chapter 11, page 228) as it says:
(Use migration with Storage vMotion to relocate a virtual machine’s configuration and virtual disks while the virtual machine is powered on.)
Same as Source: If you select this option for an RDM disk in virtual compatibility mode, the RDM is converted to a virtual disk.
I’m currently studying for VCP5 and would appreciate clarification from VMware as to the document wording.
Hi B,
Looks like a typo in the admin guide.
I’ll feed this back.
Thanks you
Cormac
I am quite new to VMware. And i might also have made the mistake of relying on storage snapshots therefore the usage of RDMs and also lumping the various OSe into 1 datastore.
The bad thing about this is that Database servers have issue with backups. I cannot backup Whole VMs with SQL on RDMs.
The RDM to VMDK tool will help us move VMs into their own datastore with the data.
I would also like to add…if would be great if we do not have to rely on vMotion or the Standalone Converter tool to migrate VM Server with RDM to VM Server with HDD
I really interested in this topics and my friends also. I must shear this with my friends.
http://www.taskcanon.com
Hi Cormac,
Can you please confirm that the scenarios you cited above are true for ESX 4 only, is it not possible to storage vmotion a vRDM completely to a VMDK
Thanks,
Ray
These test were done on 5.0 Ray.
When migrating a vRDM, if you choose to change the format, you can convert it to a VMDK.
I would find that tool very useful.
Hi Cormac,
Nice post! All the above operations you mentioned is applicable for RDM’s where when a clone operation is done, the destination vmdk max size allowed is below 2TB – 512 bytes in vSphere 5.
I have a requirement, where I want to move 2TB pRDMs to vmdk’s. What do you suggest? Are there any tools.
Appreciate your support!
That will not be possible at this time Ramesh. The VMDK size is still limited to 2TB – 512bytes even in the 5.1 vSphere release.
THis would be very useful to move WIndows Server 2003 MSCS or Windows Server 2008 PRDMs between old and new storage.
Not only useful, but a tool that would migrate LUN to LUN and would also leverage VAAI would be extremely useful!
Im running into an issue right now where an HP EVA Auto Assigned Different LUN ID’s for the same vDisk throughout a vSphere environment.
There’s a step in the vdisk presentation where you can set the LUNID to be the same for each LUN.
Or you can just create the fdisk then use SSSU to add the lun and specify the desired LUN ID on the command line.
The base commands are (Linux syntax):
You can use this to find the names of the Vdisks
sssu “SELECT MANAGER 10.2.28.2 Username=Administrator Password=${PASS}” “select system ${EVA_NAME}” “ls VDISK”
You can use this to find the name of the hosts
sssu “SELECT MANAGER 10.2.28.2 Username=Administrator Password=${PASS}” “select system ${EVA_NAME}” “ls HOST”
You can use this to find existing LUNs – the LUN ID is the last number in the path that is output.
sssu “SELECT MANAGER 10.2.28.2 Username=Administrator Password=${PASS}” “select system ${EVA_NAME}” “ls LUN”
Then use this to bind a present a vdisk to a host, using a lunid that does not appear in the lun listing:
sssu SELECT MANAGER $COMMAND_VIEW_HOST} Username=Administrator Password=${PASS}” “select system ${EVA_NAME}” “add lun $LUNID host=\”\Hosts\${HOST}\” vdisk=\”${VDISK}\” ”
Obviously you can put these in a script with looping etc.
This will be really help full, and even if you introduce a new feature like live storage vmotion of MSCS cluster virtual machine with PRDM would be very usefull, during stroage migrations.
Will this feature be implemented? I’m currently facing a pRDM migration from an old SAN to a new SAN and just found out about this limitation.
I am afraid that we cannot make any comments about future development plans. Right now we are simply trying to gauge interest in such a feature were we do implement it sometime in the future.
Yeah pRDM to pRDM migration would rock.
I found this article, while googling for exactly that.
We need to migrate our RDMs from one vendor’s array to another vendors – I’m not really sure how,to go about it without downtime and creating new RDMs on the target array.
An RDM to RDM migration tool would be extraordinarily useful. We frequently run into customers who use RDM’s so they can do snaps/clones on their storage array and they cannot have interference from an added layer such as VMFS. I agree that RDM’s are being used less frequently, but they are not going away any time soon.
The ability to migrate the data from one RDM LUN to another would be extremely valuable. We are in a constant state of data migration, which takes immense planning preparation, and scripted execution. It would free us from a very arduous process.
1 – initiate data copy between arrays
2 – shut down VMs
3 – resync data
4 – unmap rdms from virtual machines
5 – unpresent original luns from hosts
6 – present new luns to hosts
7 – map rdms to new luns
8 – bring vms back online
This is a excellent article. What you have mentioed is what happend to me today. we lost a MSCS 2008 cluster today ( It was a POC cluster, hence not business impact). We powerd off the VM’s and then proceed with the storage migration and happen to see that the mapping files were converetd to VMDK’s. The VM’s were not able to come online after this.
First of all, Vmware should definetly put a warning message on what is going to happen. it will be really useful if we can have a tool to migrate pRDM’s.
I’m wonder, how will the people who are having production class MSCS clusters using pRDM’s migrate to a new array if there is a need?
Are there any documents to migrate a pRDM to a new array when configured with MSCS cluster. Any help in this regard will be appreciated.
Cormac,
Thanks for writing this post. This was very helpful. I’m planning a fairly large pRDM to pRDM migration currently (470+ pRDM’s). This is from NetApp 7-Mode to NetApp 7-Mode storage. From what I’ve researched, sVmotion will move the VM configuration and pointer files and I’ll have to do the pRDM’s with snapmirror. Having an additional feature in sVmotion that could migrate the pRDM’s hot either at the host layer or by passing the task to the storage layer, would be huge. Thanks.
Paul
Cormac,
Is there any new information available concerning whether or not such a feature will be implemented? Thanks!
Lot of SAP customers who is moving to virtual from physical workls use mix of VMDKs (for OS) RDMs (for database Data, Log volumes) and this enterprise usecase is going to be around for at least few more years. Currently when the SAP systems are cloned using vCenter or SAP LVM (Landscape Virtualization Management) tool then RDMs get copied/migrated to VMDK files and what is really needed for SAP usecase (SAP System Copy, Clone) is to clone the RDM at storage level to another RDM (using VAAI if supported by storage) and use it in the cloned system. It will be greatly help SAP customers to move from Physical to Virtual if you can implement this feature in the vSphere API as well as command line tool
I would find this tool extremely useful! I just stumbled across this blog yesterday after starting to plan out an HCIS migration from an EMC CX4-120 to a VNX5300. The healthcare organization I work for going through a combination storage migration/expansion of 7 virtual servers running Server 2008 32bit. Each server is currently configured with two 64GB RDM LUN’s that mirror via the proprietary HCIS method. The HCIS vendor insists that these LUN’s remain RDM’s and they will not support VMDK storage. If I could simply storage vMotion these RDM’s from the CX4 to the VNX, it would save me a world of work and planning. We have current support through VMware on our Enterprise Plus licenses.