VMware

ESX patching questions | Main | VMware at RSA Conference 2008

February 25, 2008

Keeping Your VMotion Traffic Secure

Recently a researcher published a proof-of-concept called Xensploit which allows an attacker to view or manipulate a VM undergoing live migration (i.e. VMware’s VMotion) from one server to another. This was shown to work with both VMware’s and Xen’s version of live migration. Although impressive, this work by no means represents any new security risk in the datacenter. It should be emphasized this proof-of-concept does NOT “take over the hypervisor” nor present unencrypted traffic as a vulnerability needing patching, as some news reports incorrectly assert. Rather, it a reminder of how an already-compromised network, if left unchecked, could be used to stage additional severe attacks in any environment, virtual or physical.

On an insecure network, man-in-the-middle attacks can target both virtual and physical machines. The techniques published are novel in that they go after the contents of migrating VM memory to target credentials and data, rather than going after similar information flowing across internal network transactions. Putting aside the question of whether it’s even worthwhile to target memory instead of network traffic directly, the sensitivity of VM memory was never the question.

Encryption of all data-in-transit is certainly one well-understood mitigation for man-in-the-middle attacks.  But the fact that plenty of data flows unencrypted within the enterprise – indeed perhaps the majority of data – suggests that there are other adequate mitigations. Unencrypted VMotion traffic is not a flaw, but allowing VMotion to occur on a compromised network can be. So this is a good time to re-emphasize hardening best practices for VMware Infrastructure and what benefit they serve in this scenario.

  1. The most important VMotion best practice is to isolate your VMotion activity from all production network traffic. The current design of VMotion assumes that the VMotion network is secure within a data center, certainly within a rack or set of adjacent racks.  In a typical situation, servers in one or more co-located racks would each have one or two network cards dedicated for VMotion; these would be connected to a switch or VLAN that has no other endpoints connected.

    Isolating VMotion takes away that most common of staging points for man-in-the-middle: some unpatched box anywhere on the production network that has already been taken over by malware.  Indeed why any non-ESX box, compromised or not, would be on this network at all would be immediately in question. The researcher’s assumption is that long-haul VMotion over wide area networks might become popular in the future. However, most companies today already use encrypted links for inter-datacenter traffic.

  2. Tightly restrict access to VI administrative accounts and roles.  With VMotion isolated, a virtual rogue presence is more plausible than a physical one, but even a compromised guest VM does not have a virtual NIC on the VMotion network, only on the production network. Therefore the rogue VM must be configured in VI to have a vNIC on the VMotion network.

  3. Don’t enable promiscuous mode on vswitches. Unlike a physical network card, someone who has taken over a guest VM cannot cannot configure a vNIC to be promiscuous. Another VI admin setting, promiscuous mode (off by default) is configured on the virtual switch port separately from a VM.  Also, to manipulate rather than snoop, the proof-of-concept technique requires traffic actually route through the rogue VM, which would not occur naturally on the vswitch.

Warren Wu
Security Product Management