For  those visiting VMworld, come and meet VMware Trust and Assurance (which includes VMware Security Response Center) in Las Vegas next week or in Barcelona in three weeks from now. Bring your questions and concerns on security issues in our products and services, and how we address these. We would also like to have feedback on the VMware Security Advisories  and our patch policies.

How to find us? We  are accepting 1:1 meetings at VMworld. If  you would like to schedule a meeting please contact your Technical Account Manager with a general idea of what you would like to speak with us about and we will  schedule time with you. Alternatively just come and meet us; we are stationed in the Listening Post located in the VM Village. This is the lounge area with seats and games on the top floor.

We share the Listening Post with other teams and they would be delighted with your visit as well! They are Support, Customer advocacy, and the Information Experience, Quality Assurance, and Product Globalization teams of the VMware R&D Central Organization.

Update August 28
We are ready to roll tomorrow when VMworld opens! Come by the Listening Post in the VM Village and talk to us about your challenges and suggestions regarding the security of our products. Our co-workers will be there to discuss quality, support, documentation, and globalization.

Today, VMware has released the following new security advisory:

VMSA-2017-0014 – VMware NSX-V Edge updates address OSPF Protocol LSA DoS

The advisory documents a hard to exploit denial of service vulnerability in the implementation of the OSPF protocol in NSX-V Edge (CVE-2017-4920). This issue is present due to incorrect handling of link-state advertisements (LSA). NSX-V Edge 6.2.8 and NSX-V Edge 6.3.3 address the issue.

We would like to thank Adi Sosnovich, Orna Grumberg and Gabi Nakibly for reporting this issue to us.

Please sign up to the Security-Announce mailing list to receive new and updated VMware Security Advisories.

Customers should review the security advisory and direct any questions to VMware Support.

The Black Hat USA 2017 conference includes a talk by Ofri Ziv of Guardicore Labs about using the VIX API to obtain privileged access to a guest operating system with unexpectedly low VIM API permissions. VMSA-2017-0012 contains details of impacted versions and workarounds. The Common Vulnerabilities and Exposures project has assigned CVE-2017-4919 for this issue, with thanks to Ofri Ziv and Itamar Tal of Guardicore for discovering and reporting it.

Permissions

Understanding the privilege escalation here requires understanding the interaction between several vSphere permissions.

• Host.Config.AdvancedConfig. This permission amounts to full host control. Setting host advanced configurations is equivalent to adjusting VIB acceptance levels or disabling firewalls.
  • In proof-of-concept code, authors used this permission to enable the “shared secret” mechanism at the host level. VMware considers use of this permission equivalent to full control (e.g. root) and thus using it in an attack chain does not amount to a privilege escalation.
  • However, some VMware products (SRM, VUM, and VIN) operate with this permission, and the setting is host-global. When such software enables “shared secret”, they allow a lower privilege user to gain guest access for the window when it is enabled.
  • The issue reporters argue that a typical datacenter would frequently enable “shared secret”. VMware believes such enablements are uncommon, and collisions are unlikely and can be mitigated. But as the collision is possible, VMware is treating this as an exploitable issue.
• VirtualMachine.Config.AdvancedConfig. This is the “vulnerable” permission. Virtual machine advanced configurations are a lower privilege level than host advanced configurations, and this setting should have been safe to give to lower-privilege users.
• VirtualMachine.Interact.GuestControl. This is the “guest control” permission; users with this permission may establish VIX API connections and thus present some form of authentication.

To compute a CVSS v3 score, VMware recommends “Access Vector: Local” to represent the need to be on the management network (equivalent to local access in a non-virtual environment), and “Attack Complexity: High” to represent the difficulty in bypassing the Host.Config.AdvancedConfig permission.

Impact and Workarounds

Accessing a virtual machine should require two authentications: a VIM API authentication to provide a “route” to a virtual machine, and a VIX API guest authentication to access the virtual machine itself. The security issue here bypasses only the second authentication; the first authentication remains a requirement.

Before worrying about this security issue, an organization should first confirm that their configuration does assume a separation of privilege, with some users being given rights to configure or access virtual machines but being denied rights to configure hosts, and vice versa. Such least-privilege configurations are a best-practice recommendation, but many deployments have so few vSphere users that all users operate with full permissions – at which point there is no escalation from a low-privilege user to a high-privilege user.

The escalation of privilege here requires both an ESXi host version capable of reading “shared secret” information, and a VMware Guest Tools version willing to accept “shared secret” authentication. All current versions of VMware ESXi are impacted by this issue. VMware Guest Tools prior to version 10.1.0 (released October 2016) are impacted by this issue, as beginning with Tools 10.1.0 we have disabled the “shared secret” authentication mechanism.

VMware has two recommended workarounds. Either is sufficient.

1. Upgrade VMware Guest Tools to at least version 10.1.0, in combination with running ESXi at least version 6.0. Without an in-guest endpoint willing to accept “shared secret” authentication, there is no privilege escalation.
   • However, some older products are incompatible with newer VMware Guest Tools (see “known issues” in the 10.1.0 release notes). Those products are incompatible precisely because they depend on “shared secret” authentication.
   • VMware Guest Tools version 9.10 and later have an in-guest configuration option to disable this functionality, which is enabled by default. Version 10.1.0 changes the default to disabled. See KB 2151027 for further details.
2. Remove the VirtualMachine.Interact.GuestControl permission from vSphere users, applicable to all vSphere versions. This permission enables VIX API access, which is necessary to present any credential at all via the VIX API (whether “shared secret”, username/password, or SAML token).
   • Most vSphere customers do not use the VIX API (and those that do, are probably aware of it). Any existing users of the VIX API should prioritize moving to the replacement Guest Operations VIM API discussed below.

VMware believes the above workarounds are sufficient for most customers. In particular, either a Tools upgrade or removing GuestControl are straightforward to apply in most scenarios. Regarding ESXi releases:

• ESXi 5.5 will retain “shared secret” behavior as a “mis-feature”; the only workaround is to disable the VIX API entirely via the GuestControl permission. This is ultimately a recognition that in vSphere 5.5, guest authentication was designed as a mechanism to run guest operations as a particular user and not as a security barrier. As described below, adequate security mechanisms to provide a barrier are not available architecturally until vSphere 6.0.
• Any future major release of ESXi will have “shared secret” authentication removed entirely, due to VIX API end-of-life as described below.

VIX API: a slow path to deprecation

The VIX API is an old VMware API written in C, primarily intended for automation use cases. Originally created for Workstation 6 (released 2008), VIX version 1.6.2 (released 2010) added vSphere support.

That heritage embedded some toxic assumptions. The automation use case around 2008 envisioned deploying VMs under API control, and “obviously” the user invoking automation would have full control of those VMs to run processing tasks – so the VIX API offered full guest control without authentication.

Around 2010, VMware recognized that holistically, the VIX API bypassed too many important checks offered by the VIM API and needed to be removed. However, as some bloggers had noticed, the VIX API was unique in being the only mechanism to run a command in guest, which turns out to be a very powerful tool for automating virtualized software. Thus, we added Guest Operations support in the VIM API, a project that started vSphere 5.0 and culminated with vSphere 6.0’s support for SAML tokens to share pre-authenticated access (allowing a guest to opt-in and map VIM users to guest users as a bridge between two security domains). Simultaneously, we moved the guest-unauthenticated access to be off by default and enabled only via higher-privilege “shared secret” settings, to prevent new usages of this functionality.

Deprecation of the VIX API was announced in 2011 with VIX version 1.11, indicating “VIX will be unsupported in the second major release after vSphere 5”. Nominally those next two releases were ESXi 5.5 and 6.0, with VIX scheduled for removal in ESXi 6.5. That didn’t happen.

Now for the bad news. As anyone familiar with IT or software engineering knows, anytime a feature is deprecated, somebody will build a new product depending on it before the deprecation window expires. We initially tried disabling VIX’s unauthenticated guest access entirely… and regressed many VMware products, so needed another release. SRM ported to the Guest Operations API in the 6.5 release. VUM dropped the only feature which used VIX (appliance upgrade). vRealize Infrastructure Navigator chose to end-of-life, with the replacement being the Service Discovery Management Pack. As most of those were completed late in the release, we were uncomfortable removing VIX (or even VIX’s “shared secret”) in the 6.5 release, so delayed removal to the first major release after 6.5.

Astute observers will notice the dilemma here. On one hand, bypassing guest authentication is increasingly an untenable security posture. On the other hand, removing this functionality will break mission-critical functionality like SRM (for disaster recovery). And there are no good answers.

Conclusions

Deprecating and removing concerning code is always a tradeoff between the risk of unknown security bugs, and unknown functional regression – a hard tradeoff to make for enterprise products where functional regressions can bring an entire business down. The removal of the entire VIX API will occur promptly as planned; thanks to this security research, the security costs of any further delay are more obvious.