Technical

vSphere Authentication, Microsoft Active Directory LDAP, and Event ID 2889

VMware AppDefense IconIf you’re running VMware vSphere and using Microsoft Active Directory (AD) for authentication you’ve likely been party to the confusion around the LDAP Channel Binding & Signing changes that were proposed by Microsoft, first as a change to the shipping defaults, and now as a recommended hardening step. We at VMware support hardening IT systems, especially ones like Active Directory that are such rich targets for attackers. However, changing existing systems has nuances, so this post is intended to help answer a lot of the questions that continue to bubble up, and fill in gaps between the resources out there.

What is the change that Microsoft wants Active Directory admins to make?

Microsoft would like Active Directory administrators to require LDAP signing & LDAP channel binding. These improve the security of connections to the LDAP servers that are part of Active Directory by helping to prevent “man in the middle” attacks where an attacker could intercept communications between the systems.

We wrote about these changes in our post, “VMware vSphere & Microsoft LDAP Channel Binding & Signing (ADV190023).” That post has been updated to reflect current guidance.

Does VMware vSphere support this change?

It will depend on your configuration. Supported versions of vSphere have been tested and this is what we have found:

  • AD over LDAP: If your authentication is configured as “AD over LDAP” these changes to Active Directory will break your authentication. This is expected – AD over LDAP is not natively secure. Switch to AD over LDAPS or Identity Federation instead.
  • AD over LDAPS: You are fine, your authentication communications are secure.
  • Integrated Windows Authentication (IWA): Not completely compatible. Authentication is secure and will continue working but you will be unable to search the Active Directory, because searching is done over an LDAP (not LDAPS) connection that does not sign the connections. This means that adding new AD users and groups to SSO may be problematic. We are investigating improvements to Integrated Windows Authentication that might help with this, but there is not a timeframe for this. If you have immediate needs please consider switching to AD over LDAPS. You may also be seeing Event ID 2889 log entries — please read below for more information.
  • Identity Federation: You are fine, your authentication communications are secure.

My Active Directory Domain Controllers have auditing enabled and are getting Event ID 2889 log entries on connections from our vCenter Server. Does this mean it’s insecure?

It depends on what method you’re using for authentication:

  • AD over LDAP: Yes, it is insecure. Switch to a connection type that protects communications with TLS, like AD over LDAPS or Identity Federation.
  • AD over LDAPS: You will not see Event ID 2889 log entries for this method.
  • Integrated Windows Authentication (IWA): Check out VMware KB 78644. Integrated Windows Authentication uses GSSAPI & Kerberos to authenticate users and uses credential sealing with SASL to protect credentials. It also uses Kerberos tokens to authenticate the LDAP connection it uses for searching Active Directory. As such, it is not sending credentials in the clear. In addition to authentication, in IWA configuration, vSphere queries Active Directory via LDAP on port 389/tcp for other, non-credential data, such as group membership and user properties. It uses sealing (encryption) to satisfy the protection against the man-in-the-middle attack, but Windows logs Event ID 2889 anyway. For more details please see the “Logging anomaly of Event ID 2889” section of Microsoft’s How to enable LDAP signing in Windows Server
  • Identity Federation: You will not see Event ID 2889 log entries for this method.

We connect our ESXi hosts to Active Directory, is that secure?

Yes. That uses the same techniques for authentication as Integrated Windows Authentication. We continue to recommend that ESXi management activities be directed through the Role-based Access Controls (RBAC) present in vCenter Server, rather than administration activities happening directly on ESXi hosts.

I read that Integrated Windows Authentication is deprecated in vSphere 7. How will I connect to Active Directory?

Deprecation means we intend to remove the feature, but it is still there & fully supported for now. We cover this thoroughly in our post “vSphere 7 – Integrated Windows Authentication (IWA) Deprecation.

How can I switch my authentication methods?

You can remove the old authentication method and then recreate it with a different protocol using the same domain information. A great example of this is shown in TAM Lab 048, done by Bill Hill, one of our Technical Account Managers. His video shows switching vSphere from LDAP to LDAPS.

We always encourage vSphere Admins to test changes before they make them in their production environments. A great way to do that is with nested ESXi. Deploy a small vCenter Server for testing and install ESXi in a VM for that vCenter Server to manage it (when you’re configuring the new VM choose “ESXi 6.5 and newer” from the list of operating systems). Once it is set up you can shut it all down and take a snapshot, so that if the environment gets messy you can restore it to a working & clean state. While we do not support nested ESXi directly, it is how the Hands-on Labs work, and how many of us do our testing. It’s a great way to test things like authentication.

What does VMware recommend?

Moving forward, AD over LDAPS and Identity Federation are the two primary recommendations for connecting vSphere to Active Directory.

Conclusion

When it comes to vSphere & security we’re working to make it easy for vSphere admins to be secure, and to make vSphere secure by default. Change can be tough, but security is a process, and as our adversaries change their tactics we need to grow, too, in order to protect our systems and data.

As always, thank you for being our customer, and please let us know how we can help make your lives and infrastructure more secure.