Update: VMSA-2021-0028 continues to evolve to follow updated recommendations from the Apache Software Foundation. There are also regular scheduled updates and a growing list of Q&A over at the VMSA-2021-0028: Questions & Answers page.
On December 10, 2021 VMware released VMSA-2021-0028 to track the impact of an Apache Software Foundation security advisory for their extremely popular Log4j Java logging component on VMware products and services. An updated workaround for CVE-2021-44228, as well as guidance on a second vulnerability, CVE-2021-45046 was released by the Apache Software Foundation on December 14. These advisories outline critical remote code execution vulnerabilities in the Log4j component, scoring 10 of 10 on the Common Vulnerability Scoring System (CVSS) for all affected VMware products.
This update needs your immediate attention because the log4j component is used by many vendors and software packages, not just in VMware products, but also for all other software in your environment. The updated advisory means that all systems should be reevaluated. Threat intelligence experts across the industry are observing active attacks based on these vulnerabilities, especially against workloads accessible from the Internet.
VMware Security Advisory VMSA-2021-0028 is the source of truth for VMware’s response to this situation and these vulnerabilities, and has information about affected products, workarounds, and patches as they become available.
VMSA-2021-0028: Questions & Answers is a corollary to the VMSA and addresses common customer questions. It also includes a longer list of resources, such as the CISA.gov Log4j Guidance, and common questions that are asked. Please visit and bookmark that page.
VMware Security has a growing list of terrific resources dedicated to detecting, containing, and understanding the Log4j and Log4Shell vulnerabilities, including:
Please subscribe to our Security Advisories mailing list (found on the right side of the VMSA page), and revisit the VMSA-2021-0028 advisory and this page periodically to get the latest information. You can also subscribe to Knowledge Base articles so that you are notified when an update occurs.
VMware Cloud on AWS, Workspace ONE, and other VMware Cloud services customers are being actively protected with mitigations and patches, installed as part of the VMware commitment to security in the cloud shared responsibility model. Customers will continue to see maintenance notifications where appropriate and can check service status at any time using https://status.vmware-services.io/ and https://status.workspaceone.com/. Some VMware Cloud on AWS customers with overly permissive management gateway firewall rules have had action taken to reduce their exposure from scanning and exploit activity occurring across the Internet.
Customers with on-premises components for VMware Cloud & hosted services should review the VMSA to determine if workarounds need to be implemented on those appliances. This includes appliances for hybrid cloud tools like HCX, Cloud Gateway, Workspace ONE Access, and so on.
NSX Intrusion Prevention & Threat Analysis functionality, available to on-premises and VMware Cloud on AWS Advanced Firewall customers, is constantly being updated to detect and block traffic with Log4Shell exploit signatures.
Who is affected?
A VMware Security Advisory will always list the specific supported products and versions that are affected. This situation is developing, and the VMSA will be updated with more information.
When do I need to do something about this?
Immediately. The ramifications of this vulnerability are serious for any system, especially ones that accept traffic from the open Internet. Because of the suddenness of this “zero-day” disclosure, affected software is still being updated. This gives attackers the advantage.
Organizations that practice change management using the ITIL definitions of change types would consider this an “emergency change.” All environments are different, have different tolerance for risk, and have different security controls and defense-in-depth to mitigate risk, so the decision on how to proceed is up to you. However, given the severity, we strongly recommend that you act.
Why am I affected?
CVE-2021-44228 is in an Apache Software Foundation component called “log4j” that is used to log information from Java-based software. It has industry-wide impact. The vulnerability is critical, rated 10 out of 10 on the CVSS 3.1 scoring scale, because it is an unauthenticated remote code execution (RCE) vulnerability. It allows attackers to run commands on affected systems and workloads by simply getting them to log a specific string. The library that does the logging interprets that string as a command, instead of just writing it to the log. For example, an attacker might be able to use a login page, placing the attack string in the username field where they know it will be logged.
Every piece of software that has ever used log4j is potentially vulnerable. VMware uses log4j as well, which is why we have issued VMSA-2021-0028. However, this vulnerability also affects customer workloads. Customers need to assess their entire environment for use of log4j, in both infrastructure and workloads, and remediate it as soon as possible either through patches or workarounds.
The vulnerability was announced suddenly, as a “zero-day” vulnerability, taking the industry by surprise. Normally a vulnerability is reported privately to the software maintainers, who then have time to repair the issue and release an update, so attackers don’t gain a temporary advantage. With a zero-day disclosure like this one, attackers have an advantage while software maintainers scramble to develop the fix.
Regardless of the timing, the ubiquitous use of log4j guaranteed the vulnerability would have enormous impact, no matter when it surfaced.
A remote code execution (RCE) vulnerability is where an attacker who can reach the affected software over the network can execute commands on it and bypass the security controls in place. This leaves perimeter and microsegmented firewall controls as the last line of defense until the vulnerability is remediated. Beyond firewalling, most organizations employ good security strategies, such as defense-in-depth, zero trust and isolation, good password and account hygiene, etc. and usually these tactics help immensely when a security vulnerability is disclosed. Every vulnerability disclosure is different, though, and organizations should take steps to proactively & positively confirm that there are no open attack vectors that make them vulnerable.
Knowing that even a failed login to an impacted system or workload can be used as an attack vector, organizations who have placed affected systems and workloads on networks that are directly accessible from the Internet, or even from a wide audience inside their own organization, may not have these lines of defense and should audit their systems for compromise. They should also take steps to implement additional security controls to limit the scope of traffic until a workaround or patch is implemented. Ransomware groups have repeatedly demonstrated to the world that they are able to compromise corporate networks while remaining extremely patient, waiting for a new vulnerability in order to attack from inside a network. This is universally observed behavior throughout the industry and informs our suggestions here. Organizations may want to consider additional security controls and isolation between their IT infrastructure and other corporate networks as part of an effort to implement modern zero-trust security strategies.
What should I do to protect myself?
First, check VMSA-2021-0028 to ensure you are running an affected version of VMware software. If a workaround or patch is listed, apply it. If not, be sure to check back regularly as new information is being added regularly. Like other software vendors who use log4j in their products, VMware found out about this in a zero-day scenario and is now working nonstop to help protect customers and test updates.
Check your other workloads, too. Log4j is an incredibly popular logging library. It is especially important to protect anything that accepts traffic from the Internet.
You may have other security controls in your environment that can help protect you until you are able to patch. Use network perimeter access controls or NSX IDS/IPS and NDR technologies to detect and contain attacks against your workloads. For Cloud Infrastructure products like VMware vSphere, VMware Cloud Foundation, and VMware Cloud, as well as cloud add-on components like the HCX, Site Recovery Manager, NSX-T, and Cloud Gateway Appliances, we strongly suggest limiting access to management interfaces to only Virtualization Admins. Drive any direct workload management activity through the VM network connections instead of the VM console. This simplifies access control and makes the RDP or ssh management traffic subject to other security controls, such as IDS/IPS and monitoring.
Please ensure that network perimeter access controls are as restrictive as they can be. “Any/any” rules that make system access easy for you also make access easy for attackers and should be avoided.
Critical security advisories are always difficult conversations, and unfortunately, part of the landscape in IT. They are no easier in a zero-day situation when all critical stakeholders are surprised at once. Please check back with the VMSA, subscribe to the VMware Security Advisory mailing list, and let us know what other questions we can answer. We appreciate you very much. Thank you for being our customers, and we wish the very best for you and others around you.
