Home > Blogs > Support Insider > Category Archives: Patches

Category Archives: Patches

NSX for vSphere Field Advisory – July 2016 Edition

This blog has been updated to reflect new information as it was provided. Changes are marked with an *.

VMware NSX for vSphere 6.2.3 Update

  • NSX for vSphere 6.2.3 has an issue that can affect both new NSX customers as well as customers upgrading from previous versions of NSX. The NSX for vSphere 6.2.3 release has been pulled from distribution. The current version available is NSX for vSphere 6.2.2, which is the VMware minimum recommended release.  Refer to KB 2144295. VMware is actively working towards releasing the next version to replace NSX for vSphere 6.2.3 *
  • VMware NSX for vSphere version 6.2.3 delivered a security patch to address a known SSL VPN security vulnerability (CVE-2016-2079) . This issue may allow a remote attacker to gain access to sensitive information. Customers who use SSL VPN are strongly advised to review CVE-2016-2079 and contact VMware support to request immediate assistance. For questions or concerns, contact VMware Support. *
  • The next version of NSX for vSphere contains fixes for bugs that have been found in NSX 6.2.3.
  • Customers who have already upgraded to 6.2.3 are advised to review the following  KB articles:
    • VMware knowledge base article 2146227, VMs using Distributed Firewall (DFW) and Security Groups (SG) may experience connectivity issues. A workaround is available*
    • VMware knowledgebase article 2146293, Virtual machines lose network connectivity in NSX 6.2.x. *
    • VMware Knowledgebase article 2146413, VMs lose network connectivity in NSX with DLR HA. *

Critical Alert for Edge DLR users on NSX 6.2.3 and 6.2.3a *

  • NSX 6.2.3 DLR HA nodes remain in a split brain state (2146506) *
    • A new issue has been identified that can cause both primary and secondary HA nodes into an Active State, causing network disruption.
    • This issue will occur after approximately 24 days of BFD uptime and will continue to reoccur every 24 days.
    • Customers who are using NSX-V 6.2.3 or 6.2.3a are strongly advised to review KB 2146506, review how to prevent or remediate the issue and plan to upgrade to the next version of NSX.

For questions or concerns, contact VMware Support. To contact VMware support, see Filing a Support Request in My VMware (2006985) or How to Submit a Support Request.

Top NSX for vSphere issues for July 2016

NSX for vSphere 6.2.3 other new and changed issues


  • vCloud Director 8.0.1 is now interop-tested and supported with NSX 6.2.3.  For more information, see the VMware Interoperability Matrix
  • VMware is working actively with anti-virus solution partners to influence completion of their certification testing efforts with both NSX 6.2.2 and 6.2.3.  For more information, see the VMware Compatibility Guide (VCG)

Other trending issues

Known interoperability issues during upgrade to NSX for vSphere 6.2.3

Note: VMware vSphere 6.0 supports VIB downloads over port 443 (instead of port 80). This port is opened and closed dynamically. The intermediate devices between the ESXi hosts and vCenter Server must allow traffic using this port.

How to track Top Field Issues

What’s New in vSphere 5.5 Update 3b

noticeSSLv3 Protocol Disabled by Default


Across the industry, enterprise software products and solutions are dropping use of and support for the SSLv3 protocol. The Internet Engineering Task Force (IETF) officially deprecated the SSLv3 protocol in RFC 7568 due to its obsolescence and inherent unfixability. Instead, IETF recommends the latest version of TLS.

VMware is therefore dropping support for SSLv3 on both the server side and the client side in vSphere. The release of vSphere 5.5 Update 3b from VMware disables SSLv3 by default to meet current standards and compliance.

Disabling SSLv3 by default also brings some restrictions with respect to installation, upgrading, and compatibility. This blog summarizes these limitations which are also documented in detail in the respective release notes and KB articles.

Below are some of the key aspects that you should be aware of when you upgrade to vSphere 5.5 Update 3b

  1. Upgrade sequence: As recommended in KB 2057795 you must upgrade vCenter Server to 5.5 Update 3b first and then update the hosts to ESXi 5.5 Update 3b.

Earlier releases of vCenter Server won’t be able to manage ESXi 5.5 Update 3b. As a workaround, you can re-enable SSLv3 protocol on ESXi by following the configuration described in KB 2139396. However, VMware strongly recommends against re-enabling the SSLv3 protocol.

  1. Upgrade both vCenter Server and ESXi to 5.5 Update 3b: In order to disable SSLv3 completely in your vSphere environment, we recommend that you update both vCenter Server and ESXi to vSphere 5.5 Update 3b.
  2. View Composer earlier than version 6.2 will have connection failures with ESXi 5.5 Update 3b. Refer to KB 2121021
  3. SSLv3 can be re-enabled by the configuration described in KB#2139396. Re-enablement of SSLv3 protocol has to be consistent across all ESXi and vCenter Server services and require mandatory service restart. However, VMware strongly recommends against re- enabling the SSLv3 protocol.

Note: Hostprofile will be able to capture SSLv3 protocol enablement configuration changes for all the services except Hostd service in ESXi.

Using vSphere ESXi Image Builder to create an installable ISO that is not vulnerable to Heartbleed

Here is a follow-up post from Andrew Lytle, member of the VMware Mission Critical Support Team. Andrew is a Senior Support Engineer who is specializes in vCenter and ESXi related support.

VMware recently released updates to all products affected by the vulnerability dubbed “Heartbleed” (CVE-2014-0160): http://www.vmware.com/security/advisories/VMSA-2014-0004.html

As per KB article: Resolving OpenSSL Heartbleed for ESXi 5.5 – CVE-2014-0160 (2076665), the delivery method for this code change in the VMware ESXi product is through an updated ESXi vSphere Installation Bundle (VIB). VIBs are the building blocks of an ESXi image. A VIB is akin to a tarball or ZIP archive in that it’s a collection of files packaged into a single archive.

Typically a new ESXi ISO file will be made available only during major revisions of the product (Update 1, Update 2, etc). If you need an ESXi 5.5 ISO which is already protected from Heartbleed, you can make your own ISO easily using vSphere PowerCLI.

The PowerCLI ImageBuilder cmdlets are designed to make custom ESXi ISOs which have asynchronous driver releases pre-installed, but it can also be used in a situation like this to make an ISO which lines up with a Patch Release instead of a full ESXi Update Release.

In this post we will cover both the ESXi 5.5 GA branch, as well as the ESXi 5.5 Update 1 branch. Choose the set of steps which will provide the ISO branch you need for your environment.

Creating an ISO based on ESXi 5.5 GA (Pre-Update 1)

These steps are for downloading the requirements for creating an ISO which is based on the ESXi 5.5 “GA” release, which was originally released 2013-09-22.

Step 1: Download the Required Files

When creating a custom ESXi image through Image Builder, we need to start by downloading the required files:

Install PowerCLI through the Windows MSI package, and copy the zip files to a handy location. For the purposes of this example, I will copy these files to C:\Patches\

Step 2: Import the Software Depot

  • Add-EsxSoftwareDepot C:\Patches\ESXi550-201404020.zip

Step 3: Confirm the patched version (optional)

If you wish to confirm the esx-base VIB (which includes the Heartbleed vulnerability code change) is added correctly, you can confirm the VIB has Version of 5.5.0-0.15.1746974 and the Creation Date of 4/15/2014.

  • Get-EsxSoftwarePackages –Name esx-base

Step 4: Export the Image Profile to an ISO

  • Export-EsxImageProfile –ImageProfile ESXi-5.5.0-20140401020s-standard –ExportToISO –FilePath C:\Patches\ESXi5.5-heartbleed.iso

Creating an ISO based on ESXi 5.5 Update 1

These steps are for creating an ISO which is based on the ESXi 5.5 “Update 1” release, which was originally released 2014-03-11.

Step 1: Download the Required Files

When creating a custom ESXi image through Image Builder, we need to start by downloading the required files:

Copy the zip files to a handy location. For the purposes of this example, I will copy it to C:\Patches\

Step 2: Import the Software Depot

  • Add-EsxSoftwareDepot C:\Patches\ESXi550-201404001.zip

Step 3: Confirm the patched version (optional)

If you wish to confirm the esx-base VIB (which includes the Heartbleed vulnerability code change) is added correctly, you can confirm the VIB has the Version of 5.5.0-1.16.1746018 and Creation Date of 4/15/2014.

  • Get-EsxSoftwarePackages –Name esx-base

Step 4: Export the Image Profile to an ISO

  • Export-EsxImageProfile –ImageProfile ESXi-5.5.0-20140404001-standard –ExportToISO –FilePath C:\Patches\ESXi5.5-update1-heartbleed.iso

Installing the ESXi ISO

The ISO file which was created in this steps can be used in exactly the same manner as the normal VMware ESXi 5.5 ISO. It can be mounted in a remote management console, or burned to a CD/DVD for installation.

Patching ESXi 5.5 for Heartbleed without installing Update 1

On April 19th, VMware released a series of patches for ESX 5.5 and ESX 5.5 Update 1 to re-mediate the CVE dubbed “Heartbleed” (CVE-2014-0076 and CVE-2014-0160).

VMware also recently announced that there was an issue in the newest version of ESXi 5.5 (Update 1 and later), which can cause difficulties communicating with NFS storage. This NFS issue is still being investigated, and customers are encouraged to subscribe to KB article: Intermittent NFS APDs on ESXi 5.5 U1 (2076392) for updates.

Due to the confluence of these two unrelated issues, you might find yourself trying to patch ESXi to protect yourself from the Heartbleed vulnerability, while at the same time trying to avoid installing ESXi 5.5 Update 1.

Here is the information from the VMware Knowledge Base on the topic:


The note at the bottom is the key. Stated simply, if you are…

  • Using NFS storage
  • Concerned about patching to Update 1 due to change control
  • Not already running ESXi 5.5 Update 1 (build-1623387)

… then you should patch your install for the Heartbleed issue and at the same time stay at ESX 5.5 by applying Patch Release ESXi550-201404020, and not ESXi550-201404001.

An Explanation of Patch Release Codes

To better understand the Patching process in a VMware environment, it is valuable to understand the codes which are used in VMware Patch Releases.

When VMware releases a patch, or a series of patches, they are bundled together in what is known as a Patch Release. A Patch Release will have a coded name which is formed using the following structure. I have added braces to demonstrate the different sections better in each example.


For example, the patch release for ESXi 5.5 that was released in January 2013 would be coded like this (without the explanatory braces):


As part of a Patch Release, there will be at least one Patch. Each Patch is given a Patch (or Bulletin) ID. Patch IDs are similarly structured to Patch Release codes, but also have a two letter suffix. For Security Bulletins, the prefix will be SG. For Bug Fix Bulletins, the prefix will be BG.

For example, the two Patch IDs which were released to patch Heartbleed are:


Note that the only difference in the Patch IDs here is in the three digit release number (401 vs 420).

Patching with VMware Update Manager

There are a number of methods for patching ESXi hosts, and the most commonly used is VMware Update Manager (VUM). VUM will present a pair of Dynamic Baselines which will be automatically updated when patches are available. The danger in this case is that VUM may show you both the Pre-Update 1 patch, as well as the Post-Update 1 patch. If you are not careful as to which patches you apply, you might accidentally end up patching your host to Post-Update 1.

Here are the patches which were released on April 19th, as seen in VUM. The Update 1 patch is highlighted in red, while the Pre-Update 1 patch is marked in green.


Note: VMware also released two other ESXi 5.5 patches on April 19th, as part of Patch Release but these are not related to the Heartbleed vulnerability in any fashion. (ESXi550-201404402-BG, and ESXi550-201404403-BG).

Creating a Fixed Baseline

Patching a host using ESXi550-201404420-SG (Pre-Update 1), while avoiding ESXi550-201404401-SG (Post-Update 1) requires the use of a Fixed Baseline in Update Manager.

  1. Start in the Update Manager Admin view.
  2. Select the Baselines and Groups tab.
  3. Click Create… in the Baselines column.
  4. Give the new Baseline a descriptive Name (and optionally a Description).
  5. Click Next.
  6. For Baseline type, select Fixed.
  7. Use the Search feature to find the only Patch we want to apply. You will need to select the Patch ID option from the dropdown menu to ensure the search scans for the appropriate column.
  8. Enter the Patch ID into the search field: ESXi550-201404420-SG and click Enter to search.
  9. Select the Patch which shows up in the filtered list, and click the Down Arrow to move it into the selected Baselines.
  10. Click Next and confirm that the Patch ESXi550-201404420-SG is the only one selected.
  11. Click Finish.

The Baseline is now created and available for use.

Remediating a Host using the Fixed Baseline

Once the Fixed Baseline has been created, we can use it to Scan and Remediate an ESXi host.

  1. Select the host you wish to patch, and place it into Maintenance Mode.
  2. Click the Update Manager tab.
  3. Make sure that there are no Dynamic Baselines attached to the host you wish to patch. Detach any baselines which are currently attached:
    Critical Host Patches (Predefined)
    Non-Critical Host Patches (Predefined)
    Any other Custom Baselines which you have created
  4. Click the Attach link.
  5. Select the newly created Baseline and click Attach.
  6. Click the Scan link and make sure Patches and Extensions is selected. Click Scan again.
  7. When you are ready to patch the host, select Remediate.
  8. Complete the Remediation wizard.

Once the host is patched, it will reboot automatically.

Patching an ESXi host manually via the command line

Another option to patch an ESXi host is to use the esxcli command line tool. The patch files required are the same. For more information on how to proceed with this route, refer to the vSphere 5.5 Documentation under the heading Update a Host with Individual VIBs.


Author: Andrew Lytle
As a member of the VMware Mission Critical Support Team, Andrew Lytle is a Senior Support Engineer who is specializes in vCenter and ESXi related support.

New ESX/ESXi Patches

A number of patches have been posted for ESX and ESXi. Details can be found in the following KB articles:

VMware ESX 4.1, Patch Release ESX410-201312401-SG: Updates VMkernel, CIM, Tools, and others (2061196)
VMware ESX 4.1, Patch Release ESX410-201312403-SG: Updates nss and nspr libraries (2061203)
VMware ESX 4.1, Patch Release ESX410-201312001 (2061209)
VMware ESXi, Patch Release ESXi410-201312001 (2061210)
VMware ESXi 4.1, Patch Release ESXi410-201312401-SG: Updates Firmware (2061205)
VMware ESXi 4.1, Patch ESXi410-201312402-BG: Updates”>VMware Tools (2061206)
VMware ESX 4.1, Patch Release ESX410-201312402-BG: Updates e1000e drivers (2061201)

Some new ESXi patches today

Some new patches for ESXI out today you might want to be aware of:

vCenter Server 5.1 Update 1a

Back on April 29th we posted an alert: ALERT: Login issue after updating to vCenter 5.1 Update 1 which detailed a scenario whereby customers might be unable to log in using the vSphere Web Client or domain username/password credentials via the vSphere Client after updating to vCenter 5.1

Tonight at 7:30pm PST, vCenter Server 5.1 Update 1 will be removed from the VMware download site and will be replaced by vCenter Server 5.1 Update 1a. The primary aim of the 5.1 U1a release is to address the regression that was identified in 5.1 U1.

Customers are urged to read the README included with the new update before they apply the update.

Details of what has and has not been fixed are provided in this KB article: http://kb.vmware.com/kb/2037410

vCenter Server issue with Oracle client resolved

Back in December, we posted the support alert: vCenter Server 5.x does not function correctly when installed with Oracle Patch 10. The support alert was documented fully in KB article: vCenter Server 5.x does not function correctly when installed with Oracle Patch 10 (2039874).

There is news on this alert. Oracle has since resolved the issue in the Windows Oracle Client p16656151 (Patch 19) 64-bit, which can be downloaded today.

A couple more product upgrade notices

Here are a couple of other important items to note if you are applying updates today. We want to get this information out to our readers as quickly as possible. Please share with your colleagues:

Update vSA after upgrading to vCenter Server 5.1U1
Description: vSA 5.1.1 customers must upgrade to vSA 5.1.3 after upgrading their vCenter Server to 5.1 Update 1. For more information, see the release notes linked below.
Link: vSA Release Notes

Advisory: Upgrading to vCloud Director 5.1.x
Description: Upgrade to vCloud Director 5.1.x can affect virtual machines network connectivity due to network adapter mismatches. VMware KB article 2047922 provides tools and procedures to detect and correct network adapter type mismatches before the upgrade.
Link: Detecting and resolving network adapter type mismatches before upgrading vCloud Director to version 5.1.x (2047922)

To Patch, Perchance to Upgrade

Today we have another guest post from Tech Support Engineer Mike Bean.

'Morning everyone, I must take a moment to say thank you to people, both internally and externally, who’ve expressed support for the first column I wrote. To quote a web comic I enjoy, “we must learn, lest we stagnate”, if readers have enjoyed it as well; then I take that as a compliment!

At the risk of digressing from the “most wanted theme”, I wish to approach a new subject today. By the time you read this, ESX vSphere update 2 will be publicly available.


I gladly extend congratulations to all our development and QA teams the world over. I can say with complete sincerity they’re my heroes, for it is on their backs that our product is built, one feature spec and bug report at a time. Congratulations lords and ladies, hug your significant others, have a beer, and enjoy the moment. You’re our warriors!

I began my morning with a soda and a copy of the release notes, and I can safely say, it’s not light reading. Speaking as an army-ant in Global Support Services, I don’t see any issues we’ve been breathless with anticipation for, but there’s also far too many things being addressed to engage in any sweeping generalization.


In the course of my time here, I’ve often been asked the eternal question “should I get the patch?” I’m never quite sure how to answer this question, but it’s honestly one worth asking. So it’s worth taking a moment to examine.

Asking a software company if you should apply the patch is a little like asking a lawyer if you should sue; let’s face it, we have a slight bias. On one hand, if we didn’t think our customers would benefit, we wouldn’t have released the patch in the first place. On the other hand, many of my customers are system admins, and I’ve walked a mile in their shoes. In that sense, I’m well aware that they don’t have the liberty of applying patches/FW flashes whenever any number of numerous vendors they do business with, releases the latest update. My college economics classes would’ve called it “opportunity cost”. Downtimes must be scheduled, approvals must be obtained, benefits assessed. It is precisely because I have experienced both the software development point of view, and the system administration point of view, that I’m well aware that many of our customers may not have had problems. Risk is relative, myself and my co-workers routinely speak to customers who’ve literally ran for years without issues. Why then, upgrade?

Typically, when a customer asks me that question on the phone, I often end up trying to explain that I can’t really answer that question. It’s the natural course of software development that a changing code base means a changing landscape. Old problems are solved, and new ones arise, and I won’t imply to the contrary. However, that should not be interpreted as carte blanche to never patch. Risk may be relative, but as available security and stability fixes accumulate, so does risk, and so does benefit.

It’s a matter of risk assessment, as a GSS TSE (technical support engineer) it’s my responsibility to try and help present the facts and the options, but ultimately, the final decision always belongs to the customer. Only they know their networks, and basically, only they can realistically decide when the benefits of patching will exceed the costs. Inevitably, the only safe policy is one of shared information (informed consent); In that spirit, I encourage most everyone I speak to familiarize themselves with the available resources, both in documents and communities. Examine the facts for yourself, and let the update speak for itself.

In closing, I would briefly highlight at least one real case from memory. I spoke with a customer some weeks ago who’d been having substantial issues with hangs on his cluster of Dell 2900s. I sincerely hope he’s watching update 2’s contents, very carefully!


VMware ESX might fail to boot on Dell 2900 servers
If your Dell 2900 server has a version of BIOS earlier than 2.1.1, ESX VMkernel might stop responding while booting. This is due to a bug in the Dell BIOS, which is fixed in BIOS version 2.1.1.

Workaround: Upgrade BIOS to the version 2.1.1 or later.

You spoke, and we were listening! Till next time!