This blog was authored by Sudhir Balasubramanian,Vas Mitra and Oleg Ulyanov.
With this post we are starting a blog series dedicated to the Persistent Memory (PMem) feature introduced in vSphere 6.7. We will cover initial configuration of the PMem in vSphere and the way how different virtualized business critical applications (like SQL Server, Oracle, SAP) can profit from using the PMem.
Before we dig deep in the application content let us review the basic – what is the Persistent Memory
Introducing Persistent Memory and Configuring ESXi host
Non-Volatile Dual In-line Memory Module (NVDIMM-N) is a persistent memory technology that uses memory DIMMs to store the data (just like DRAM) but retains the data in the event of a power lost or a system shut down, making the data persistent. The SNIA website provides a great overview and lot of resources on this technology.
Persistent Memory (PMem) technology layer provides the performance of memory (DRAM) with the persistence of traditional storage as depicted below:
PMem support is introduced in vSphere version 6.7 and can be combined with the following to increase performance of high-load applications: native PMem support in Windows Server 2016 and SQL Server 2016 SP1 and higher ; Enterprise Red Hat 7.4 or later ; SUSE Linux Enterprise 12, Service Pack 2 or later.
Exposing PMem modules to an ESXi host
Before a virtual machine (VM) will be able to consume a PMem based device, a hardware NVDIMMs must be made visible to an ESXi host. Most of configuration steps are automated and does not require user intervention, but it’s important to know where to find relevant information.
vSphere Clients (both Flash and HTML5 based) can be used to monitor the presence of PMem modules on the host. It can be done by checking the Summary Tab, and expanding the hardware frame with focus set on the ESXi server as shown below:
Host configuration menu does not have PMem related information in vSphere Clients now.
Much more details will be available after logging directly to an ESXi host with the VMware Host Client. Use a browser with the ESXi host name or IP address and credentials with the root level access to the host. If hardware NVDIMMs are present on the server, a new tab will be available on the ESXi host configuration page – “Persistent Memory”, containing three sub-tabs providing the information about available NVDIMMs:
ESXi hypervisor reads namespaces and combines multiple namespaces into one logical volume by writing GPT headers. ESXi uses VMFS-L as a file system format. Each namespace must be marked “In Use” for ESXi to create a logical volume. Refer to the troubleshooting section if this is not the case:
The result of the valid hardware configuration will be a datastore with the type “PMEM”, automatically created and ready to be consumed.
Note: A “PMEM” backend datastore is recreated after each reboot of the host. Before reboot, ALL VMs must be migrated of the host.
In vSphere Clients all available PMem datastores can be listed by navigating to the Policies and Profiles from the main menu and then selecting the VM Storage Policies section and choosing Storage Compatibility in policy details:
Note following:
- VM with a vPMem device will be not protected by vSphere HA. More details can be found in the vSphere Resource Management Guide on the page 47.
- VM snapshot is not supported.
- VM with a vPMem device must be excluded from VM level backup. If a VM was previously part of such backup, ensure to disable CBT before adding vNVDIMM to a VM.
Troubleshooting PMem
In rare cases, the PMem datastore will not be created automatically. It can be validated by using the hardware frame on the summary page of the ESXi host, showing 0 B Persistent memory present:
Navigating to the “Persistent Memory” – “Namespaces”, some namespaces are shown as “Not in use”.
In this case, all namespaces must be removed by using the “Delete” button and a host needs to be restarted. After restart, all namespaces were correctly processed. Such configuration was observer on a dual boot physical server with multiple operating system installed and NVDIMMs being owned by another instance of the operation system.
Using the PMem
Persistent memory (PMem) can be consumed by virtual machines in two different modes:
- Virtual Persistent Memory (vPMem) or Direct-access mode. The hardware NVDIMM is exposed to a guest operating system as a virtual NVDIMM device. This enables a guest OS to use the PMem in byte addressable random mode. Support from the operation system and application is required.
- Virtual Persistent Memory Disk (vPMemDisk) or Virtual disk mode. The PMem can be accessed by the guest OS as a virtual SCSI device, and the virtual disk is stored on the PMem datastore. Any operating system can consume such device as a usual disk.
Both modes could be profitable for a virtualized application. In future posts we will see how applications can utilize virtual PMem using both modes mentioned above.
