Virtualize Business Critical Applications

Looking at a Sizing Example

Figure 4, shows the most commonly encountered JVM and virtual machine size. This may be a fairly busy JVM with 100 to 250 concurrent threads (actual thread count varies as it depends on the nature of the workload), 4GB of heap, approximately 4.5GB for the JVM process, and 0.5GB for the guest OS. This results in a total recommended memory reservation for the virtual machine of 5GB with 2 vCPU and 1 JVM process.

Figure 4. Most Commonly Encountered Configuration

Figure 5 takes a closer look at the sizing example within the Java process, the memory layout, and for various sizes..

Figure 5. 5GB RAM Virtual Machine with One JVM Process and Two CPUs

The general sizing equation can be summarized as follows:

Let’s assume that, through load testing, a JVM max heap (-Xmx) of 4096m has been determined as necessary. Proceed to size as follows:

• Set -Xmx4096m and set –Xms4096m.
• Set –XX:MaxPermSize=256m. This value is a common number and depends on the memory footprint of the class-level information within your Java application code base..
• The other segment of NumberOfConcurrentThreads*(-Xss) depends mostly on the NumberOfConcurrentThreads the JVM will process, and the –Xss value you have chosen. A common range of –Xss is 128k-192k. If, for example, NumberOfConcurrentThreads is 100, then 100*192k => 19200k (assuming you set –Xss to 192k).

Note: The stack -Xss is application dependent, i.e. if the stack is not sized correctly you will get a StackOverflow. The default value is sometimes quite large, but you can size it down to help save on memory consumption.

• Assume the OS has a requirement of about 500m to run as per the OS spec.
• Total JVM memory (java process memory) = 4096m (-Xmx) + 256m (–XX:MaxPermSize) + 100*192k (NumberOfConcurrentThreads*-Xss) + “other mem”.
• Therefore, JVM memory is approximately 4096m+256m+19.2m + “other mem” = 4371m + “other mem”.
• Typically “other mem” is not significant. However, it can be quite large if the application uses lots of NIO buffers and socket buffers. Otherwise, assuming about 5 percent of the total JVM process memory (i.e., 4 to 5 percent of 4371=> assume 217m), should be enough, although proper load testing should be used to verify.
• This implies that JVM process memory is 4371m+217m=4588m
• To determine the virtual machine memory, assume you are using Linux with only this single Java process and no other significant process running. The total configured memory for the virtual machine translates to: VM memory = 4588m + 500m = 5088m.
• Finally, you should set the virtual machine memory as the memory reservation. You can choose to set the memory reservation as 5088m. However, over time you should monitor the active memory used by the virtual machine that houses this JVM process and adjust the memory reservation to that active memory value, which could be less than 5088m.

Stay tuned. In Part 3 we will conclude with an overview of the recently released book, Enterprise Java Applications Architecture on VMware.

This week we will publish three blogs on the topic of sizing virtual machines for Java workloads. In the bogs we will discuss various sizing considerations, best practices, sizing limits, and the most common configuration used by our customers.

A sizing exercise in a virtualized environment for Java workloads is similar to that in physical environment. The only difference is that virtualized environment provides more flexibility, such as the ability to easily change the compute resource configuration. For more detailed information, we encourage you to review the Enterprise Java Applications on VMware – Best Practices Guide  (http://www.vmware.com/resources/techresources/1087)

Sizing Virtual Machines for JVM workloads – Part 1

Before delving into various sizing considerations we’ll provide some background information about the practical sizing limits of JVMs.

Background: JVM Practical Sizing Limits

Figure 1 illustrates the theoretical and practical sizing limits of Java workloads. These are critical limits that you need to be aware of when sizing JVM workloads.

Figure 1. Theoretical and Practical Sizing Limits of JVMs

The JVM theoretical limit is 16 exabytes, but there is no practical system that can provide this amount of memory, so we present this as the first theoretical limit.

• The second limit is the amount of memory a guest OS can support. In most practical cases, this is several terabytes and depends on the operating system used.
• The third limit is the ESXi 5 1TB RAM per virtual machine limit, which is ample for any workload that we have encountered.
• The fourth limit (really, the first practical limit) is the amount of RAM that is cost-effective on typical ESXi hosts.
• The fifth limit is the total amount of RAM across the server, and how this is divided into the number of NUMA nodes, where each processer socket will have one NUMA node worth of NUMA-local memory. The NUMA-local memory can be calculated as the total amount of RAM within the server divided by the number of processor sockets. We know that for optimal performance you should always size a virtual machine within the NUMA node memory boundaries. ESXi has many NUMA optimizations that come into play, but even so, it is best to stay NUMA local.

For example, if the ESX host has 256GB of RAM across two processor sockets, it has 2 NUMA nodes with 128GB (256GB/2) of RAM across each NUMA node. This implies that when you are sizing a virtual machine, it should not exceed the 128GB limit in order for it to be NUMA local.

The limits outlined above can help drive your design and sizing decision as to how practical and feasible it is to size large JVMs. However, there are other considerations that come with sizing very large JVMs such as GC tuning complexity and the knowledge needed to maintain large JVMs. In fact, most commonly sized JVMs within the VMware customer base are around 4GB of RAM for a typical enterprise Web application. On the other hand, larger JVMs exist, and we have customers that run large scale monitoring systems and large distributed data platforms on JVMs ranging from 4GB to 128GB.

With large JVMs comes the need to better understand GC tuning. VMware has helped many customers with their GC tuning activities, even though GC tuning on physical is no different than on virtual. The reason is that VMware has uniquely integrated vFabric Java and vSphere expertise into one spectrum, which has helped our customers optimally run many Java workloads on vSphere. When faced with the question of whether to vertically scale the size of the JVM and virtual machine, first consider a horizontal scale out approach. VMware has consistently found that our customers get better scalability with the horizontal scale out approach.

Furthermore, when sizing, it is helpful to categorize the size of the JVMs and virtual machines based on the Java workload types, as shown in Figure 2. 

Figure 2. Common JVM Sizes and Workload Categories

We usually find that the reason customers vertically scale a JVM is due to perceived simplicity of deployment and leaving existing JVM processes intact. Be aware of workload-related choices.

• For example, a customer initially deploys one JVM process and as demand increases for more applications to be deployed, instead of horizontally scaling out by creating a second JVM and virtual machine, a vertical scale up approach is taken. As a consequence, the existing JVM is forced to vertically scale and carry many different types of workloads with varied requirements.
• Keep in mind that some workloads, such as, a job scheduler, have a need for high throughput, while a public facing Web application demands fast response time. Stacking these types of applications on top of each other within one JVM complicates the GC cycle tuning opportunity. When tuning GC for higher throughput it is usually at the cost of decreased response time, and vice-versa.
• You can achieve both higher throughput and better response time with GC tuning, but it unnecessarily extends the GC tuning activity. When faced with this deployment choice it is always best to split out different types of Java workloads into their own JVMs. One approach is to run the job scheduler type of workload in its own JVM and virtual machine, and the Web-based Java application on its own JVM and virtual machine.

In Figure 3, JVM-1 is deployed on a virtual machine that has mixed application workload types, which complicates GC tuning and scalability when attempting to scale up this application mix in JVM-2. A better approach is to split the Web application into JVM-3 and the job scheduler application into JVM-4 (that is, scaled out horizontally with the flexibility to vertically scale if needed). If you compare the vertical scalability of JVM-3 and JVM-4 versus vertical scalability of JVM-2 you will find JVM-3 and JVM-4 always scale better and are easier to tune. 

Figure 3. Splitting Workload Types to Improve Scalability

In Part 2 we will look at an actual sizing example with some practical numbers that you can directly apply. 

I thought it would be beneficial to folks to summarize the sessions I am presenting at VMworld-2011 Las Vegas. 

The breakout sessions BCA-1950 will cover details of running enterprise Java on VMware and in production environments, additionally the session will have a guest customer co-speaker that will show case how they virtualized Websphere.  The other sessions GD01 and EXPERTS-02 are intended to be much more interactive and you will have plenty of time to ask questions.

I hope to see you there! 

Thanks for reading. 

Regards, 

Emad Benjamin,

Staff Solutions Architect on vFabric and Java

In this blog I would like to drill down with an actual sizing example based on the information presented in a prior blog here. 

Let’s take the generalized equation presented in Figure-1 below and show an actual sizing example:

Figure-1 Shows a Virtual Machine (VM) with 1 HotSpot JVM.

Figure 2 – Shows the generalized memory Sizing equation of all the memory segments shown in Figure-1

Where:

• Guest OS Memory is approx 0.5G-1G (depends on OS/other processes)
• -Xmx, is the JVM max Heap Size
• -Xss, is the Java Thread Stack Size, the default is OS and JVM dependent, and it can range 256k-to-1MB. 
• The default should be tuned down to a range that doesn’t cause StackOverflow.  I often use 128k-192k.  Since the default –Xss is high, tuning it down can help save on memory used and given back to the Guest OS.
• Perm Size is an area additional to the –Xmx (Max Heap) value and is not GC-ed because it contains class-level information.
• “other mem” is additional mem required for NIO buffers, JIT code cache, classloaders, Socket Buffers (receive/send), JNI, GC internal info
• If you have multiple JVMs (N JVMs) on a VM then:
  • VM Memory = Guest OS memory + N * JVM Memory

Let’s assume that through load testing a JVM Max heap (-Xmx) of 4096m has been determined as necessary. 

You would proceed to size as follows:

• Set -Xmx4096m, also set –Xms4096m
• Set –XX:MaxPermSize=256m, you have chosen to use –XX:MaxPermSize of 256m which a common number and depends on the memory footprint of the Class level information within your Java application code base.
• The other segment of NumberOfConcurrentThreads*(-Xss) depends largely on NumberOfConcurrentThreads the JVM will process, and the –Xss value you have chosen.  A common range of –Xss is 128k-192k.
  • Note: -Xss is OS and JVM dependent; if the stack is not sized correctly you will get a StackOverflow.  As noted earlier the default value is sometimes quite large and you can benefit from sizing it down to help save on memory consumption.
  • If for example NumberOfConcurrentThreads is 4, then 4*192k => 768k (assuming you set –Xss to 192k)
  • Assume the OS has a requirement of about 500m to run as per the OS spec
  • Total JVM memory (Java process memory) = 4096m (-Xmx) + 256m (–XX:MaxPermSize) + 4*192k (NumberOfConcurrentThreads*-Xss) + “other mem”
    • Therefore JVM memory approximately = 4096m+256m+0.768m + “other mem” = 4352.768m + “other mem”
    • Now typically “other mem” is not significant however can be quite large if the application uses lots of NIO buffers, and socket buffers.  Otherwise assuming about 5% of the total JVM Process memory, i.e. 5% * 4352.768= 217m, should be enough, although proper load testing should be used to verify.
    • This now implies that JVM process memory = 4352.768m+217m=4570m
    • Now to determine the VM memory, assume you are using Linux with no other significant process is running on it, only this single Java process, the total configured memory for the VM translates to:
      • VM memory =  4570m + 500m = 5070m
      • Now next you should set the VM memory as the memory reservation, you can chose to set the memory reservation as 5070m, however over time you should monitory the Active memory used by the VM that houses this JVM Process and adjust the memory reservation to that Active memory value, which could be less than 5070m

NOTE: a common miss-conception is to assume that this –Xmx value is equal to the Java Process memory needed, but clearly as described by the equation in Figure-2 the JVM Memory (or Java Process Memory) is greater than JVM Max heap (greater than –Xmx) and this is due to the other additional segments outside the heap that make up the memory space of the total Java process such as JVM Perm Size, NumberOfConcurrentThreads*(-Xss), and the “other memory” section.

Figure 3 – Shows Figure-1 with an actual sizing example we just discussed above, in order to illustrate amount of memory each segment has been set to.

Thanks for reading.

Regards,

Emad Benjamin,

Java and vFabric Solutions Architect.

For those that don’t really have time to read the entire blog, I can simply answer as, yes, to the above question.  Indeed Java is independent of the underlying hypervisor such as VMware bare metal hypervisor, and the Operating System.  But for those folks that want to delve a little more into what this means, please read on.

This question has recently started to surface from customers new to Virtualization, and not so new to Java.  Now while the answer to the question posed was candidly obvious, it was a legitimate question to address as it seemed to have been asked a few times.

There is no doubt that Java’s main design tenets are based on a cross-platform language that is Operating Systems independent, so long as there is an Operating System supported underlying runtime.  We know this runtime as the Java Virtual Machine (JVM) that has become a permanent fixture of many enterprise application platforms.  You could write a Java application and run it on various JVMs on different Operating Systems, and without needing to recompile.  Now of course many of VMware’s customers have one vendor targeted JVM in production and wouldn’t have to worry about moving a Java application from one JVM implementation to another, but if they chose to they could easily do it, primarily due to Java’s cross-platform and Operating System independence facilitated by a JVM. 

Hence you can conclude that the Java applications don’t really care which JVM is being targeted to run on and hence independent of the specific JVM implementation and Operating System. 

NOTE: Of course some of you may say “what about all of the different internal behaviors of one JVM vs. another?”  At the end of the day they all adhere to the JVM Spec, and while some JVM Options/(-XX) flags etc. are named differently; they more or less behave in a similar manner.  Certainly the differences are not in the language, but in the way the Java process can be optimized with various JVM options passed at the Java command line.)

Fast forward to the infrastructure side of things, VMware ESXi is a bare metal hypervisor that makes it possible to run multiple operating systems on a particular piece of hardware.  It alleviates the infrastructure administrators from having to worry about installing one kind of operating system for one piece of hardware vs. another.  VMware takes care of making the Operating System run independent of the underlying hardware (bare metal) and hence creating a degree of independence between the Operating System and the bare metal/hardware.

While the answer to the question of “Is Java both OS and Hypervisor Independent?” Is clearly yes, it is so due to two degrees of independence, where the first degree being that of Java’s main tenet of cross platform and OS independence, and then second degree is that of VMware ESXi Hypervisor making the Operating System independent of the hardware that it runs on.  In fact when a Java application runs on an Operating System that is in a ESXi based Virtual Machine, ESXi has no notion of knowing it is a Java workload running on the Operating System, making the ESXi hypervisor completely independent of the workload that is running on it – a further testament to this is when you deploy a Java application on a Virtual Machine no Operating System changes are needed due to this independence.

Conversely the JVM doesn’t really know that is running on a Virtual Machine sitting on an ESXi hypervisor, to the JVM the VM appears like any other server, with compute resources (CPUs/RAM etc.) presented to it.

In conclusion so long as the JVM you are using is supported on the Operating System that your applications are running on then there is no need for additional concern and/or dependency for additional support from the downstream virtual machine and ESXi layers.  However, should you feel you would like to get additional support when running Java in Virtual Machine on VMware ESXi, you are welcome to submit a support ticket, and we will be glad to help you.

Below diagram illustrates all the layers we discussed in this blog.



 

As previously mentioned many of our customers have successfully deployed various Java application servers on ESXi based virtual machines in production environments.

Drop me a question, otherwise check out the Enterprise Java Applications on VMware Best Practices Guide: here.

Emad Benjamin, Java and vFabric Solutions Architect

Many of our customers have run critical enterprise Java applications on vSphere successfully.  The best practices guide is often referred to by our customers when embarking on running Java applications on vSphere.

December last year version 2 of the best practices paper was released here: http://www.vmware.com/resources/techresources/1087,

The paper covers quite few best practices and definitely recommended for anyone doing deployments of Java applications on vSphere, however it is worthwhile to note that the top 3 most sought after best practices from this paper are as follows and based on customer interaction:

• BP4 – VM Memory Sizing
• BP5 – Setting Memory Reservation
• BP6 – Using memory large pages

BP4 – VM memory Sizing: Whether you are using Windows or Linux as your guest OS, refer to the technical specification of the various vendors for memory requirements. It is common to see the guest OS allocated about 1GB in addition to the JVM memory size. However, each installation may have additional processes running on it, for example monitoring agents, and you need to accommodate their memory requirements as well.   Figure 2 shows the various segments of JVM and VM memory, and the formula summarizes VM Memory as:

VM Memory (needed) = guest OS memory + JVM Memory,

whereJVM Memory = JVM Max Heap (-Xmx value) + Perm Gen (-XX:MaxPermSize) + NumberOfConcurrentThreads * (-Xss)

The -Xmx value is the value that you found during load testing for your application on physical servers. This value does not need to change when moving to a virtualized environment. Load testing your application when deployed on vSphere will help confirm the best –Xmx value.  It is recommended that you do not overcommit memory because the JVM memory is an active space where objects are constantly being created and garbage collected. Such an active memory space requires its memory to be available all the time. If you overcommit memory ballooning or swapping may occur and impede performance.  ESX host employs two distinct techniques for dynamically expanding or contracting the amount of memory allocated to virtual machines. The first method is known as memory balloon driver (vmmemctl). This is loaded from the VMware Tools package into the guest operating system running in a virtual machine. The second method involves paging from a virtual machine to a server swap file, without any involvement by the guest operating system.

BP5- Setting Memory Reservation: JVMs running on VMs have an active heap space requirement that must always be present in physical memory. Use the VMware vSphere Client to set the reservation equal to the needed VM memory.

Reservation Memory = VM Memory = guest OS Memory + JVM Memory

You may set this reservation to the active memory being used by the VM for a more efficient use of the amount of memory available. Or, a simpler approach is to set the reservation equal to the total configured memory of the VM.

BP6- Use memory large Pages:  Large memory pages help performance by optimizing the use of the Translation Look-aside Buffer (TLB), where virtual to physical address translations are performed. Use large memory pages as supported by your JVM and your guest operating system. The operating system and the JVM must be informed that you want to use large memory pages, as is the case when using large pages in physical systems.

• Set the -XX:+UseLargePagesat the JVM level for Sun HotSpot.
• On the IBM JVM it is -Xlp, and JRockit -XXlargePages.
• You also need to enable this at the guest OS level. For information, see Large Page Performance: ESX Server 3.5 and ESX Server 3i v3.5.

Hello and welcome to the new VMware Business Critical Applications blog, a source for information, insight, and updates to help you virtualize your business critical applications on the VMware platform.

The blog will cover the following applications/functional areas and we will continue to add more over time:

• Microsoft Applications
  • Exchange
  • SQL Server
  • SharePoint
• SAP
• Oracle
• Java Applications 

We intend to use this blog to:

• Communicate best practices for deploying on the vSphere platform.
• Provide Updates on current solution projects and activities including solution rollouts, new collaterals etc.
• Provide Update on various product and solution enablement activities including events, webinars, partner engagements, trainings etc.
• Publish results of lab testing in the aforementioned functional areas.  Testing and results will include functional and technical use cases, workload characterization study and deployment  "how-to".  For official performance test results, please refer to the VROOM! Blog at http://blogs.vmware.com/performance/.
• Communicate general application design principles that we've discovered through research and work with our customers.
• Communicate step-by-step procedures for common application and infrastructure management tasks.
• Highlight VMware and partner products or features than can enhance the overall solution.
• Provide links to relevant online documentation.
• Provide insight on optimizing software licensing costs for virtualization.

We plan on posting regularly so grab the RSS feed or sign up for an e-mail alert to receive notification of new entries as they are posted.

For published whitepapers, including technical whitepapers and customer success stories, please visit our Business Critical Applications website at http://www.vmware.com/solutions/business-critical-apps/.  On the right-hand side, you'll find links to the individual application pages.

Thanks for reading!

Scott Salyer

Solutions Manager, VMware