Home > Blogs > VMware vFabric Blog


Dynamic Memory Management of vFabric Postgres

In a nutshell, dynamic memory management in vFabric Postgres is conceptually like Elastic Memory for Java (EM4J), but for a virtualized, enterprise-class, open source database instead of an application server.

Compared to a normal PostgreSQL server, vFabric Postgres brings two additions necessary for flexible virtualization of the database server. These two features can help companies realize the benefits of virtualizing the database and the associated cost savings from running an open source database on an extremely cost-effective infrastructure.

  1. Elastic shared memory management
  2. Automatic memory configuration

Elastic Shared Memory Management

Directly embedded with PostgreSQL core, the elastic shared memory management is a new feature of vFabric Postgres. This capability allows memory to be released or obtained according to the other virtual machine needs on the same server.

For example, let’s imagine that a Postgres server in a VM is set up to run with 4GB of shared memory on a certain server with 6GB of shared memory available. At a given moment, an additional virtual machine is added on this server and requests for an amount of memory of 4GB for its initial setup, possibly higher than usual to manage the first application loads. In the case where the old virtual machine uses only 2GB of its 4GB allocated, 2GB are wasted. And, this memory cannot be used by the new virtual machine–it is already allocated and taken away. In consequence, the total amount of memory requested is greater than what is available on the server. So, you will face resource conflict problems. Here are typical resolutions:

  • Restarting the old virtual machine to reduce its memory and give it to the new virtual machine
  • Reduce the memory allocated for the new server, resulting in a lower performance than expected for the new machine startup

If you are looking to run a virtualized data grid with shared infrastructure, both cases cause problems. For example, what if the old virtual machine runs applications that cannot face downtime? Or, what if the new virtual machine needs extra memory to achieve good performance during a certain time period? These types of issues don’t happen with vFabric Postgres because the new virtual machine can request the extra 2GB, and the old virtual machine will adapt automatically to this allocation, saving resources and maintaining performance for the whole system.

Managing Memory with PostgreSQL

Setting up a PostgreSQL server can cause headaches, especially regarding memory settings. If you’ve ever manipulated PostgreSQL, there are high chances that you already had to answer the following questions:

  • How do you set shared_buffers in a 8GB server?
  • For this amount of shared buffers, what do we set temp_buffers to?
  • And, what do we set work_mem to?
  • How should we change memory settings after a server migration? This is even worse if the spec of the old server is forgotten or lost

It has been difficult, even for advanced PostgreSQL users, to find a good balance between performance and memory usage. Optimizing the memory can also take quite a bit of time and potentially money to tune properly. In a shared data infrastructure, it can be more difficult.

This is where the second technology embedded with vFabric Postgres is really important.

Automatic memory parameter setting management allows the database servers to run without having to think about the different memory settings you need for your applications. This capability can be used effectively by beginners or advanced users. The following database server settings are tuned automatically:

  • shared_buffers
  • temp_buffers
  • work_mem
  • maintenance_work_mem

By providing automated optimization, the database reduces the overall burden on DBAs.

Getting Started with vFabric Postgres

These features are only available in the appliance of vFabric Postgres, not the RPM packages.  If you would like to set up your own database server, here are some example settings. These are used when changing the memory of the virtual Machine when an appliance of vFabric Postgres is used with VMware Fusion.

RAM shared_buffers temp_buffers work_mem Maintenance_work_mem
2GB 1400MB 2048kB 1024kB 100MB
1GB 693MB 1013kB 506kB 49MB
512MB 338MB 800kB 247kB 24MB

Hopefully, this makes your life easier!

>> For more information:

About the Author:  Michael Paquier is a member of PostgreSQL technical staff at VMware. He is involved for many years with community development of PostgreSQL and Postgres-XC, and has worked on multi-master database technologies. He has also interest in parallel query processing and concurrent SQL processing.

One thought on “Dynamic Memory Management of vFabric Postgres

  1. Pingback: Scaling for the Information Explosion: Master-Slave Cluster with vFabric Postgres 9.2 on vSphere | VMware vFabric Blog - VMware Blogs

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>