Home > Blogs > VMware vFabric Blog > Tag Archives: performance

Tag Archives: performance

How fast is a Rabbit? Basic RabbitMQ Performance Benchmarks

One of the greatest things about RabbitMQ is the community that surrounds it. With open source at its roots, people come together to share their code, their knowledge and their stories of how they’ve deployed it in their projects. At a recent meetup near Nice, France, database engineer Adina Mihailescu shared a presentation on choosing messaging systems. Supported by Murial Salvan’s benchmark comparing ActiveMQ, RabbitMQ, HornetQ, Apollo, QPID, and ZeroMQ, they shared some interesting performance comparisons that we’d like to share with you.

In a single laptop benchmark, Salvan ran four different scenarios in order to obtain some insight on performance of the default setups for these messaging solutions. Each test had 1 process dedicated to enqueuing and another dedicated to dequeuing. The message volume and size ranged from 200 to 20,000 to 200,000 messages and 32 to 1024 to 32768 bytes. Both persistent and transient queues and messages were used. Continue reading

Behind the Scenes: Patching PostgreSQL for Performance–vFabric Team Contributes to Open Source

After PostgreSQL 9.2 was released, users that relied on PostgreSQL for scale, may have noticed a performance hit. In fact, the PostgreSQL community alongside the VMware vFabric Postgres team, was able to prove that the new version demonstrated a 10% performance hit over version 9.1. As part of the VMware Postgres team, we wanted to fix this problem for our own distribution, but as mentioned in previous posts, we also wanted to contribute our fixes back to the common core.  This post provides additional detail on how this problem was identified and how we worked with the open source PostgreSQL community to restore performance.

Background on the Performance Issue in PostgreSQL 9.2

Last year, during routine regression testing of vFabric Postgres, we found that PostgreSQL 9.2, the latest major release of PostgreSQL, demonstrated a significant performance regression from version 9.1. Using DBT-2, an open-source and fair-use implementation of TPC-C benchmark [1], we noticed a 10% performance degradation, which we then reported to the community [2].

To troubleshoot the problem we used git bisect to find the type of commit that caused the performance problem and cross-examined the statistical profiles using oprofile. As it turns out, the regression was caused by a commit that changed the way memory was allocated when SPI queries were executed. The commit was intended to reduce the number of allocations for queries using a cached plan at the cost of more logistics work. However, according to the DBT-2 test, we could see that this tradeoff was unfavorable for dynamic queries. So to fix it, we would need reintroduce the original tradeoff on its intended queries using conditions [3].

We proposed the fix to the wider PostgreSQL community and the ensuing discussion led to a refined resolution which was implemented in a patch [4]. This patch has been back-ported to the latest PostgreSQL 9.2.3 release and is included in the latest vFabric Postgres release [5]. Continue reading

3 Key Performance and Scale Improvements with VMware vFabric Postgres 9.2

Recently, vFabric Postgres 9.2 launched with additional cloud computing capabilities like elastic memory management. Some of the most compelling new features are performance-related and take linear scaling to new levels.

This article will cover 3 key improvements as listed below:

  • 4x Improvement with vertical linear scaling for reads
  • 2x Improved write efficiency for write ahead logs
  • Index Only Scans and More

4x Improvement with vertical linear scaling for reads

Modern websites are almost all database driven. When consumers browse online retailer catalogs, 99% of the load is reads and 1% of the load is updates to the data on the tables. Even in highly updated websites, the grand majority of load is from reads. In these high-read usage scenarios, the database needs to handle a high read load on certain tables compared to the other tables in the database. We’ve seen this behavior drive enhancements within databases. For example, many application designs started putting a caching mechanism in their application to limit the database hits. Continue reading

Advanced GemFire + Lucene + Spring Data for Text Searching

In this article, we’ll talk about how to integrate the Lucene text searching solution using Spring Data and GemFire to provide a flexible, parallel fast search engine. By combining the two independent products we can leverage each product to its fullest capability. The end result provides an elastic search capability with the in memory data speeds of a distributed cache platform and high availability.

Motivation—Why Combine These?

The motivation of the project was to provide an alternative search capability for GemFire while providing users a natural method to define searchable domain object attributes. Performance was also a key driver to ensure constant search performance irrespective of scale. The solution outlined below provides a baseline approach for developers to build upon. Continue reading

Scaling and Modernizing .NET and Java: SQLFire Performance Test Blows Away Traditional RDBMS

We all know the devil is in the details when it comes to technology.

Yet, our recent vFabric SQLFire performance test (a benchmark from vFabric SQLFire Best Practices Guide) is certainly worth review if you need to scale a Java app, .NET app, or other legacy data source.

If you don’t know what vFabric SQLFire is, it is basically what happens when Apache Derby gets married to vFabric GemFire:

  • Apache Derby is used for its RDBMS components, JDBC driver, query engine, and network server.
  • The partitioning technology of GemFire is used to implement horizontal partitioning features of vFabric SQLFire.
  • vFabric SQLFire specifically enhances the Apache Derby components, such as the query engine, the SQL interface, data persistence, and data eviction, as well as adding additional components like SQL commands, stored procedures, system tables, functions, persistence disk stores, listeners, and locators, to operate a highly distributed and fault tolerant data management cluster.

Continue reading

Messaging Architecture: Using RabbitMQ at the World’s 8th Largest Retailer

Today, we are pleased to have a guest blogger from a VMware customer share with us their story of how RabbitMQ transformed their business by “solving some really interesting problems”. The following is sent courtesy of Pablo Molnar of MercadoLibre:

If you haven’t heard of MercadoLibre (NASDAQ: MELI), we are the largest e-commerce ecosystem in Latin America. Our website offers a wide range of services to sellers and buyers throughout the region including marketplace, payments, advertising, and e-building solutions. Our products are present in over 14 countries, and the company is ranked as 8th largest online retailer in the world. We were also on Fortune’s list of the fastest growing companies in 2012, and we use RabbitMQ to solve some interesting problems.

About Our Technology Stack and How RabbitMQ Helps

In terms of technology infrastructure, MercadoLibre is fully committed to the open source development model. Most of our apps are primarily written in Grails, Groovy, and NodeJS,  but we don’t stick to any language or framework. We entrust tool selection responsibilities to the Software Engineers on each team. Almost all applications are hosted by our in-house cloud computing provisioning system and implemented via OpenStack with more than +7000 virtual instances at the moment. Also, we have successfully launched applications using emerging storage solutions like Redis and MongoDB. With an average of 20 million requests per minute and 4GB bandwidth per second, our traffic management layer is crucial and most of the routing rules job is done by Nginx proxy servers. Our labs department includes a huge Apache Hadoop cluster to perform complex analytical queries, and we are experimenting with real-time data processing using Apache Kafka and Storm.

Continue reading

The Best VMware vFabric Stories of 2012 & What’s In Store for 2013

As this year comes to a close, it’s time to be reflective of what happened in the past and start planning for a new year. The vFabric team has had some major achievements this year, introducing several new products to the market including the innovative vFabric Application Director, the widely anticipated Project Serengeti to enable rapid cloud deployments for Hadoop, and a new tool to the vFabric Suite users called vFabric Administration Server (VAS).  We announced a new VMware Cloud Applications Marketplace to help further accelerate application development with a professionally moderated library of enterprise grade, ready-to-use application components that can be run on any cloud.

Next year is going to be even bigger with the Pivotal Initiative where several of the products covered on this blog will be following the new venture. This is still in the planning stages, so we will be expecting to share with you the plans for our products alongside the formal communications from each of the companies involved. (Sorry — no extra information is available right now)

One thing that we are going to be doing in early 2013 is to move the conversation of how you manage applications to be with the conversations of how you manage virtual infrastructure. To that end, we will be moving all topics of Application Performance Manager, AppInsight, Application Director, Hyperic, and Spring Insight to the VMware Management Blog as of January 1st. To make sure you keep up with the management topics, please be sure to follow us @vmwareappmgmt and @vmwaremgmt.

In the meantime, we’d like to reshare with you the top 20 stories we had for 2012, and invite you to comment here on what stories you would like to see us cover on either blog for 2013.

Continue reading

How to Prepare for a Hyperic 5.0 Upgrade

Recently, we announced a new version of our popular web infrastructure management tool, Hyperic 5.0.  We also explained how, in this new version, the backend data store is moving to support PostgreSQL. In this article, we are providing existing Hyperic users the information they need to plan their upgrade to Hyperic 5.0. This article is for anyone already running Hyperic HQ (open source), vFabric Hyperic, vFabric Application Performance Manager, or vCenter Operations Manager.

This post is meant to augment the knowledge base article, KB 2033940, published back in August.

Planning the Upgrade

We know platform changes can make an upgrade more difficult and certainly raise eyebrows. So, we’ve taken measures to help make the migration as seamless and simple as possible. So far, the cases we’ve seen take about an hour. As with any data migration, the greater the volume of database records, the longer it can take. Continue reading

A Tale of Modernization: Stopping Bank Robbers as a Service (20 Billion Times to Date)

Pirates. Pick pockets. Bank Robbers.

What do these have to do with applications modernization?

Well, bank robbers have turned digital, and this article is about modernizing one of the most successful systems used to stop them.

Stealing credit card numbers, emptying online bank accounts and stealing identities is now big business for thieves and consequently a big area for software companies and banks to collaborate and stop them. Cybercrime and malware have become such an issue that, according to Gartner, the web fraud detection market (where RSA competes) grew 35% in 2010 and 25% in 2011.

RSA is in the business of stopping banks and their customers from being robbed (among other things). Their technology has protected people, businesses, and financial institutions from online fraud for almost 20 years. Their Adaptive Authentication solution is deployed at over 8000 companies, used by over 200 million people, and has protected over 20 billion transactions to date. To jump on the “everything as a service” bandwagon, Adaptive Authentication is literally embarking on a project to “Stop Bank Robbers as a Service.”

We had the opportunity to catch up with the RSA engineering team that is developing this next-generation service using a wide range of VMware capabilities, including VMware vCloud Suite and vFabric. We learned that VMware virtualization infrastructure management, application performance management, automated provisioning, and application servers are already providing benefits along with improved runtimes, and got a peek into how they plan to do the financial side of IT business management using VMware tools.

Continue reading

How We Improved APM’s Monitoring Capacity by 500% with Gemfire, RabbitMQ, vPostgres, and a Few Scalability Design Patterns

Earlier this week, we announced the general availability of a major upgrade to vFabric Application Performance Manager (APM). This release started one year ago, after we released the first version of the product to market. When we started work on this release, we knew we would need to invest heavily in scalability. APM is designed to help simplify monitoring and management for highly dynamic, large web applications living in the cloud. To succeed, we needed to make sure our product could scale gracefully with our customers. So, we set out with a challenging goal to increase the capacity of APM by a factor of 5.

Transforming a complex product such as APM into a more scalable architecture is not an easy task, let alone doing so in a single release. For this reason we’ve started by modifying the architecture in steps, starting with local improvements inside our virtual appliance, (available in the APM 5.0 release) and moving towards a horizontal scale solution in future releases. Continue reading