Despite what people tell you, managing on-line applications on a cloud-scale is hard. One of the main challenges is related to the fact that as an application gets more and more popular, the underlining database often becomes the bottleneck.

When demand spikes, organizations are comfortable scaling their Web and App Server layers. However, as they increase the number of application instances to accommodate the growing demand, their data layer is unable to keep up.

We all know that a solution’s overall performance is only as good as it’s lowest common denominator. Increasingly, the lowest common denominator of today’s on-line applications is the database.

A Customer Example

Recently, a large retail customer spoke to us about their experiences in dealing with demand spikes during holidays. Their virtualized infrastructure was more than capable of scaling horizontally to address the growing demand. However, their underlying, traditional database could not handle the large load increases. The database started to experience deadlocks, connection timeouts, and various other problems.

There could be many reasons for this behavior, but the overarching cause is clear. Traditional databases were never designed to support thousands of concurrent users and don’t scale well to meet the ever-increasing needs of today’s on-line applications.

Two Key Limits of Traditional Database Architecture

The two specific areas where the traditional database architecture is particularly limiting is IO and Transaction management.

Let’s start with IO. Bottom line, there is simply too much of it! The implicit disk synchronization requirements create bottlenecks. Given the constraints under which the original database architects worked, it was only natural for them to focus on maintaining strict data consistency through many of the locking and latching techniques. Despite recent advances in IO, these patterns simply no longer scale to meet today’s demand patterns.

The second limiting factor of traditional databases is transaction management. Here is some data from recent study conducted by MIT and Brown, which analyzed time consumption of the transactions in traditional databases.

As you can see, only about 12% of the time goes to processing data requests. The rest is related to managing disk buffers, locks and latches. Again, the focus on IO and fascination with ACID do not scale in the cloud!

If you follow data-related issues on the Web, you may hear that an expensive database clustering system or a lengthy re-write effort using one of those “big data” solutions are the only two approaches to achieve scale.

A Solution for Elastic, Distributed Data in the Cloud

While there may be some scenarios where these options are necessary, there is perhaps a better alternative:

Consider SQLFire. It was built specifically for the kinds of cloud-scale data challenges we just described. As an elastic, distributed, in-memory data platform, SQLFire delivers both the necessary speed and latency demanded by today’s on-line applications. And, the best part, SQLFire can do it using the already familiar SQL interface, along with client drivers for JDBC, and ADO.NET.

For examples of integrating SQLFire into your application see the Spring Petstore sample code. It includes SQLFire integration with QueryDSL. (https://github.com/SpringSource/spring-sqlfire-samples)

SQLFire delivers a pool of shared memory and disk resources to create one virtual data fabric. This data fabric is optimized for main memory to manage highly concurrent data structures and has no need for buffering contiguous disk blocks.

SQLFire also delivers a new approach to ACID transactions. Instead of worrying about “read ahead” transaction logs on disk, SQLFire avoids any single point of contention by using a two-phase commit protocol optimized for small duration, non-overlapping transactions.

Some of the other unique characteristics of SQLFire worth considering are:

• Partition aware database design which spreads workloads across both data set and physical nodes,
• Shared nothing disk architecture which prevents application writes from being exposed to the disk seek latencies, and
• Dynamic rebalancing of data as cluster size grows/shrinks, assuring the most efficient way of managing resources/data.

SQLFire = Speed and Scale

We recently released a sample app, blog post, and a video about specific performance comparisons of SQLFire against traditional databases. You can find the details here.

As far as scale,  we are going to release a webcast this coming Wednesday on spanning your fast data around the globe with a real-life demonstration of long distance data synchronization. To register for the webinar, “SQLFire – Fast Data that Spans the Globe” go to the signup page.

Hopefully, this short overview illustrated that SQLFire is better in supporting on-line applications than traditional databases. As with any technology, there is no substitute for rolling up your sleeves and trying it out for yourself. SQLFire licensing allows for up to three nodes to be deployed free of charge for development purposes. Go ahead and see for yourself how you can benefit your on-line applications with a free 60 day trial of SQLFire.