Home > Blogs > VMware vFabric Blog


Increase performance and scalability of existing applications using SQLFire

The Challenge

Traditional databases should not be used to do things for which they were never designed; like supporting thousands of concurrent users.

The main challenge of managing Web applications on a Cloud-scale is performance. Disk-based database architectures are fine when you have a small number of users, but they lack the facilities for horizontal scaling, and, are unable to address the variable access patterns.

In contrast, SQLFire, an in-memory database from VMware, was designed specifically for these kinds of challenges. With its speed and low latency, SQLFire delivers dynamic scalability and high performance for modern, data-intensive applications, all through a familiar SQL interface.

In this post, I will demonstrate one of the ways SQLFire can increase throughput and decrease latency of your current Web applications.

The Bottleneck

One of the most prevalent bottlenecks of today’s online applications is the database. Web and Mobile applications place increasing pressure on the data tier of these solutions, and, their current disk-based architectures simply inherit too much latency to deal with these kinds of workloads. Traditional databases are the wrong tool for this job.

New-db-preasure
Scaling Web applications creates database bottleneck

Disk-based database architecture simply can’t deal with these kinds of read-write access patterns, and, beyond a certain levels, exhibit a catastrophic collapse phenomenon.

The Solution

While it would be easy to assume that the only way to elevate this bottleneck is to embark on a lengthy effort to re-write your applications against one of those “big data” solutions everyone seems to be talking about, the truth is, that this kind of effort for many enterprise applications would be very expensive and only further balkanize their data by limiting it accessibility to other data consumers.

To increase throughput and decrease latency of your solution at that scale, you need to bring the data closer to each one of the data consumption points.

New-db-preasure-resolved
SQLFire scales horizontally, while preserving legacy database access

SQLFire provides that extra performance with the familiarity of SQL interface. It accelerates your application performance, minimizing latency and increasing overall reliability by pooling memory, CPU and network resources across a cluster of machines.

In addition to the performance, SQLFire also delivers numerous build-in data synchronization features. For example, in order to continue supporting legacy data-flows in your current solution, SQLFire can make your new “fast data” also available to back-office applications.

SQLFire’s asynchronous “write behind” pattern has been designed for very high reliability and can handle many failure conditions as persistent queues ensure write operations are performed even when the backend database is temporarily unavailable. SQLFire supports multiple options for how these events can be queued, batched, and written to the underlining database.

The Implementation

Whether you are managing Java or .NET solution, in most scenarios, pointing your application to SQLFire database is as simple as replacing the application database connection string. But, before you can do that, you will need to replicate your current data identities into a SQLFire database.

Because SQLFire supports standard ANSI SQL-92, the act of replicating databases schema in most cases is pretty simple and verily automated. Most DBAs pose numerous commercial tools to perform these tasks.

If necessary, there are also readily available, open-source utilities like Apache DdlUtils or SQLF, the SQLFire command line.

However, to truly benefit from SQLFire’s in-memory performance and linear scalability, we should take some time to design a horizontal data partitioning strategy. This is the act of spreading a large data set (many rows in a table) across multiple servers.

SQLFire uses a partition key to ensure that data is uniformly balanced across all members of a data grid. Correctly identifying the partition key will avoid expensive queries across multiple partitions and enable your highly concurrent system to handle thousands of connections and allow multiple queries to be uniformly spread across the entire data.

In contrast to the highly volatile data, which greatly benefits from partitioning, some of your data elements are pretty static (like lookup tables) and are better replicated across all nodes in the data grid to further increase the performance of individual queries.

You can apply all these extended DDL attributes to tour SQLFire schema after creation, using the “ALTER” command. For more information on these and other ways of managing your SQLFire data, see the Managing Your Data in vFabric SQLFire documentation.

The Source

To go along with this post, I put together a sample application illustrating integration of SQLFire in your online application. The source code of this application is available on GitHub.

The Demo

To demonstrate the impact of SQLFire, I deployed this sample application to a shared hosting environment. While the specific database performance depend on many variables, the purpose of this short demo is to focus on the relative gains SQLFire delivers to a generic Web application while continuing to support the legacy database.

The Conclusion

Hopefully, you got a glimpse of the scale-out nature of SQLFire and the flexibility of the different use-cases, which it can address. As with any technology, there is no substitute for rolling up your sleeves and trying it yourself. SQLFire licensing allows for up to three nodes to be deployed free of charge for development purposes. Go ahead and see for yourself how much faster SQLFire can make your data.

This entry was posted in SQLFire and tagged , , , , , , , , on by .
Mark Chmarny

About Mark Chmarny

During his 15+ year career, Mark Chmarny has worked across various industries. Most recently, as a Cloud Architect at EMC, Mark developed numerous Cloud Computing solutions for both Service Provider and Enterprise customers. As a Data Solution Evangelist at VMware, Mark works in the Cloud Application Platform group where he is actively engaged in defining new approaches to distributed data management for Cloud-scale applications. Mark received a Mechanical Engineering degree from Technical University in Vienna, Austria and a BA in Communication Arts from Multnomah University in Portland, OR.

7 thoughts on “Increase performance and scalability of existing applications using SQLFire

  1. web design company

    Popular implementations of SQL commonly omit support for basic features of Standard SQL, such as the DATE or TIME data types. The most obvious such examples, and incidentally the most popular commercial, proprietary SQL DBMSs, are Oracle (whose DATE behaves as DATETIME, and lacks a TIME type) and the MS SQL Server (before the 2008 version). As a result, SQL code can rarely be ported between database systems without modifications.
    Alexis Bob

    Reply
  2. Adam BloomAdam Bloom

    Alexis, sorry for the late reply–didn’t see it originally come across. Thank you for pointing that out. Yes, every situation is unique…we don’t mean to imply otherwise. Many customers find the required changes to have relatively little heavy lifting, and the overall business value often outweighs minor modifications.

    As background, the SQL engine for SQLFire is based on Apache Derby. For Grails apps, there is a sample code change here: http://www.theserverside.com/news/thread.tss?thread_id=71354

    Reply
  3. پنل ارسال اس ام اس در مشهد

    vos sms panel in mashad

    Reply
  4. کاندوم خاردار

    دانلود آهنگ

    Reply
  5. جهیزیه عروس

    ما به همه شما توصیه میکنیم اگر قصد خرید سرویس جهیزیه عروس را دارید از سایت نوین جهاز که در حوزه لوازم آشپزخانه فعالیت می کند این کار را انجام دهید.

    Reply
  6. دانلود آهنگ خارجی

    Thank you very much for your good post

    Reply
  7. دی موزیک

    tnx merc v

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *

*