Home > Blogs > VMware vFabric Blog > Tag Archives: SQLFire

Tag Archives: SQLFire

Scaling and Modernizing .NET and Java: SQLFire Performance Test Blows Away Traditional RDBMS

We all know the devil is in the details when it comes to technology.

Yet, our recent vFabric SQLFire performance test (a benchmark from vFabric SQLFire Best Practices Guide) is certainly worth review if you need to scale a Java app, .NET app, or other legacy data source.

If you don’t know what vFabric SQLFire is, it is basically what happens when Apache Derby gets married to vFabric GemFire:

  • Apache Derby is used for its RDBMS components, JDBC driver, query engine, and network server.
  • The partitioning technology of GemFire is used to implement horizontal partitioning features of vFabric SQLFire.
  • vFabric SQLFire specifically enhances the Apache Derby components, such as the query engine, the SQL interface, data persistence, and data eviction, as well as adding additional components like SQL commands, stored procedures, system tables, functions, persistence disk stores, listeners, and locators, to operate a highly distributed and fault tolerant data management cluster.

Continue reading

6 Simple Steps: Installing vFabric Administration Server (VAS) and Provisioning tc Server across 3 Nodes

As a suite of products, vFabric offers a single SKU model that includes the entire suite. Besides offering an economically beneficial licensing model that is flexible, based on average use, and hardware independent, the vFabric team has developed the vFabric Administration Server (VAS) to help make installation, configuration, and operations easier.

If you missed a previous article explaining how VAS work and what it can do for you, VAS is based on a single system image design (SSI) and allows you to define a group of nodes where each operation on the group is applied to each node who is a member of the group. This approach provides time savings and reduces errors by keeping systems more consistent, allowing easier ways to scale, and simplifying administration. VAS also provides a robust set of REST APIs for virtually all of its functionality. In addition to vFabric tc Server, VAS can manage vFabric RabbitMQ, vFabric GemFire, vFabric SQLFire, and vFabric Web Server. VAS 1.1.1 was recently released with new features and many bug fixes.

In this article (and demonstration further below), we will show you six steps that give you an idea of how easy it is to provision using VAS. We will show you how to install VAS and use it to provision vFabric tc Server across three nodes along with a WAR file. The explanation below refers to examples from RubyGems.org and GitHub/vFabric/VAS-Ruby-API along with the latest VAS documentation.

Continue reading

Big, Fast Data Opportunities in Mobile Applications

Mobile applications are one thing, but mobile apps WITH fast data requirements are another.

The combination of mobile apps and fast data requirements can cause major data scale issues. Whether you are trying to update an existing application or build a new application, mobile apps with personalization, pricing, location, or gaming functionality must consider data architecture differently from the outset.

The Fundamental Growth Characteristics of Mobile

It’s official, mobile applications now account for over 50% of the computing market. It crept up on us quickly, and it’s easy to forget that mobile growth is different than what we’ve seen before. Of course, most people in the mobile space know about the growth. You would have to be living in a country without internet, TV, or radio if you haven’t heard that mobile usage is growing tremendously along with mobile applications. But, let’s put this in perspective: AOL took 2.5 years to reach 50 million users and Angry Birds took 35 days. Yes, Angry Birds was a smash hit, but so was AOL.

An AT&T Senior EVP recently wrote, “Over the past five years, AT&T’s wireless data traffic has grown 20,000%. The growth is now primarily driven by smartphones.” In fact, many say that mobile use will cause a spectrum deficit in the U.S. According to the Telegraph, smartphones are mostly used for internet (24 minutes and 29 seconds per day) and social media (17 minutes and 29 seconds per day) while phone calls are ranked 5th (12 minutes and 6 seconds per day).  Similarly, mobile commerce is planned to rise from 1% of all e-commerce sales in 2010 to 7% in 2016 (i.e. from $3 billion to $31 billion in a 6 years period). Apps are also accounting for more minutes of usage. So, no wonder business groups are clamoring for mobile-centric programs and applications.

The bottom line is that mobile applications are growing data differently than traditional database applications.

Continue reading

3 Signs Your Relational Database Must Go

Application and operations teams sometimes reach a point where they must upgrade the database. Whether it’s due to data growth, lack of throughput, too much downtime, the need to share data globally, adding ETLs, or otherwise, it’s never a small project. Since these projects are expensive, any recommendation requires a solid justification.  This article a) characterizes 3 signs where traditional databases hit a wall, b) explains how vFabric SQLFire provides an advantage over traditional databases in each case, and c) should help you make a case for moving towards an in-memory, distributed data grid based on SQL.

For those of us tasked with upgrading (or architecting) the data layer, we all go through similar steps. We build a project plan, make projections and sizing estimates, perform architecture and code reviews, create configuration checklists, provide hardware budgets and plans, talk to vendors about options, and more.  Then, we work to plan the deployment with the least downtime, procure hardware and software, test different data load times, evaluate project risks, develop back-up plans, prepare communications to users about downtime, etc. You know the drill. These projects can take months and consume a fair amount of internal resources or consulting dollars. If you are starting or working on one of these types of projects with a traditional database architecture in mind, are you considering these 3 signs as you consider your options? Continue reading

3 Key Stages to Evolve from Legacy Databases to a Modern Cloud Data Grid

How do you plan a roadmap for moving from a legacy data architecture to a cloud-enabled data grid? In this article, we will offer a pragmatic, three-stage approach. At SpringOne-2012, the “Effective design patterns with NewSQL” session (see presentation embedded below) generated a lot of interest. (Thank you to everyone who joined us!) Jags Ramnarayan and I discussed problems with legacy RDBMS systems, NewSQL driving principles, SQLFire architecture, application design patterns as well as data consistency and reliability.

We went deep into vFabric SQLFire which is a pragmatic solution that addresses these data challenges:

  • How do I architect my data tier for very high concurrent workloads?
  • How do I achieve predictability both for data access response time and availability?
  • How do I distribute data efficiently and real time to multiple data centers (and to external clouds)? 
  • How do I process these large quantities of data in an efficient manner to allow for better real-time decision-making? 

Continue reading

vFabric @ SpringOne Next Week

The vFabric team is headed to SpringOne 2GX 2012 next week – from October 15-18 in Washington, DC. This is set to be a great event to learn the latest on Spring with over 100 sessions covering a wide variety of topics. For those of you looking to learn more about how vFabric is the best place to run Spring applications, here are the highlights you won’t want to miss:

1. Sessions:  There are a number of speakers from SpringSource, CloudFoundry, and the VMware vFabric team on the schedule, including:

Continue reading

Field Report VMworld Europe Day 3

Over 37,000 VMs have been started in Hands on Labs. Hands on Labs run in Hall 6 through Thursday. vFabric labs include Application Director, Data Director, CloudFoundry, SQLFire and scaling with vFabric.

As noted in the Field Report from Day 2, we greatly appreciate the opportunity to listen and learn from our VMworld Europe customers, prospects, and partners! VMworld this year has not dissapointed. We’ve met with hundreds of partners, thousands of customers and spun up tens of thousands of virtual machines in our hands on labs. Over 37,000 in fact at the start of Wednesday, with two more full days to go! An impressive event all around.

Not surprisingly after the keynote yesterday, customer discussions have continued on Application Director, with many stating explicitly that they are going to use this as the cornerstone opportunity to build out a self-service infrastructure, just like VMware’s own IT did.

Beyond that, a few other key themes have emerged:

Significant vFabric Interest from VMware Partners
Monday was a partner day, and there is a lot of interest in vFabric from partners.  They are looking to VMware to learn and improve how they are building apps for their customers in cloud environments. For one of our presentations on Monday, we had a packed house with over 100 people in the audience.
Continue reading

VMware Expert on Virtualizing and Tuning Large Scale Middleware

Virtualization TuningAt VMworld-2012 San Francisco, the session APP-CAP1426 – The Benefits of Virtualization for Middleware was greatly attended, and we want to thank all of the attendees that helped us score 4.5 out of 5 in our survey results.  Because of this, the session is going to be presented at VMworld-2012 Barcelona, and we are posting related information here in this article.  Before reading this article, you might want to take a look at the related blog post we released before VMworld-2012 San Francisco.

NOTE: Just like in VMworld-2012 San Francisco, at Barcelona we will raffle copies of my book the Enterprise Java Applications Architecture on VMware

This year at the session, we went deep into tuning large-scale middleware and the discussion around JVM tuning was well received.  Hence as a follow-up, I wanted to share some of my more recent research, which will be discussed at VMworld-2012 Barcelona.  This focuses on tuning in-memory data management systems such as vFabric SQLFire.

The article below covers:

  • An Overview of GC Tuning
  • Parallel GC in Young Generation and CMS in Old Generation
  • GC Tuning Recipe
    • Step A – Young Generation Tuning
    • Step B – Old Generation Tuning
    • Step C – Survivor Spaces Tuning
  • JVM and GC Best Practices for vFabric SQLFire Members

Continue reading

3 Insightful vFabric Implementations to Learn From

At VMworld in San Francisco, several partners shared success stories at the vFabric partner panel.  As moderator for this session, I worked extensively with the three vFabric Partners prior to the event and uncovered a common theme – these experts are top notch at what they do and enjoy doing it.  Each Partner had a unique vFabric story about successful implementations.  Partners on the panel included Nancy Turbe with EMC consulting, Jeff Reed with Logicalis and Juan Garcia from Williams and Garcia.  Their stories covered:

 

Register for VMworld!
Click Here

Register for Session PAR1896 – vFabric Products and Services: Strategies for SI’s to Grow Revenue, Undercut the Competition:
Click Here

Follow all vFabric updates at VMworld on Twitter:
Click Here

Follow us @vmwareappmgmt:
Click Here

  1. A low-cost, high performance financial trading platform for the cloud based on vFabric Nanotrader,
  2. A fast, scalable software architecture deployed in the cloud for a loyalty card program, and
  3. An elastic data management solution for reloadable payment cards.

Products and services represented across the three stories ran the entire length of the vFabric stack including tc Server, EM4J, SQLFire, GemFire, Data Director, Application Director, RabbitMQ, Application Performance Manager, and more. Below, I’ve included a short overview and some compelling insights from each panelist at the session. Continue reading

Load 8 Million Rows in 88 Seconds – NewSQL Speed

At some point, any data modernization project is going to require a load of legacy data. With an in-memory, distributed data store like SQLFire, customers often ask (like in this case) about load times because they can be sitting on 50-100 GB of data and don’t want to wait days. For those unfamiliar with NewSQL databases, this post should give you a good sense of how we loaded 8 million rows in 88 seconds. The test shows how we should be able to load roughly 40GB of data in about 1 hour.

For Java developers who want ideas about speeding up large volumes of calculations, transforms, or validations, you may want to consider a previous post, where SQLFire is used with Spring Batch.

With SQLFire, we take a multi-threaded load approach from a CSV file. Below, I’ve outlined 8 steps to the load strategy with an explanation of why things were done. Continue reading