Apache Derby is used for its RDBMS components, JDBC driver, query engine, and network server.
The partitioning technology of GemFire is used to implement horizontal partitioning features of vFabric SQLFire.
vFabric SQLFire specifically enhances the Apache Derby components, such as the query engine, the SQL interface, data persistence, and data eviction, as well as adding additional components like SQL commands, stored procedures, system tables, functions, persistence disk stores, listeners, and locators, to operate a highly distributed and fault tolerant data management cluster.
In this article (and demonstration further below), we will show you six steps that give you an idea of how easy it is to provision using VAS. We will show you how to install VAS and use it to provision vFabric tc Server across three nodes along with a WAR file. The explanation below refers to examples from RubyGems.org and GitHub/vFabric/VAS-Ruby-API along with the latest VAS documentation.
Mobile applications are one thing, but mobile apps WITH fast data requirements are another.
The combination of mobile apps and fast data requirements can cause major data scale issues. Whether you are trying to update an existing application or build a new application, mobile apps with personalization, pricing, location, or gaming functionality must consider data architecture differently from the outset.
An AT&T Senior EVP recently wrote, “Over the past five years, AT&T’s wireless data traffic has grown 20,000%. The growth is now primarily driven by smartphones.” In fact, many say that mobile use will cause a spectrum deficit in the U.S. According to the Telegraph, smartphones are mostly used for internet (24 minutes and 29 seconds per day) and social media (17 minutes and 29 seconds per day) while phone calls are ranked 5th (12 minutes and 6 seconds per day). Similarly, mobile commerce is planned to rise from 1% of all e-commerce sales in 2010 to 7% in 2016 (i.e. from $3 billion to $31 billion in a 6 years period). Apps are also accounting for more minutes of usage. So, no wonder business groups are clamoring for mobile-centric programs and applications.
The bottom line is that mobile applications are growing data differently than traditional database applications.
Application and operations teams sometimes reach a point where they must upgrade the database. Whether it’s due to data growth, lack of throughput, too much downtime, the need to share data globally, adding ETLs, or otherwise, it’s never a small project. Since these projects are expensive, any recommendation requires a solid justification. This article a) characterizes 3 signs where traditional databases hit a wall, b) explains how vFabric SQLFire provides an advantage over traditional databases in each case, and c) should help you make a case for moving towards an in-memory, distributed data grid based on SQL.
For those of us tasked with upgrading (or architecting) the data layer, we all go through similar steps. We build a project plan, make projections and sizing estimates, perform architecture and code reviews, create configuration checklists, provide hardware budgets and plans, talk to vendors about options, and more. Then, we work to plan the deployment with the least downtime, procure hardware and software, test different data load times, evaluate project risks, develop back-up plans, prepare communications to users about downtime, etc. You know the drill. These projects can take months and consume a fair amount of internal resources or consulting dollars. If you are starting or working on one of these types of projects with a traditional database architecture in mind, are you considering these 3 signs as you consider your options? Continue reading →
How do you plan a roadmap for moving from a legacy data architecture to a cloud-enabled data grid? In this article, we will offer a pragmatic, three-stage approach. At SpringOne-2012, the “Effective design patterns with NewSQL” session (see presentation embedded below) generated a lot of interest. (Thank you to everyone who joined us!) Jags Ramnarayan and I discussed problems with legacy RDBMS systems, NewSQL driving principles, SQLFire architecture, application design patterns as well as data consistency and reliability.
We went deep into vFabric SQLFire which is a pragmatic solution that addresses these data challenges:
How do I architect my data tier for very high concurrent workloads?
How do I achieve predictability both for data access response time and availability?
How do I distribute data efficiently and real time to multiple data centers (and to external clouds)?
How do I process these large quantities of data in an efficient manner to allow for better real-time decision-making?
The vFabric team is headed to SpringOne 2GX 2012 next week – from October 15-18 in Washington, DC. This is set to be a great event to learn the latest on Spring with over 100 sessions covering a wide variety of topics. For those of you looking to learn more about how vFabric is the best place to run Spring applications, here are the highlights you won’t want to miss:
Over 37,000 VMs have been started in Hands on Labs. Hands on Labs run in Hall 6 through Thursday. vFabric labs include Application Director, Data Director, CloudFoundry, SQLFire and scaling with vFabric.
As noted in the Field Report from Day 2, we greatly appreciate the opportunity to listen and learn from our VMworld Europe customers, prospects, and partners! VMworld this year has not dissapointed. We’ve met with hundreds of partners, thousands of customers and spun up tens of thousands of virtual machines in our hands on labs. Over 37,000 in fact at the start of Wednesday, with two more full days to go! An impressive event all around.
Not surprisingly after the keynote yesterday, customer discussions have continued on Application Director, with many stating explicitly that they are going to use this as the cornerstone opportunity to build out a self-service infrastructure, just like VMware’s own IT did.
Beyond that, a few other key themes have emerged:
Significant vFabric Interest from VMware Partners
Monday was a partner day, and there is a lot of interest in vFabric from partners. They are looking to VMware to learn and improve how they are building apps for their customers in cloud environments. For one of our presentations on Monday, we had a packed house with over 100 people in the audience. Continue reading →
At VMworld-2012 San Francisco, the session APP-CAP1426 – The Benefits of Virtualization for Middleware was greatly attended, and we want to thank all of the attendees that helped us score 4.5 out of 5 in our survey results. Because of this, the session is going to be presented at VMworld-2012 Barcelona, and we are posting related information here in this article. Before reading this article, you might want to take a look at the related blog post we released before VMworld-2012 San Francisco.
NOTE: Just like in VMworld-2012 San Francisco, at Barcelona we will raffle copies of my book the Enterprise Java Applications Architecture on VMware
This year at the session, we went deep into tuning large-scale middleware and the discussion around JVM tuning was well received. Hence as a follow-up, I wanted to share some of my more recent research, which will be discussed at VMworld-2012 Barcelona. This focuses on tuning in-memory data management systems such as vFabric SQLFire.
The article below covers:
An Overview of GC Tuning
Parallel GC in Young Generation and CMS in Old Generation
GC Tuning Recipe
Step A – Young Generation Tuning
Step B – Old Generation Tuning
Step C – Survivor Spaces Tuning
JVM and GC Best Practices for vFabric SQLFire Members
At VMworld in San Francisco, several partners shared success stories at the vFabric partner panel. As moderator for this session, I worked extensively with the three vFabric Partners prior to the event and uncovered a common theme – these experts are top notch at what they do and enjoy doing it. Each Partner had a unique vFabric story about successful implementations. Partners on the panel included Nancy Turbe with EMC consulting, Jeff Reed with Logicalis and Juan Garcia from Williams and Garcia. Their stories covered:
At some point, any data modernization project is going to require a load of legacy data. With an in-memory, distributed data store like SQLFire, customers often ask (like in this case) about load times because they can be sitting on 50-100 GB of data and don’t want to wait days. For those unfamiliar with NewSQL databases, this post should give you a good sense of how we loaded 8 million rows in 88 seconds. The test shows how we should be able to load roughly 40GB of data in about 1 hour.
For Java developers who want ideas about speeding up large volumes of calculations, transforms, or validations, you may want to consider a previous post, where SQLFire is used with Spring Batch.
With SQLFire, we take a multi-threaded load approach from a CSV file. Below, I’ve outlined 8 steps to the load strategy with an explanation of why things were done. Continue reading →