Home > Blogs > VMware vFabric Blog > Tag Archives: partitioning

Tag Archives: partitioning

3 Game Changing Capabilities in SQLFire

According to one of our partners, vFabric SQLFire is a product he wishes more customers would use.

“SQLFire is a game-changer.  I think many companies underestimate the value of scaling the data later horizontally.  Every project I propose has a business case, and I see a tremendous amount of value being unlocked with this product—not just for the CIO or CTO’s agenda, but for the CFO and CEO.  Then, you add the fact that the whole application stack is virtualized and has solid integrations. It’s a simple story, the product allows you to add a lot of value in a really cost effective way.”

What makes SQLFire such a game-changer?

In this article, we’ll talk more about three game-changing capabilities: server groups, partitioning, and redundancy.

If you haven’t been following our stories on SQLFire, see the end of this article for a  list of posts and key capabilities that help explain how transformative SQLFire can be to your data management strategies. Continue reading

Part 2: The Value, Architecture, & Code for Building Geography-Based Apps

In our last post, we 1) covered how geographic data can release value in mobile and machine-based applications, 2) explained how technology is used to overcome barriers to these types of big data scenarios, and 3) detailed the architecture for a data fabric or grid (like vFabric GemFire) that works with geographic data and specialized or alternative indexes. There were also code examples to explain the object model, the spatial index, and data changes.

Now, we will continue the examples, show you how to make the index highly available, and use a function to access the data via the index.

The Scenario for a Highly Available Index

In some cases, a piece of data may be added to a node, or become primary on a node without a clean method call. This happens in the cases of both failover and rebalancing. In the case of failover, a bucket that is on a node (that was also a redundant copy) may suddenly become the primary copy if the node that held the primary failed.

In the case of rebalancing, an entire bucket can be moved to a new node that was added to the system without the benefit of capturing the “put” call on each piece of data. Continue reading

Scaling and Modernizing .NET and Java: SQLFire Performance Test Blows Away Traditional RDBMS

We all know the devil is in the details when it comes to technology.

Yet, our recent vFabric SQLFire performance test (a benchmark from vFabric SQLFire Best Practices Guide) is certainly worth review if you need to scale a Java app, .NET app, or other legacy data source.

If you don’t know what vFabric SQLFire is, it is basically what happens when Apache Derby gets married to vFabric GemFire:

  • Apache Derby is used for its RDBMS components, JDBC driver, query engine, and network server.
  • The partitioning technology of GemFire is used to implement horizontal partitioning features of vFabric SQLFire.
  • vFabric SQLFire specifically enhances the Apache Derby components, such as the query engine, the SQL interface, data persistence, and data eviction, as well as adding additional components like SQL commands, stored procedures, system tables, functions, persistence disk stores, listeners, and locators, to operate a highly distributed and fault tolerant data management cluster.

Continue reading

Map-Reduce Style: Data Aware Distributed Querying in vFabric GemFire

vFabric GemFire ArchitectureBig, fast data is powering some of the most interesting computing opportunities in today’s market. But in order to get there, we need to change our approach to the data tier. Enterprises are trying to move from costly mainframe architectures to virtualized datacenters and utilize commodity hardware more efficiently. With the data tier, this means an architecture that scales horizontally by adding more commodity-based computing and storage at runtime.

To scale the data tier horizontally, companies use systems like vFabric GemFire, a distributed data system that is designed to specifically accommodate large data sets across commodity hardware nodes. In GemFire, data is spread across members of a cluster with members referred to as “nodes,” and the distribution of data across those nodes is called “partitioning.” vFabric GemFire then allows developers to query the data that resides across many nodes while retaining core values of very high performance at scale. How? In short, the answer is “Data Aware Querying” – a query API that allows a query to execute on selective nodes instead of all nodes (i.e. execute in a map-reduce style).

Continue reading

3 Signs Your Relational Database Must Go

Application and operations teams sometimes reach a point where they must upgrade the database. Whether it’s due to data growth, lack of throughput, too much downtime, the need to share data globally, adding ETLs, or otherwise, it’s never a small project. Since these projects are expensive, any recommendation requires a solid justification.  This article a) characterizes 3 signs where traditional databases hit a wall, b) explains how vFabric SQLFire provides an advantage over traditional databases in each case, and c) should help you make a case for moving towards an in-memory, distributed data grid based on SQL.

For those of us tasked with upgrading (or architecting) the data layer, we all go through similar steps. We build a project plan, make projections and sizing estimates, perform architecture and code reviews, create configuration checklists, provide hardware budgets and plans, talk to vendors about options, and more.  Then, we work to plan the deployment with the least downtime, procure hardware and software, test different data load times, evaluate project risks, develop back-up plans, prepare communications to users about downtime, etc. You know the drill. These projects can take months and consume a fair amount of internal resources or consulting dollars. If you are starting or working on one of these types of projects with a traditional database architecture in mind, are you considering these 3 signs as you consider your options? Continue reading