Home > Blogs > VMware vFabric Blog > Tag Archives: cache

Tag Archives: cache

Scaling Real-Time Comments @ HuffPost Live with RabbitMQ

Last week, one of the engineering leaders from Huffingtonpost.com presented at the Erlang Factory, a 2-day conference held in San Francisco with 50+ speakers across 8 tracks (not the cassette kind). Among the presenters was, Adam Denenberg, VP of Engineering at the Huffington Post and HuffPost Live. Besides leading the engineering teams for Huffingtonpost.com and HuffPost Live, Adam is also responsible for community products and advertising platforms.

By using RabbitMQ, SockJS, Cowboy, Erlang, and more, his team built a highly scalable, customizable solution for real-time comments based on websockets.  His talk, Realtime Web @ HuffingtonPost, was for developers of real-time, Erlang-based solutions—he covered architecture, lessons learned, pitfalls, and future improvements. The presentation objectives included:

  • Why they went with Erlang and SockJS for the commenting platform
  • How they integrated RabbitMQ routing to power the subscription architecture
  • How they managed subscriptions for real time channels
  • Their plans to extend the framework into an open source solution
  • Where they want other mechanisms for publishing comments outside of RabbitMQ Continue reading

Part 2: The Value, Architecture, & Code for Building Geography-Based Apps

In our last post, we 1) covered how geographic data can release value in mobile and machine-based applications, 2) explained how technology is used to overcome barriers to these types of big data scenarios, and 3) detailed the architecture for a data fabric or grid (like vFabric GemFire) that works with geographic data and specialized or alternative indexes. There were also code examples to explain the object model, the spatial index, and data changes.

Now, we will continue the examples, show you how to make the index highly available, and use a function to access the data via the index.

The Scenario for a Highly Available Index

In some cases, a piece of data may be added to a node, or become primary on a node without a clean method call. This happens in the cases of both failover and rebalancing. In the case of failover, a bucket that is on a node (that was also a redundant copy) may suddenly become the primary copy if the node that held the primary failed.

In the case of rebalancing, an entire bucket can be moved to a new node that was added to the system without the benefit of capturing the “put” call on each piece of data. Continue reading

Scaling and Modernizing .NET and Java: SQLFire Performance Test Blows Away Traditional RDBMS

We all know the devil is in the details when it comes to technology.

Yet, our recent vFabric SQLFire performance test (a benchmark from vFabric SQLFire Best Practices Guide) is certainly worth review if you need to scale a Java app, .NET app, or other legacy data source.

If you don’t know what vFabric SQLFire is, it is basically what happens when Apache Derby gets married to vFabric GemFire:

  • Apache Derby is used for its RDBMS components, JDBC driver, query engine, and network server.
  • The partitioning technology of GemFire is used to implement horizontal partitioning features of vFabric SQLFire.
  • vFabric SQLFire specifically enhances the Apache Derby components, such as the query engine, the SQL interface, data persistence, and data eviction, as well as adding additional components like SQL commands, stored procedures, system tables, functions, persistence disk stores, listeners, and locators, to operate a highly distributed and fault tolerant data management cluster.

Continue reading

3 Signs Your Relational Database Must Go

Application and operations teams sometimes reach a point where they must upgrade the database. Whether it’s due to data growth, lack of throughput, too much downtime, the need to share data globally, adding ETLs, or otherwise, it’s never a small project. Since these projects are expensive, any recommendation requires a solid justification.  This article a) characterizes 3 signs where traditional databases hit a wall, b) explains how vFabric SQLFire provides an advantage over traditional databases in each case, and c) should help you make a case for moving towards an in-memory, distributed data grid based on SQL.

For those of us tasked with upgrading (or architecting) the data layer, we all go through similar steps. We build a project plan, make projections and sizing estimates, perform architecture and code reviews, create configuration checklists, provide hardware budgets and plans, talk to vendors about options, and more.  Then, we work to plan the deployment with the least downtime, procure hardware and software, test different data load times, evaluate project risks, develop back-up plans, prepare communications to users about downtime, etc. You know the drill. These projects can take months and consume a fair amount of internal resources or consulting dollars. If you are starting or working on one of these types of projects with a traditional database architecture in mind, are you considering these 3 signs as you consider your options? Continue reading

Why Hyperic is Going to Support PostgreSQL Only As a Backend Database

The next release of Hyperic is coming up soon and the biggest change is to the backend. In the next release, we will only support one database, namely PostgreSQL. Those of you who have been with Hyperic for a while as long as I have may be surprised considering our history with PostgreSQL, but, as you read though this blog, it will start to make sense.

History of PostgreSQL and Hyperic

For the last few years Hyperic has supported only two databases for production use at scale—Oracle and MySQL. This in itself was a big change since at one point, PostgreSQL was our bread and butter.  Hyperic was originally designed on PostgreSQL 7.x. As an open source project, PostgreSQL has a very easy license for distribution. As a startup company we had to get our product out into the marketplace quickly and affordably, so therefore PostgreSQL made sense.

Continue reading

3 Key Stages to Evolve from Legacy Databases to a Modern Cloud Data Grid

How do you plan a roadmap for moving from a legacy data architecture to a cloud-enabled data grid? In this article, we will offer a pragmatic, three-stage approach. At SpringOne-2012, the “Effective design patterns with NewSQL” session (see presentation embedded below) generated a lot of interest. (Thank you to everyone who joined us!) Jags Ramnarayan and I discussed problems with legacy RDBMS systems, NewSQL driving principles, SQLFire architecture, application design patterns as well as data consistency and reliability.

We went deep into vFabric SQLFire which is a pragmatic solution that addresses these data challenges:

  • How do I architect my data tier for very high concurrent workloads?
  • How do I achieve predictability both for data access response time and availability?
  • How do I distribute data efficiently and real time to multiple data centers (and to external clouds)? 
  • How do I process these large quantities of data in an efficient manner to allow for better real-time decision-making? 

Continue reading

NanoTrader: Now The SpringTrader Reference Architecture

What is SpringTrader?

Back in August, we provided a sneak peak at NanoTrader, VMware’s vFabric Reference Architecture.  It was referenced in several posts and then featured in a session at SpringOne recently.  The application has now been named SpringTrader, and we wanted to a) share more information about the SpringTrader app including some updated architecture graphics, b) provide a new tool (a version you can log into online), and c) share the location of the Spring Trader bits for download.

If you haven’t heard, the SpringTrader reference architecture is used to help Java-based application architects, developers, infrastructure, and operations teams advance their application roadmaps and provide reusable patterns.  Some might also consider how vFabric Application Director can be used with the SpringTrader app to enable continuous deployment or automatically provision and scale the app in a completely virtual data center (i.e. a software defined data center). As well, vFabric Application Performance Manager can be used to monitor the entire stack and trigger automated scaling events like adding a new JVM and tc Server to the SpringTrader app’s production environment.
Continue reading

Using Visual Statistics Display to Analyze GemFire Runtime Configuration, Resources, and Performance

vFabric GemFire is a sophisticated product in a complex problem space: data management in distributed systems. In order to help our users get the most out of GemFire, we are starting a “cookbook” series, which will provide tried and tested recipes that we hope every GemFire user will find useful.

Our first topic is the Visual Statistics Display (VSD). VSD is a visual tool for analyzing GemFire statistics. It reads GemFire statistics from special statistics archive files created by GemFire, and renders their graphs for analysis. It is not a real-time online monitoring tool, such as vFabric Hyperic, so it does not have the real-time monitoring and alerting capabilities that they have. On the other hand, it is the most powerful tool for examining the state of a vFabric GemFire system, as it provides access to all the statistics collected by GemFire. No real-time monitoring tool can do that, as the amount of statistics that GemFire collects is prohibitive for real-time collection in a distributed system.

Continue reading

JavaOne Architect Viewpoint: The Inexorable March toward Simplicity

Here is a quick field report from JavaOne!

We’ve talked to dozens of people, and the theme we keep hearing is simplicity, simplicity, simplicity.

Many amazingly bright application architects have stopped by to understand and learn more about the vFabric application architecture, and these folks hail from a number of industries – giant telecom manufacturers, government ministries of defense, and multi-industry service companies to name a few.

 

These conversations with architects have tended to fall into one of the falling categories: Continue reading

Introducing VMware vFabric Suite 5.1: Automated Deployment, New Components, and Open Source Support

Today, VMware announced the release of VMware vFabric Suite 5.1, an integrated family of components that streamline how companies build, run and manage Java Spring applications whether on-premise or in the cloud. At VMware, we understand that smart, agile and cost-effective cloud infrastructure services are important. But ultimately, it’s about the applications that run atop that cloud infrastructure. 

Spring-vfabric-vsphere-stack_320x

This latest release of vFabric Suite puts the largest set of modern cloud services within easy reach for companies that need to build cloud-ready Java applications that are deployed easily, scale quickly, and use popular open source application infrastructure technologies.

Continue reading