Home > Blogs > VMware vFabric Blog > Tag Archives: asynchronous

Tag Archives: asynchronous

How We Improved APM’s Monitoring Capacity by 500% with Gemfire, RabbitMQ, vPostgres, and a Few Scalability Design Patterns

Earlier this week, we announced the general availability of a major upgrade to vFabric Application Performance Manager (APM). This release started one year ago, after we released the first version of the product to market. When we started work on this release, we knew we would need to invest heavily in scalability. APM is designed to help simplify monitoring and management for highly dynamic, large web applications living in the cloud. To succeed, we needed to make sure our product could scale gracefully with our customers. So, we set out with a challenging goal to increase the capacity of APM by a factor of 5.

Transforming a complex product such as APM into a more scalable architecture is not an easy task, let alone doing so in a single release. For this reason we’ve started by modifying the architecture in steps, starting with local improvements inside our virtual appliance, (available in the APM 5.0 release) and moving towards a horizontal scale solution in future releases. Continue reading

3 Signs Your Relational Database Must Go

Application and operations teams sometimes reach a point where they must upgrade the database. Whether it’s due to data growth, lack of throughput, too much downtime, the need to share data globally, adding ETLs, or otherwise, it’s never a small project. Since these projects are expensive, any recommendation requires a solid justification.  This article a) characterizes 3 signs where traditional databases hit a wall, b) explains how vFabric SQLFire provides an advantage over traditional databases in each case, and c) should help you make a case for moving towards an in-memory, distributed data grid based on SQL.

For those of us tasked with upgrading (or architecting) the data layer, we all go through similar steps. We build a project plan, make projections and sizing estimates, perform architecture and code reviews, create configuration checklists, provide hardware budgets and plans, talk to vendors about options, and more.  Then, we work to plan the deployment with the least downtime, procure hardware and software, test different data load times, evaluate project risks, develop back-up plans, prepare communications to users about downtime, etc. You know the drill. These projects can take months and consume a fair amount of internal resources or consulting dollars. If you are starting or working on one of these types of projects with a traditional database architecture in mind, are you considering these 3 signs as you consider your options? Continue reading

3 Key Stages to Evolve from Legacy Databases to a Modern Cloud Data Grid

How do you plan a roadmap for moving from a legacy data architecture to a cloud-enabled data grid? In this article, we will offer a pragmatic, three-stage approach. At SpringOne-2012, the “Effective design patterns with NewSQL” session (see presentation embedded below) generated a lot of interest. (Thank you to everyone who joined us!) Jags Ramnarayan and I discussed problems with legacy RDBMS systems, NewSQL driving principles, SQLFire architecture, application design patterns as well as data consistency and reliability.

We went deep into vFabric SQLFire which is a pragmatic solution that addresses these data challenges:

  • How do I architect my data tier for very high concurrent workloads?
  • How do I achieve predictability both for data access response time and availability?
  • How do I distribute data efficiently and real time to multiple data centers (and to external clouds)? 
  • How do I process these large quantities of data in an efficient manner to allow for better real-time decision-making? 

Continue reading

NanoTrader: Now The SpringTrader Reference Architecture

What is SpringTrader?

Back in August, we provided a sneak peak at NanoTrader, VMware’s vFabric Reference Architecture.  It was referenced in several posts and then featured in a session at SpringOne recently.  The application has now been named SpringTrader, and we wanted to a) share more information about the SpringTrader app including some updated architecture graphics, b) provide a new tool (a version you can log into online), and c) share the location of the Spring Trader bits for download.

If you haven’t heard, the SpringTrader reference architecture is used to help Java-based application architects, developers, infrastructure, and operations teams advance their application roadmaps and provide reusable patterns.  Some might also consider how vFabric Application Director can be used with the SpringTrader app to enable continuous deployment or automatically provision and scale the app in a completely virtual data center (i.e. a software defined data center). As well, vFabric Application Performance Manager can be used to monitor the entire stack and trigger automated scaling events like adding a new JVM and tc Server to the SpringTrader app’s production environment.
Continue reading

vFabric VMworld Sessions available On Demand

For anyone that attended VMworld 2012, you can access content for all VMware sessions here.

If you did not attend, you can purchase a subscription to access the content online. Content from all previous years is free for all community members along with the 2012 General Session Keynotes.

For your convenience, we’ve provided a list and links to all the vFabric-related sessions below:
Continue reading

Making Sense of Spaghetti Transactions

Today’s modern applications are built from an ever growing number of moving parts (e.g. queues, caches, services connecting different application tiers, and outgoing calls to external services). These parts are also dynamic—layers grow or shrink to scale, are load balanced, and since they are virtual or in the cloud, they can also physically move quickly. Transactions can literally pass through hundreds of different paths in your average application. These are spaghetti transactions.

Every piece of spaghetti has a start and an end, but how each one passes is difficult to see when it’s in a pile. Understanding spaghetti transactions, and how they flow between all the parts of the application’s topology has become more time consuming than ever.

Efficient management of these applications requires a new approach – moving from individually managing the health of each server, operating system, and the workload it runs, to managing applications and multi-tier transactions. Continue reading

Case Study: CircuitLab Cuts Rendering time from 44 Hours to 57 Minutes Using RabbitMQ

Rendering-16_000-circuits-in-the-cloudCloud computing and open source models have helped the team over at CircuitLab.com perform a re-rendering job on 16,544 production circuit graphics in 57 minutes at a cost of a few CPU dollars and “just a bit of modification” to the existing codebase.  Does this sound unrealistic? Impossible?  Well, they did it. In an article they wrote last week, CircuitLab did a great job of articulating the size and scope of the job they set out to achieve and how RabbitMQ helped.

Solution Requirements

CircuitLab.com allows people to design and share the designs of electronic circuits.  When a user saves a circuit design, the system stores information and generates a render request message to generate various thumbnails, images, and more.  Each render request takes about 10+ seconds of 1 core CPU time as it travels through various components (see article for more).  To process all 16K+ circuit images, the total rendering time would add up to 160,000 seconds or 44 hours.  This was too long.

Additionally, the two founders (bright MIT grads) knew they could solve the problem faster using a parallel processing solution “in the cloud.”  They began working with M5 Cloud Hosting and provisioned 8 virtual machines on separate hypervisors with 8 cores each to get to 64 total high-speed CPU cores.  But, how would they distribute and manage the jobs across cores while production ran?

Continue reading