Home > Blogs > VMware vFabric Blog > Tag Archives: NewSQL

Tag Archives: NewSQL

Why Every Database Must Be Broken Soon

Have you ever heard of a zettabyte? If you work in IT, you’ll be hearing more and more about zettabytes, exabytes, and petabytes while the data terms we think are big, such as terabytes and gigabytes wane away from our vocabulary. Right now, we are growing our data stores by 50% year-over-year, and its only accelerating.

In 2010, we crossed the barrier of the zettabyte (ZB) across all online data. This year, we will produce 4 ZB of data worldwide. In 2016, global IP traffic will reach 1.3 ZB.

While data volumes are skyrocketing, the type of data is also becoming more difficult for traditional databases to handle. Over 80% of it will be unstructured file based data that does not work well with block-based data storage typical of your typical relational databases (RDBMS).  So, even if hardware innovations could keep up to support greater volume, the kinds of data we are now storing break traditional RDBMS at today’s speeds.

The bottom line is the volume and types of data being stored is unrealistic for a single, monolithic, structured RDBMS data store. They need to be broken apart and re-architected to survive the Information Explosion we are experiencing today.

Continue reading

The Best VMware vFabric Stories of 2012 & What’s In Store for 2013

As this year comes to a close, it’s time to be reflective of what happened in the past and start planning for a new year. The vFabric team has had some major achievements this year, introducing several new products to the market including the innovative vFabric Application Director, the widely anticipated Project Serengeti to enable rapid cloud deployments for Hadoop, and a new tool to the vFabric Suite users called vFabric Administration Server (VAS).  We announced a new VMware Cloud Applications Marketplace to help further accelerate application development with a professionally moderated library of enterprise grade, ready-to-use application components that can be run on any cloud.

Next year is going to be even bigger with the Pivotal Initiative where several of the products covered on this blog will be following the new venture. This is still in the planning stages, so we will be expecting to share with you the plans for our products alongside the formal communications from each of the companies involved. (Sorry — no extra information is available right now)

One thing that we are going to be doing in early 2013 is to move the conversation of how you manage applications to be with the conversations of how you manage virtual infrastructure. To that end, we will be moving all topics of Application Performance Manager, AppInsight, Application Director, Hyperic, and Spring Insight to the VMware Management Blog as of January 1st. To make sure you keep up with the management topics, please be sure to follow us @vmwareappmgmt and @vmwaremgmt.

In the meantime, we’d like to reshare with you the top 20 stories we had for 2012, and invite you to comment here on what stories you would like to see us cover on either blog for 2013.

Continue reading

3 Signs Your Relational Database Must Go

Application and operations teams sometimes reach a point where they must upgrade the database. Whether it’s due to data growth, lack of throughput, too much downtime, the need to share data globally, adding ETLs, or otherwise, it’s never a small project. Since these projects are expensive, any recommendation requires a solid justification.  This article a) characterizes 3 signs where traditional databases hit a wall, b) explains how vFabric SQLFire provides an advantage over traditional databases in each case, and c) should help you make a case for moving towards an in-memory, distributed data grid based on SQL.

For those of us tasked with upgrading (or architecting) the data layer, we all go through similar steps. We build a project plan, make projections and sizing estimates, perform architecture and code reviews, create configuration checklists, provide hardware budgets and plans, talk to vendors about options, and more.  Then, we work to plan the deployment with the least downtime, procure hardware and software, test different data load times, evaluate project risks, develop back-up plans, prepare communications to users about downtime, etc. You know the drill. These projects can take months and consume a fair amount of internal resources or consulting dollars. If you are starting or working on one of these types of projects with a traditional database architecture in mind, are you considering these 3 signs as you consider your options? Continue reading

Webcast: Big, Fast, Flexible Data with Cloud Delivery

As we’ve previously covered, data growth is quite unbelievable, and this means traditional database models are being stretched. On Tuesday, November 13, 2012 at 9:00 AM PST, VMware’s Joe Russell will be presenting on several topics related to Big, Fast, Flexible Data and how VMWare’s key data management technologies help companies overcome some of the key challenges with traditional RDBMS.

Attend to learn:

  • How Hadoop and new analytics technologies are allowing companies to use Big Data in new ways to gain meaningful business insights
  • What’s new with Project Serengeti, a VMware initiative to help you deploy and manage elastic Hadoop clusters in minutes
  • How Fast Data is bringing data logic in-memory, allowing for dramatic scale, reduced costs, and improved performance
  • How Flexibile Data, includng NoSQL and open source relational data technologies can improve your data model
  • How virtualizing the database layer enables a new Cloud Delivery Model, allowing enterprise IT departments to offer self-service data services elastically on demand, maintain centralized control, and operate within regulatory guidelines

>> Register for the webinar Big, Fast, Flexible Data with Cloud Delivery on Tuesday, November 13, 2012 at 9:00 AM PST.

3 Key Stages to Evolve from Legacy Databases to a Modern Cloud Data Grid

How do you plan a roadmap for moving from a legacy data architecture to a cloud-enabled data grid? In this article, we will offer a pragmatic, three-stage approach. At SpringOne-2012, the “Effective design patterns with NewSQL” session (see presentation embedded below) generated a lot of interest. (Thank you to everyone who joined us!) Jags Ramnarayan and I discussed problems with legacy RDBMS systems, NewSQL driving principles, SQLFire architecture, application design patterns as well as data consistency and reliability.

We went deep into vFabric SQLFire which is a pragmatic solution that addresses these data challenges:

  • How do I architect my data tier for very high concurrent workloads?
  • How do I achieve predictability both for data access response time and availability?
  • How do I distribute data efficiently and real time to multiple data centers (and to external clouds)? 
  • How do I process these large quantities of data in an efficient manner to allow for better real-time decision-making? 

Continue reading

NanoTrader: Now The SpringTrader Reference Architecture

What is SpringTrader?

Back in August, we provided a sneak peak at NanoTrader, VMware’s vFabric Reference Architecture.  It was referenced in several posts and then featured in a session at SpringOne recently.  The application has now been named SpringTrader, and we wanted to a) share more information about the SpringTrader app including some updated architecture graphics, b) provide a new tool (a version you can log into online), and c) share the location of the Spring Trader bits for download.

If you haven’t heard, the SpringTrader reference architecture is used to help Java-based application architects, developers, infrastructure, and operations teams advance their application roadmaps and provide reusable patterns.  Some might also consider how vFabric Application Director can be used with the SpringTrader app to enable continuous deployment or automatically provision and scale the app in a completely virtual data center (i.e. a software defined data center). As well, vFabric Application Performance Manager can be used to monitor the entire stack and trigger automated scaling events like adding a new JVM and tc Server to the SpringTrader app’s production environment.
Continue reading

vFabric VMworld Sessions available On Demand

For anyone that attended VMworld 2012, you can access content for all VMware sessions here.

If you did not attend, you can purchase a subscription to access the content online. Content from all previous years is free for all community members along with the 2012 General Session Keynotes.

For your convenience, we’ve provided a list and links to all the vFabric-related sessions below:
Continue reading

vFabric @ SpringOne Next Week

The vFabric team is headed to SpringOne 2GX 2012 next week – from October 15-18 in Washington, DC. This is set to be a great event to learn the latest on Spring with over 100 sessions covering a wide variety of topics. For those of you looking to learn more about how vFabric is the best place to run Spring applications, here are the highlights you won’t want to miss:

1. Sessions:  There are a number of speakers from SpringSource, CloudFoundry, and the VMware vFabric team on the schedule, including:

Continue reading

Load 8 Million Rows in 88 Seconds – NewSQL Speed

At some point, any data modernization project is going to require a load of legacy data. With an in-memory, distributed data store like SQLFire, customers often ask (like in this case) about load times because they can be sitting on 50-100 GB of data and don’t want to wait days. For those unfamiliar with NewSQL databases, this post should give you a good sense of how we loaded 8 million rows in 88 seconds. The test shows how we should be able to load roughly 40GB of data in about 1 hour.

For Java developers who want ideas about speeding up large volumes of calculations, transforms, or validations, you may want to consider a previous post, where SQLFire is used with Spring Batch.

With SQLFire, we take a multi-threaded load approach from a CSV file. Below, I’ve outlined 8 steps to the load strategy with an explanation of why things were done. Continue reading

5 Labs You Don’t Want to Miss @VMworld

What if you could spend a few days in San Francisco and learn…

  • How to reduce deployment time from weeks to hours.
  • How to develop without worrying about scaling infrastructure (and vice versa).
  • How to drastically reduce database license, hardware, and management costs.
  • How to scale the data layer and increase speed drastically using standard SQL.
  • How the vFabric suite scales at the web, app, and data layer.

Would it be worth the trip? Well, we think so for three reasons: Continue reading