Home > Blogs > VMware vFabric Blog

New Serengeti Release Extends Cloud Computing Support for Hadoop Community

Today VMware is releasing a significant new release of their big data virtualization open source project Serengeti called M4 or version 0.8.0. Designed to help make it easier for Hadoop users to deploy, run and manage mixed workload clusters on a virtualized platform, this release broadens support across the various distributions of the Hadoop community, including new support for Cloudera CDH4, MapR, and HBase. Additionally as part of this release, Serengeti M4, includes updated performance configuration improvements and a hardware reference architecture guide.

This release comes at a perfect time for an exploding data market. This year, worldwide we will create 4 zettabytes of new data, and more than 80% of that will be unstructured data that does not work in a traditional database management system. At the same time, businesses are learning to harness that data and use it to better their business.

A popular strategy to succeed in the data market is Hadoop, an open source data framework that that allows for the massive distributed processing of large data sets across clusters of nodes using simple programming models.  Additionally, Hadoop offers a scalable file system (HDFS) that allows users to store huge amounts of data leveraging inexpensive disks on commodity servers.  The powerful framework has spawned many new startups in Silicon Valley and has Enterprise IT departments clamoring to harness the power of this technology. Huge web applications like Facebook, LinkedIn, Yahoo! and eBay all rely on Hadoop to process and store data for hundreds of millions of users. Continue reading

What 10 Partners Are Saying About vFabric Today

When you are a software company like VMware, you are tied to the success of your partners in the field standing shoulder to shoulder with your customers, guiding them to a successful deployment. Their success and growth directly reflects your customer’s success as your own success in the marketplace.

Partners are focused on solutions that customers care about because it is the only way to stay in business and grow.  Today, 10 representatives from technology consulting organizations share their perspectives on how they are using the VMware vFabric Suite to achieve success with customers. Each includes a video or you can visit the full vFabric playlist on VMwareTV.

1. Modernizing Legacy Apps to Cloud and SaaS with vFabric

“In vFabric we see a very attractive modern cloud application platform.  So while spring and vFabric represent very attractive alternatives for new application development, what we see with our clients are is the desire to modernize and transform existing applications to take advantage of those same benefits of vFabric.  The run time benefits, the design time benefits.”—Chase Crawson, Director of Application Services, CSC Continue reading

Scaling Real-Time Comments @ HuffPost Live with RabbitMQ

Last week, one of the engineering leaders from Huffingtonpost.com presented at the Erlang Factory, a 2-day conference held in San Francisco with 50+ speakers across 8 tracks (not the cassette kind). Among the presenters was, Adam Denenberg, VP of Engineering at the Huffington Post and HuffPost Live. Besides leading the engineering teams for Huffingtonpost.com and HuffPost Live, Adam is also responsible for community products and advertising platforms.

By using RabbitMQ, SockJS, Cowboy, Erlang, and more, his team built a highly scalable, customizable solution for real-time comments based on websockets.  His talk, Realtime Web @ HuffingtonPost, was for developers of real-time, Erlang-based solutions—he covered architecture, lessons learned, pitfalls, and future improvements. The presentation objectives included:

  • Why they went with Erlang and SockJS for the commenting platform
  • How they integrated RabbitMQ routing to power the subscription architecture
  • How they managed subscriptions for real time channels
  • Their plans to extend the framework into an open source solution
  • Where they want other mechanisms for publishing comments outside of RabbitMQ Continue reading

How Indeed.com Handles 35 Million Job Postings Per Day Using RabbitMQ

Indeed.com is hosting a tech talk that will be a case study of how they scaled their aggregation engine to handle 35 million job postings per day using RabbitMQ. The talk will be hosted at Indeed’s engineering site in Austin, TX at 7 pm on March 27th.

For those of you not familiar with how large Indeed is, it is interesting to note that the job search company Indeed.com is one of the largest web sites in the world. According to Alexa.com, Indeed is currently the #224th biggest website in the world, and in cities like Atlanta and Chicago, it’s the 55th most popular website overall. According to research by SilkRoad, 2 out of every 5 hires came via Indeed (based on data from 150,000 hires).

As expected, the engineering team behind this large-scale application needs to support some very large scale numbers. In a recent post on their company blog, the Indeed team shared just how big those numbers are:

  • More than 100 million monthly unique visitors
  • More than 3 billion searches per month
  • More than 1000 searches per second
  • 50 country-specific sites in 26 languages

The scale of their application, both in terms of processing throughput and geographic diversity, means that the team relies on a messaging layer powered by RabbitMQ. Continue reading

Tips and Tricks for Internal Use of Your vFabric Postgres VM

Ever since VMware released our new version of the popular PostgreSQL, we have been working hard to publish a series of informative articles on what’s new in the vFabric Postgres (vPostgres) 9.2 release. There are a number of cool things like major performance and scale improvements, elastic memory for vPostgres, contributions to open source, master-slave clusters, and new GUI capabilities. In this post, we are going to dig into some tips and tricks for internal use of a vFabric Postgres virtual machine, talk about scripts, and explain the management of the network interface.

First, to see what we are talking about, it would be helpful to login to your vFabric Postgres VM. vPostgres supports SSL by default (see last weeks post on securing your vPostgres deployment for more security tips). Therefore, it is pretty easy to connect to a server and set things up inside for those who really want to personalize things at a very low level (like pg_bha.conf for connection restrictions). After the first initialization, you can connect either as user postgres or root with the same password you used at first boot.

As with every other system, it is never really recommended to connect with the user root for security reasons. By connecting with user postgres you will find the following things once connected. Continue reading

Securing your vFabric PostgreSQL VM

Especially in today’s world, security is top of mind for app developers, DBAs, and CIOs alike. One of the benefits that VMware strives to include in every product is a system of reasonable defaults for security. This generally means that users should expect a reasonably secure middleware application when they deploy a VMware app by default.

vFabric Postgres (vPostgres) is no different. There are not that many security settings in vFabric Postgres. However, there are a few things you can look at as options to further harden your deployment, and of course, the virtual machine that you are deploying them on, particularly if it is exposed to an external environment.

SSL Connection Restrictions

vFabric Postgres has as default users postgres and root, and both can connect to the virtual machine with SSL. If you want to restrict access to the virtual machine for certain users or a group of users, here is some advice to follow:

1. In order to restrict SSL connection only to the members of the group vFabric (the user postgres is a member of this group by default), add this line at in /etc/ssh/sshd_config.

AllowGroups vfabric

Continue reading

Exploring the New Database Server GUI Features in vFabric Postgres 9.2

The vFabric Postgres 9.2 release seriously upped the user interface (UI) experience. In our post last week, we talked about the built in, VM-based GUI to help manage the system, network, and updates. This week, we’d like to take you through the changes your DBAs and developers will see when using the updated database server—listing the databases on the server, seeing database global data, and drilling into processes and locks. All of this comes out of the box with the vFabric Postgres appliance.

Connecting to the Database Management Interface

Once your vFabric Postgres server is up and running, you can connect to its database management interface using this URL in a web browser. The connection is made with https, port 8443 at the IP address or domain for the server: Continue reading

10 Lessons from Spring Applied to Java Virtualization with vFabric

The Spring Framework became the de-facto standard for developing enterprise Java applications, and its radical simplicity was fundamental to its success. Why the “radical” simplicity? Because at the time, it was hard to imagine how creating such applications could be made simple.

By tackling issues such as portability, understanding the importance of cross-cutting concerns, and making it trivial to develop automated tests, Spring allowed developers to focus on what matters: what makes their application unique.

As I was pulling together my presentation for SpringOne2GX 2012, I reflected on the parallels between Spring’s success and the direction we were going with EM4J. Why did Spring succeed? Why did simplification win? Where are we replicating these patterns within VMware, vFabric, and Java?

In short, complexity is expensive, and simplification has many economic benefits. By giving people better, simpler, and easier to use tools to help build, run, and manage applications, we create economic advantages.

In a nutshell, there are some core reasons why Spring succeeded, “Spring values” if you will: Reducing complexity, increasing productivity, provisioning flexibility, tooling and monitoring, extensibility, automation, flexible integration and ease of testing. Continue reading

VMware vFabric SQLFire 1.1.0 Released

A new version of SQLFire is now available for immediate download or trial.

Highlight of changes in this release:

  • vFabric SQLFire in 15 Minutes gets new users up and running on SQLFire with a quick tour of features and functionality.
  • The ALTER TABLE command now supports modifying table columns even after you have loaded data into the table.
  • SQLFire query plan output and documentation have been updated.
  • The CREATE INDEX command now supports creating case-insensitive indexes for CHAR and VARCHAR columns.
  • Improves performance and usability for disk stores and persistence. Continue reading

Putting the ‘Single’ Back in Single Sign-On (SSO)

Modern companies and IT organizations have many applications, both internal and customer facing.  With these many applications your users are faced with the challenge of not only managing multiple sets of credentials, but are also forced to login to each and every individual application separately.  This creates a bad experience for your users.

To improve user experience, IT created a concept called Single Sign On (SSO). The idea was users could sign on once, and the SSO software would automatically authenticate them for all their applications. This not only helped the user experience, but also helped IT by cutting down on the number of ‘forgot password’ tickets opened and when users left the organization, it made de-authenticating them really easy. The idea is great, but in practice it frequently stopped short at authentication.  Continue reading