Home > Blogs > VMware vFabric Blog > Monthly Archives: March 2013

Monthly Archives: March 2013

Scaling Real-Time Comments @ HuffPost Live with RabbitMQ

Last week, one of the engineering leaders from Huffingtonpost.com presented at the Erlang Factory, a 2-day conference held in San Francisco with 50+ speakers across 8 tracks (not the cassette kind). Among the presenters was, Adam Denenberg, VP of Engineering at the Huffington Post and HuffPost Live. Besides leading the engineering teams for Huffingtonpost.com and HuffPost Live, Adam is also responsible for community products and advertising platforms.

By using RabbitMQ, SockJS, Cowboy, Erlang, and more, his team built a highly scalable, customizable solution for real-time comments based on websockets.  His talk, Realtime Web @ HuffingtonPost, was for developers of real-time, Erlang-based solutions—he covered architecture, lessons learned, pitfalls, and future improvements. The presentation objectives included:

  • Why they went with Erlang and SockJS for the commenting platform
  • How they integrated RabbitMQ routing to power the subscription architecture
  • How they managed subscriptions for real time channels
  • Their plans to extend the framework into an open source solution
  • Where they want other mechanisms for publishing comments outside of RabbitMQ Continue reading

How Indeed.com Handles 35 Million Job Postings Per Day Using RabbitMQ

Indeed.com is hosting a tech talk that will be a case study of how they scaled their aggregation engine to handle 35 million job postings per day using RabbitMQ. The talk will be hosted at Indeed’s engineering site in Austin, TX at 7 pm on March 27th.

For those of you not familiar with how large Indeed is, it is interesting to note that the job search company Indeed.com is one of the largest web sites in the world. According to Alexa.com, Indeed is currently the #224th biggest website in the world, and in cities like Atlanta and Chicago, it’s the 55th most popular website overall. According to research by SilkRoad, 2 out of every 5 hires came via Indeed (based on data from 150,000 hires).

As expected, the engineering team behind this large-scale application needs to support some very large scale numbers. In a recent post on their company blog, the Indeed team shared just how big those numbers are:

  • More than 100 million monthly unique visitors
  • More than 3 billion searches per month
  • More than 1000 searches per second
  • 50 country-specific sites in 26 languages

The scale of their application, both in terms of processing throughput and geographic diversity, means that the team relies on a messaging layer powered by RabbitMQ. Continue reading

Tips and Tricks for Internal Use of Your vFabric Postgres VM

Ever since VMware released our new version of the popular PostgreSQL, we have been working hard to publish a series of informative articles on what’s new in the vFabric Postgres (vPostgres) 9.2 release. There are a number of cool things like major performance and scale improvements, elastic memory for vPostgres, contributions to open source, master-slave clusters, and new GUI capabilities. In this post, we are going to dig into some tips and tricks for internal use of a vFabric Postgres virtual machine, talk about scripts, and explain the management of the network interface.

First, to see what we are talking about, it would be helpful to login to your vFabric Postgres VM. vPostgres supports SSL by default (see last weeks post on securing your vPostgres deployment for more security tips). Therefore, it is pretty easy to connect to a server and set things up inside for those who really want to personalize things at a very low level (like pg_bha.conf for connection restrictions). After the first initialization, you can connect either as user postgres or root with the same password you used at first boot.

As with every other system, it is never really recommended to connect with the user root for security reasons. By connecting with user postgres you will find the following things once connected. Continue reading

Securing your vFabric PostgreSQL VM

Especially in today’s world, security is top of mind for app developers, DBAs, and CIOs alike. One of the benefits that VMware strives to include in every product is a system of reasonable defaults for security. This generally means that users should expect a reasonably secure middleware application when they deploy a VMware app by default.

vFabric Postgres (vPostgres) is no different. There are not that many security settings in vFabric Postgres. However, there are a few things you can look at as options to further harden your deployment, and of course, the virtual machine that you are deploying them on, particularly if it is exposed to an external environment.

SSL Connection Restrictions

vFabric Postgres has as default users postgres and root, and both can connect to the virtual machine with SSL. If you want to restrict access to the virtual machine for certain users or a group of users, here is some advice to follow:

1. In order to restrict SSL connection only to the members of the group vFabric (the user postgres is a member of this group by default), add this line at in /etc/ssh/sshd_config.

AllowGroups vfabric

Continue reading

Exploring the New Database Server GUI Features in vFabric Postgres 9.2

The vFabric Postgres 9.2 release seriously upped the user interface (UI) experience. In our post last week, we talked about the built in, VM-based GUI to help manage the system, network, and updates. This week, we’d like to take you through the changes your DBAs and developers will see when using the updated database server—listing the databases on the server, seeing database global data, and drilling into processes and locks. All of this comes out of the box with the vFabric Postgres appliance.

Connecting to the Database Management Interface

Once your vFabric Postgres server is up and running, you can connect to its database management interface using this URL in a web browser. The connection is made with https, port 8443 at the IP address or domain for the server: Continue reading

10 Lessons from Spring Applied to Java Virtualization with vFabric

The Spring Framework became the de-facto standard for developing enterprise Java applications, and its radical simplicity was fundamental to its success. Why the “radical” simplicity? Because at the time, it was hard to imagine how creating such applications could be made simple.

By tackling issues such as portability, understanding the importance of cross-cutting concerns, and making it trivial to develop automated tests, Spring allowed developers to focus on what matters: what makes their application unique.

As I was pulling together my presentation for SpringOne2GX 2012, I reflected on the parallels between Spring’s success and the direction we were going with EM4J. Why did Spring succeed? Why did simplification win? Where are we replicating these patterns within VMware, vFabric, and Java?

In short, complexity is expensive, and simplification has many economic benefits. By giving people better, simpler, and easier to use tools to help build, run, and manage applications, we create economic advantages.

In a nutshell, there are some core reasons why Spring succeeded, “Spring values” if you will: Reducing complexity, increasing productivity, provisioning flexibility, tooling and monitoring, extensibility, automation, flexible integration and ease of testing. Continue reading

VMware vFabric SQLFire 1.1.0 Released

A new version of SQLFire is now available for immediate download or trial.

Highlight of changes in this release:

  • vFabric SQLFire in 15 Minutes gets new users up and running on SQLFire with a quick tour of features and functionality.
  • The ALTER TABLE command now supports modifying table columns even after you have loaded data into the table.
  • SQLFire query plan output and documentation have been updated.
  • The CREATE INDEX command now supports creating case-insensitive indexes for CHAR and VARCHAR columns.
  • Improves performance and usability for disk stores and persistence. Continue reading

Putting the ‘Single’ Back in Single Sign-On (SSO)

Modern companies and IT organizations have many applications, both internal and customer facing.  With these many applications your users are faced with the challenge of not only managing multiple sets of credentials, but are also forced to login to each and every individual application separately.  This creates a bad experience for your users.

To improve user experience, IT created a concept called Single Sign On (SSO). The idea was users could sign on once, and the SSO software would automatically authenticate them for all their applications. This not only helped the user experience, but also helped IT by cutting down on the number of ‘forgot password’ tickets opened and when users left the organization, it made de-authenticating them really easy. The idea is great, but in practice it frequently stopped short at authentication.  Continue reading

How to Use the New GUI Features in vFabric Postgres 9.2

People have said code is poetry and information is beautiful, but let’s talk about graphical user interfaces (GUI) and ease of use.

The PostgreSQL community has a multitude of choices when it comes to GUIs versus CLI. For vFabric Postgres, we’ve added some cool, new GUI tools.

In part one of this two-part series on vFabric Postgres user interfaces, we look at the built-in GUI capabilities within the VM of the latest release—vFabric Postgres 9.2. Based on VMware Studio, vFabric Postgres 9.2 uses an instinctive user interface to fully control the behavior of the database virtual machine and allows you to perform basic tasks that are normally associated with an OS. In part 2, we will cover the key features of the database server’s new GUI and explain how to get overview-level information and see both processes and locks. Continue reading

HOWTO: Deploying RabbitMQ Simulator on Cloud Foundry

Last week, we published the new RabbitMQ Simulator. I’m pleased to see how well it has been received by customers, with several very large scale RabbitMQ customers already incorporating it into their training and testing regimens. VMware employees in the field have been enjoying getting familiar with the tool as well and are using it to advance their own and their customer’s understanding of how RabbitMQ works and how to best design a messaging architecture. One sales engineer, Chris DeLashmutt even wrote a blog this week describing how to deploy the new RabbitMQ Simulator on Cloud Foundry.

For users new to RabbitMQ, certainly testing this out on Cloud Foundry can be a huge accelerator, as you can get your instance up and running for free in just a matter of minutes. Below is a summary of Chris’ instructions to deploy this simulator on Cloud Foundry, but also, if you want to shortcut the entire process, he invites you to access his deployed instance at http://rabbitmq-sim.cloudfoundry.com. Continue reading