Home > Blogs > VMware vFabric Blog > Category Archives: RabbitMQ

Category Archives: RabbitMQ

New RabbitMQ 3.1.0 Release Available

RabbitMQ 3.1.0 is now available for immediate download.

Announced this morning on the new Pivotal blog, where RabbitMQ now resides, this version includes enhancements to garbage collection, consumption, requeuing, memory use, and dead lettering.

For those on Mac OS X, there is a newly packaged, standalone release of RabbitMQ that doesn’t require a separate Erlang install.

Some key, new capabilities include eager synchronisation of mirror queue slaves, automatic cluster partition healing, and improved statistics (including charts) in the management plugin. There are also many enhancements and bug fixes to the server, Java client, Erlang client, and a number of other plugins, including federation, old-federation, shovel, Web-STOMP, STOMP, and MQTT plugins, as well as the consistent hash exchange.

RabbitMQ’s blog post on the topic shares screenshots of several new features like the ones for new charts and filters below:

Read More:

7 Myths on Big Data—Avoiding Bad Hadoop and Cloud Analytics Decisions

Hadoop is an open source legend built by software heroes.

Yet, legends can sometimes be surrounded by myths—these myths can lead IT executives down a path with rose-colored glasses.

Data and data usage is growing at an alarming rate.  Just look at all the numbers from analysts—IDC predicts a 53.4% growth rate for storage this year, AT&T claims 20,000% growth of their wireless data traffic over the past 5 years, and if you take at your own communications channels, its guaranteed that the internet content, emails, app notifications, social messages, and automated reports you get every day has dramatically increased.  This is why companies ranging from McKinsey to Facebook to Walmart are doing something about big data.

Just like we saw in the dot-com boom of the 90s and the web 2.0 boom of the 2000s, the big data trend will also lead companies to make some really bad assumptions and decisions.

Hadoop is certainly one major area of investment for companies to use to solve big data needs. Companies like Facebook that have famously dealt well with large data volumes have publicly touted their successes with Hadoop, so its natural that companies approaching big data first look to the successes of others.  A really smart MIT computer science grad once told me, “when all you have is a hammer, everything looks like a nail.” This functional fixedness is the cognitive bias to avoid with the hype surrounding Hadoop. Hadoop is a multi-dimensional solution that can be deployed and used in different way. Let’s look at some of the most common pre-concieved notions about Hadoop and big data that companies should know before committing to a Hadoop project: Continue reading

How fast is a Rabbit? Basic RabbitMQ Performance Benchmarks

One of the greatest things about RabbitMQ is the community that surrounds it. With open source at its roots, people come together to share their code, their knowledge and their stories of how they’ve deployed it in their projects. At a recent meetup near Nice, France, database engineer Adina Mihailescu shared a presentation on choosing messaging systems. Supported by Murial Salvan’s benchmark comparing ActiveMQ, RabbitMQ, HornetQ, Apollo, QPID, and ZeroMQ, they shared some interesting performance comparisons that we’d like to share with you.

In a single laptop benchmark, Salvan ran four different scenarios in order to obtain some insight on performance of the default setups for these messaging solutions. Each test had 1 process dedicated to enqueuing and another dedicated to dequeuing. The message volume and size ranged from 200 to 20,000 to 200,000 messages and 32 to 1024 to 32768 bytes. Both persistent and transient queues and messages were used. Continue reading

Upcoming Webinar: Paul Maritz on Pivotal and The New Platform for the New Era

The cloud, mobile applications and big, fast data are fundamentally changing how applications are built and modernized today. To speed this transformation at the enterprise level, Pivotal, the new venture by VMware and EMC, will host a live streaming event on April 24th at 10:00 am Pacific/1:00 pm Eastern with a special announcement and an unveiling of its plans to build “A New Platform for a New Era”.

The Pivotal platform will unite data, application, and cloud fabrics, helping enterprises to develop faster, understand more, and succeed at an even greater scale. It is a platform that makes the consumer grade enterprise a reality.

Pivotal brings together a prodigious set of technologies and talent from a number of EMC and VMware entities, which include Greenplum, Cloud Foundry, Spring, GemFire and other products from the VMware vFabric Suite, Cetas, and Pivotal Labs.

>> Register for webinar here!

Paul Maritz, the Pivotal Leadership Team, and special guests will unveil this platform, and make a special announcement during a live streaming event on Wednesday, April 24th at 10:00 am Pacific/1:00 pm Eastern.

Sign up for the event at gopivotal.com and follow @gopivotal on Twitter for updates.

How Instagram Feeds Work: Celery and RabbitMQ

Instagram is one of the poster children for social media site successes. Founded in 2010, the photo sharing site now supports upwards of 90 million active photo-sharing users. As with every social media site, part of the fun is that photos and comments appear instantly so your friends can engage while the moment is hot.  Recently, at PyCon 2013 last month, Instagram engineer Rick Branson shared how Instagram needed to transform how these photos and comments showed up in feeds as they scaled from a few thousand tasks a day to hundreds of millions.

Rick started off his talk demonstrating how traditional database approaches break, calling them the “naïve approach”. In this approach, when working to display a user feed, the application would directly fetch all the photos that the user followed from a single, monolithic data store, sort them by creation time and then only display the latest 10:

SELECT * FROM photos
WHERE author_id IN
(SELECT target_id FROM following
WHERE source_id = %(user_id)d)
ORDER BY creation_time DESC
LIMIT 10;

Instead, Instagram chose to follow a modern distributed data strategy that will allow them to scale nearly linearly. Continue reading

Scaling Real-Time Comments @ HuffPost Live with RabbitMQ

Last week, one of the engineering leaders from Huffingtonpost.com presented at the Erlang Factory, a 2-day conference held in San Francisco with 50+ speakers across 8 tracks (not the cassette kind). Among the presenters was, Adam Denenberg, VP of Engineering at the Huffington Post and HuffPost Live. Besides leading the engineering teams for Huffingtonpost.com and HuffPost Live, Adam is also responsible for community products and advertising platforms.

By using RabbitMQ, SockJS, Cowboy, Erlang, and more, his team built a highly scalable, customizable solution for real-time comments based on websockets.  His talk, Realtime Web @ HuffingtonPost, was for developers of real-time, Erlang-based solutions—he covered architecture, lessons learned, pitfalls, and future improvements. The presentation objectives included:

  • Why they went with Erlang and SockJS for the commenting platform
  • How they integrated RabbitMQ routing to power the subscription architecture
  • How they managed subscriptions for real time channels
  • Their plans to extend the framework into an open source solution
  • Where they want other mechanisms for publishing comments outside of RabbitMQ Continue reading

How Indeed.com Handles 35 Million Job Postings Per Day Using RabbitMQ

Indeed.com is hosting a tech talk that will be a case study of how they scaled their aggregation engine to handle 35 million job postings per day using RabbitMQ. The talk will be hosted at Indeed’s engineering site in Austin, TX at 7 pm on March 27th.

For those of you not familiar with how large Indeed is, it is interesting to note that the job search company Indeed.com is one of the largest web sites in the world. According to Alexa.com, Indeed is currently the #224th biggest website in the world, and in cities like Atlanta and Chicago, it’s the 55th most popular website overall. According to research by SilkRoad, 2 out of every 5 hires came via Indeed (based on data from 150,000 hires).

As expected, the engineering team behind this large-scale application needs to support some very large scale numbers. In a recent post on their company blog, the Indeed team shared just how big those numbers are:

  • More than 100 million monthly unique visitors
  • More than 3 billion searches per month
  • More than 1000 searches per second
  • 50 country-specific sites in 26 languages

The scale of their application, both in terms of processing throughput and geographic diversity, means that the team relies on a messaging layer powered by RabbitMQ. Continue reading

10 Lessons from Spring Applied to Java Virtualization with vFabric

The Spring Framework became the de-facto standard for developing enterprise Java applications, and its radical simplicity was fundamental to its success. Why the “radical” simplicity? Because at the time, it was hard to imagine how creating such applications could be made simple.

By tackling issues such as portability, understanding the importance of cross-cutting concerns, and making it trivial to develop automated tests, Spring allowed developers to focus on what matters: what makes their application unique.

As I was pulling together my presentation for SpringOne2GX 2012, I reflected on the parallels between Spring’s success and the direction we were going with EM4J. Why did Spring succeed? Why did simplification win? Where are we replicating these patterns within VMware, vFabric, and Java?

In short, complexity is expensive, and simplification has many economic benefits. By giving people better, simpler, and easier to use tools to help build, run, and manage applications, we create economic advantages.

In a nutshell, there are some core reasons why Spring succeeded, “Spring values” if you will: Reducing complexity, increasing productivity, provisioning flexibility, tooling and monitoring, extensibility, automation, flexible integration and ease of testing. Continue reading

HOWTO: Deploying RabbitMQ Simulator on Cloud Foundry

Last week, we published the new RabbitMQ Simulator. I’m pleased to see how well it has been received by customers, with several very large scale RabbitMQ customers already incorporating it into their training and testing regimens. VMware employees in the field have been enjoying getting familiar with the tool as well and are using it to advance their own and their customer’s understanding of how RabbitMQ works and how to best design a messaging architecture. One sales engineer, Chris DeLashmutt even wrote a blog this week describing how to deploy the new RabbitMQ Simulator on Cloud Foundry.

For users new to RabbitMQ, certainly testing this out on Cloud Foundry can be a huge accelerator, as you can get your instance up and running for free in just a matter of minutes. Below is a summary of Chris’ instructions to deploy this simulator on Cloud Foundry, but also, if you want to shortcut the entire process, he invites you to access his deployed instance at http://rabbitmq-sim.cloudfoundry.com. Continue reading

Introducing the RabbitMQ Simulator, Video, & Open Source Bits

In this blog post I’d like to introduce you to one of our latest developments: the RabbitMQ Simulator.

This simulator was born out of a need—we wanted a better tool to teach RabbitMQ concepts to people that were new to the message broker and its AMQP protocol.

If you know me, you are probably aware that I have given quite a few RabbitMQ presentations over the year at several technical conferences. After explaining AMQP concepts many times with static images, I decided the time had come for a better tool—something more graphic and visual, something with more life and motion. So, I started working on this RabbitMQ Simulator project. Continue reading