Announced this morning on the new Pivotal blog, where RabbitMQ now resides, this version includes enhancements to garbage collection, consumption, requeuing, memory use, and dead lettering.
For those on Mac OS X, there is a newly packaged, standalone release of RabbitMQ that doesn’t require a separate Erlang install.
Some key, new capabilities include eager synchronisation of mirror queue slaves, automatic cluster partition healing, and improved statistics (including charts) in the management plugin. There are also many enhancements and bug fixes to the server, Java client, Erlang client, and a number of other plugins, including federation, old-federation, shovel, Web-STOMP, STOMP, and MQTT plugins, as well as the consistent hash exchange.
Instagram is one of the poster children for social media site successes. Founded in 2010, the photo sharing site now supports upwards of 90 million active photo-sharing users. As with every social media site, part of the fun is that photos and comments appear instantly so your friends can engage while the moment is hot. Recently, at PyCon 2013 last month, Instagram engineer Rick Branson shared how Instagram needed to transform how these photos and comments showed up in feeds as they scaled from a few thousand tasks a day to hundreds of millions.
Rick started off his talk demonstrating how traditional database approaches break, calling them the “naïve approach”. In this approach, when working to display a user feed, the application would directly fetch all the photos that the user followed from a single, monolithic data store, sort them by creation time and then only display the latest 10:
SELECT * FROM photos WHERE author_id IN (SELECT target_id FROM following WHERE source_id = %(user_id)d) ORDER BY creation_time DESC LIMIT 10;
Instead, Instagram chose to follow a modern distributed data strategy that will allow them to scale nearly linearly. Continue reading →
By using RabbitMQ, SockJS, Cowboy, Erlang, and more, his team built a highly scalable, customizable solution for real-time comments based on websockets. His talk, Realtime Web @ HuffingtonPost, was for developers of real-time, Erlang-based solutions—he covered architecture, lessons learned, pitfalls, and future improvements. The presentation objectives included:
Why they went with Erlang and SockJS for the commenting platform
How they integrated RabbitMQ routing to power the subscription architecture
How they managed subscriptions for real time channels
Their plans to extend the framework into an open source solution
Where they want other mechanisms for publishing comments outside of RabbitMQ Continue reading →
For those of you not familiar with how large Indeed is, it is interesting to note that the job search company Indeed.com is one of the largest web sites in the world. According to Alexa.com, Indeed is currently the #224th biggest website in the world, and in cities like Atlanta and Chicago, it’s the 55th most popular website overall. According to research by SilkRoad, 2 out of every 5 hires came via Indeed (based on data from 150,000 hires).
As expected, the engineering team behind this large-scale application needs to support some very large scale numbers. In a recent post on their company blog, the Indeed team shared just how big those numbers are:
More than 100 million monthly unique visitors
More than 3 billion searches per month
More than 1000 searches per second
50 country-specific sites in 26 languages
The scale of their application, both in terms of processing throughput and geographic diversity, means that the team relies on a messaging layer powered by RabbitMQ. Continue reading →
Last week, we published the new RabbitMQ Simulator. I’m pleased to see how well it has been received by customers, with several very large scale RabbitMQ customers already incorporating it into their training and testing regimens. VMware employees in the field have been enjoying getting familiar with the tool as well and are using it to advance their own and their customer’s understanding of how RabbitMQ works and how to best design a messaging architecture. One sales engineer, Chris DeLashmutt even wrote a blog this week describing how to deploy the new RabbitMQ Simulator on Cloud Foundry.
For users new to RabbitMQ, certainly testing this out on Cloud Foundry can be a huge accelerator, as you can get your instance up and running for free in just a matter of minutes. Below is a summary of Chris’ instructions to deploy this simulator on Cloud Foundry, but also, if you want to shortcut the entire process, he invites you to access his deployed instance at http://rabbitmq-sim.cloudfoundry.com. Continue reading →
In this blog post I’d like to introduce you to one of our latest developments: the RabbitMQ Simulator.
This simulator was born out of a need—we wanted a better tool to teach RabbitMQ concepts to people that were new to the message broker and its AMQP protocol.
If you know me, you are probably aware that I have given quite a few RabbitMQ presentations over the year at several technical conferences. After explaining AMQP concepts many times with static images, I decided the time had come for a better tool—something more graphic and visual, something with more life and motion. So, I started working on this RabbitMQ Simulator project. Continue reading →
One of the most common questions I’m asked to cover when I discuss software architecture topics is the difference between the various application messaging protocols that exist today—issues like how and why the protocols came about, and which one should be used in a particular application.
Their question is valid.
Today, application architects need to use a messaging broker to speed and scale their applications, particularly in the cloud. Even once you select your messaging middleware application, application developers need to then select the protocol. Understanding the subtle differences between them can be difficult.
Today, we will consider three of the most common and popular TCP/IP-based messaging protocols, and provide a quick summary on the advantages of each: AMQP, MQTT and STOMP. Before we go on, I should also point out that all three of these protocols are supported in RabbitMQ version 3.0—something we will use as an example and come back to later.
We are very fortunate to post an interview with Shay Banon, the founder of elasticsearch. Elasticsearch is technology that is very popular among some of the coolest companies on the web today, including SoundCloud, StumbleUpon, Mozilla and Klout. These companies use elasticsearch to help them deploy powerful search capabilities in their applications that are easy to set up, scalable and built for the cloud. In this interview, we get to learn all kinds of cool things:
How Shay got into search
How he came up with the idea for elasticsearch
Why elastic search is different than other OSS search projects
Running elasticsearch on virtualized infrastructure
Without further ado, here is the interview.
Q1. So, how did you end up getting into search?
About 10 years ago, I moved from Israel to London because my wife was going to study to be a chef at the Cordon Bleu. I had no job. I was in a new country. I was unemployed. So, I started to get into the latest, cool, new technologies. Continue reading →
In this article (and demonstration further below), we will show you six steps that give you an idea of how easy it is to provision using VAS. We will show you how to install VAS and use it to provision vFabric tc Server across three nodes along with a WAR file. The explanation below refers to examples from RubyGems.org and GitHub/vFabric/VAS-Ruby-API along with the latest VAS documentation.
Application developers and data management teams continue to look for ways to modernize legacy apps, manage costs more effectively, build new apps on robust application platforms, and solve big data problems. These are some of the key reasons why vFabric is on the CIO (or CTO) agenda. With several new product releases in the vFabric Suite, VMware continues to provide a best-in-class application platform and help customers solve their top application development and data management problems.