Home > Blogs > VMware vFabric Blog > Tag Archives: AMQP

Tag Archives: AMQP

Scaling Real-Time Comments @ HuffPost Live with RabbitMQ

Last week, one of the engineering leaders from Huffingtonpost.com presented at the Erlang Factory, a 2-day conference held in San Francisco with 50+ speakers across 8 tracks (not the cassette kind). Among the presenters was, Adam Denenberg, VP of Engineering at the Huffington Post and HuffPost Live. Besides leading the engineering teams for Huffingtonpost.com and HuffPost Live, Adam is also responsible for community products and advertising platforms.

By using RabbitMQ, SockJS, Cowboy, Erlang, and more, his team built a highly scalable, customizable solution for real-time comments based on websockets.  His talk, Realtime Web @ HuffingtonPost, was for developers of real-time, Erlang-based solutions—he covered architecture, lessons learned, pitfalls, and future improvements. The presentation objectives included:

  • Why they went with Erlang and SockJS for the commenting platform
  • How they integrated RabbitMQ routing to power the subscription architecture
  • How they managed subscriptions for real time channels
  • Their plans to extend the framework into an open source solution
  • Where they want other mechanisms for publishing comments outside of RabbitMQ Continue reading

Introducing the RabbitMQ Simulator, Video, & Open Source Bits

In this blog post I’d like to introduce you to one of our latest developments: the RabbitMQ Simulator.

This simulator was born out of a need—we wanted a better tool to teach RabbitMQ concepts to people that were new to the message broker and its AMQP protocol.

If you know me, you are probably aware that I have given quite a few RabbitMQ presentations over the year at several technical conferences. After explaining AMQP concepts many times with static images, I decided the time had come for a better tool—something more graphic and visual, something with more life and motion. So, I started working on this RabbitMQ Simulator project. Continue reading

Choosing Your Messaging Protocol: AMQP, MQTT, or STOMP

One of the most common questions I’m asked to cover when I discuss software architecture topics is the difference between the various application messaging protocols that exist today—issues like how and why the protocols came about, and which one should be used in a particular application.

Their question is valid.

Today, application architects need to use a messaging broker to speed and scale their applications, particularly in the cloud. Even once you select your messaging middleware application, application developers need to then select the protocol. Understanding the subtle differences between them can be difficult.

Today, we will consider three of the most common and popular TCP/IP-based messaging protocols, and provide a quick summary on the advantages of each: AMQP, MQTT and STOMP. Before we go on, I should also point out that all three of these protocols are supported in RabbitMQ version 3.0—something we will use as an example and come back to later.

So, in alphabetical order…

AMQP in a Nutshell Continue reading

Messaging Architecture: Using RabbitMQ at the World’s 8th Largest Retailer

Today, we are pleased to have a guest blogger from a VMware customer share with us their story of how RabbitMQ transformed their business by “solving some really interesting problems”. The following is sent courtesy of Pablo Molnar of MercadoLibre:

If you haven’t heard of MercadoLibre (NASDAQ: MELI), we are the largest e-commerce ecosystem in Latin America. Our website offers a wide range of services to sellers and buyers throughout the region including marketplace, payments, advertising, and e-building solutions. Our products are present in over 14 countries, and the company is ranked as 8th largest online retailer in the world. We were also on Fortune’s list of the fastest growing companies in 2012, and we use RabbitMQ to solve some interesting problems.

About Our Technology Stack and How RabbitMQ Helps

In terms of technology infrastructure, MercadoLibre is fully committed to the open source development model. Most of our apps are primarily written in Grails, Groovy, and NodeJS,  but we don’t stick to any language or framework. We entrust tool selection responsibilities to the Software Engineers on each team. Almost all applications are hosted by our in-house cloud computing provisioning system and implemented via OpenStack with more than +7000 virtual instances at the moment. Also, we have successfully launched applications using emerging storage solutions like Redis and MongoDB. With an average of 20 million requests per minute and 4GB bandwidth per second, our traffic management layer is crucial and most of the routing rules job is done by Nginx proxy servers. Our labs department includes a huge Apache Hadoop cluster to perform complex analytical queries, and we are experimenting with real-time data processing using Apache Kafka and Storm.

Continue reading

Expert Interview: The Polyglot Rabbit: Examples of Multi-Protocol Queues in RabbitMQ

Recently, we had the opportunity to speak with architect Brett Cameron about vFabric RabbitMQ.  A popular speaker, Brett is well known for his effort to port Erlang and RabbitMQ over to the “legacy” OpenVMS operating system platform (now owned by HP). With over 19 years in the software industry, Brett specializes in systems integration and large, distributed systems. Of course, he has spent a lot of time with OpenVMS – an OS with one of the more interesting histories in the software industry.

When we started chatting with Brett, he had recently discussed the concept of the Polyglot Rabbit with Alexis Richardson and written a great article titled, “The Polyglot Rabbit: Examples of Multi-Protocol Queues in RabbitMQ.” According to Brett, the main goal of this article is about the fact that you can publish messages into this environment via one protocol and consume via one or more other protocols (simultaneously if you want). “It’s a brilliant and a very powerful capability.” Brett felt that this capability was possibly not being promoted enough, and hopefully the article will go some way towards fixing this.

Continue reading

2 Lessons on RabbitMQ from HighLoad – Extending the server with Plugins and HA Queues

Recently, I presented at the HighLoad++ conference in Moscow Russia. Many of the developers I spoke with were interested in two pieces of information about RabbitMQ, and I wanted to share these with a broader audience:

  • Extending through Plugins
  • High Availability Queuing

If you haven’t heard of HighLoad++, the conference is pretty special since it focuses only on high traffic websites (mostly from Russia). The main point of the conference is to talk about new architectures and approaches for highly complex systems and covers things like:

Continue reading

Sneak Peak at VMware’s Nanotrader Reference Architecture

(Note: See a newer article on the renamed SpringTrader.) vFabric’s Reference Application, Nanotrader, provides customers with an end-to-end solution for developing, provisioning, and managing a distributed application in a cloud environment. The reference application and architecture provide customers and partners with a blueprint for development, infrastructure, and operations teams.

The Nanotrader application is based on the web based Trading application, Day Trader. The legacy Day Trader application provides context for application modernization by representing the “before” picture while Nanotrader provides developers with a blueprint for achieving the “after” picture. Functionally, both applications allow users to login, view their portfolio, lookup stock quotes, and buy or sell stock shares. However, this is where their similarity ends. The following table describes key high-level differences in the application design: Continue reading

VMware vFabric Suite 5.1 Significantly Upgrades vFabric RabbitMQ

Rabbitmq_logo_strapvFabric RabbitMQ, based on the open source RabbitMQ project, has significantly updated the underlying RabbitMQ version to 2.8.1 in the latest VMware vFabric Suite 5.1 update. 

The new vFabric RabbitMQ release represents a significant upgrade from the open source RabbitMQ 2.4.1 that vFabric 5.0 was based on, and brings the commercially supported vFabric RabbitMQ to parity with recent open source RabbitMQ releases by rolling up all of the interim code changes. Furthermore, users now have the option of choosing to use the commercially bundled vFabric RabbitMQ from VMware or using the open source RabbitMQ distribution with a single license. Either way a user chooses, they can receive the same reliable support from VMware.

Performance Improvements

  • New internal message flow control to limit memory use and make performance more predictable if the server is overloaded
  • Recovery has been simplified, improving startup times when many exchanges or bindings exist
  • Better performance under high load and memory pressure
  • Improved inbound network performance
  • Improved routing performance
  • Significant performance improvements to connection creation, durable queues, message storage, and large numbers of consumers

Continue reading

Three Reasons why you need vFabric Administration Server

In the 5.1 release of the vFabric Suite, we’ve added a new tool – vFabric Administration Server (VAS). VAS makes it significantly easier to administer some of the components of the vFabric Suite across small deployments with a handful of machines or something much larger. The first release of the Administration Server provides support for administering GemFire, RabbitMQ, and tc Server via a REST API. Some examples of VAS’s capabilities include installing GemFire, RabbitMQ, or tc Server, starting and stopping them, managing their configuration files, and deploying web applications to tc Server.

While there are more capabilities, here are three reasons why you need vFabric Administration Server:

  1. Improved Deployment Consistency and Time Savings
  2. Quick and Easy Scale Up and Down
  3. Simplified and Centralized Administration

Improved Deployment Consistency and Time Savings

VAS-group-membersA common problem encountered by our customers is ensuring that their deployments are consistent when spread across multiple machines. It’s all too easy for a deployment that started out with identically configured machines to slowly diverge as changes are made inconsistently across the machines. vFabric Administration Server addresses this by building upon the concept of a single system image, i.e. a collection of systems that are used as a single system. In VAS, a single system image is known as a group, with each system in the group being called a node.

Continue reading