By using RabbitMQ, SockJS, Cowboy, Erlang, and more, his team built a highly scalable, customizable solution for real-time comments based on websockets. His talk, Realtime Web @ HuffingtonPost, was for developers of real-time, Erlang-based solutions—he covered architecture, lessons learned, pitfalls, and future improvements. The presentation objectives included:
Why they went with Erlang and SockJS for the commenting platform
How they integrated RabbitMQ routing to power the subscription architecture
How they managed subscriptions for real time channels
Their plans to extend the framework into an open source solution
Where they want other mechanisms for publishing comments outside of RabbitMQ Continue reading →
In this blog post I’d like to introduce you to one of our latest developments: the RabbitMQ Simulator.
This simulator was born out of a need—we wanted a better tool to teach RabbitMQ concepts to people that were new to the message broker and its AMQP protocol.
If you know me, you are probably aware that I have given quite a few RabbitMQ presentations over the year at several technical conferences. After explaining AMQP concepts many times with static images, I decided the time had come for a better tool—something more graphic and visual, something with more life and motion. So, I started working on this RabbitMQ Simulator project. Continue reading →
One of the most common questions I’m asked to cover when I discuss software architecture topics is the difference between the various application messaging protocols that exist today—issues like how and why the protocols came about, and which one should be used in a particular application.
Their question is valid.
Today, application architects need to use a messaging broker to speed and scale their applications, particularly in the cloud. Even once you select your messaging middleware application, application developers need to then select the protocol. Understanding the subtle differences between them can be difficult.
Today, we will consider three of the most common and popular TCP/IP-based messaging protocols, and provide a quick summary on the advantages of each: AMQP, MQTT and STOMP. Before we go on, I should also point out that all three of these protocols are supported in RabbitMQ version 3.0—something we will use as an example and come back to later.
Today, we are pleased to have a guest blogger from a VMware customer share with us their story of how RabbitMQ transformed their business by “solving some really interesting problems”. The following is sent courtesy of Pablo Molnar of MercadoLibre:
If you haven’t heard of MercadoLibre (NASDAQ: MELI), we are the largest e-commerce ecosystem in Latin America. Our website offers a wide range of services to sellers and buyers throughout the region including marketplace, payments, advertising, and e-building solutions. Our products are present in over 14 countries, and the company is ranked as 8th largest online retailer in the world. We were also on Fortune’s list of the fastest growing companies in 2012, and we use RabbitMQ to solve some interesting problems.
About Our Technology Stack and How RabbitMQ Helps
In terms of technology infrastructure, MercadoLibre is fully committed to the open source development model. Most of our apps are primarily written in Grails, Groovy, and NodeJS, but we don’t stick to any language or framework. We entrust tool selection responsibilities to the Software Engineers on each team. Almost all applications are hosted by our in-house cloud computing provisioning system and implemented via OpenStack with more than +7000 virtual instances at the moment. Also, we have successfully launched applications using emerging storage solutions like Redis and MongoDB. With an average of 20 million requests per minute and 4GB bandwidth per second, our traffic management layer is crucial and most of the routing rules job is done by Nginx proxy servers. Our labs department includes a huge Apache Hadoop cluster to perform complex analytical queries, and we are experimenting with real-time data processing using Apache Kafka and Storm.
Recently, we had the opportunity to speak with architect Brett Cameron about vFabric RabbitMQ. A popular speaker, Brett is well known for his effort to port Erlang and RabbitMQ over to the “legacy” OpenVMS operating system platform (now owned by HP). With over 19 years in the software industry, Brett specializes in systems integration and large, distributed systems. Of course, he has spent a lot of time with OpenVMS – an OS with one of the more interesting histories in the software industry.
When we started chatting with Brett, he had recently discussed the concept of the Polyglot Rabbit with Alexis Richardson and written a great article titled, “The Polyglot Rabbit: Examples of Multi-Protocol Queues in RabbitMQ.” According to Brett, the main goal of this article is about the fact that you can publish messages into this environment via one protocol and consume via one or more other protocols (simultaneously if you want). “It’s a brilliant and a very powerful capability.” Brett felt that this capability was possibly not being promoted enough, and hopefully the article will go some way towards fixing this.
If you haven’t heard of HighLoad++, the conference is pretty special since it focuses only on high traffic websites (mostly from Russia). The main point of the conference is to talk about new architectures and approaches for highly complex systems and covers things like:
(Note: See a newer article on the renamed SpringTrader.) vFabric’s Reference Application, Nanotrader, provides customers with an end-to-end solution for developing, provisioning, and managing a distributed application in a cloud environment. The reference application and architecture provide customers and partners with a blueprint for development, infrastructure, and operations teams.
The Nanotrader application is based on the web based Trading application, Day Trader. The legacy Day Trader application provides context for application modernization by representing the “before” picture while Nanotrader provides developers with a blueprint for achieving the “after” picture. Functionally, both applications allow users to login, view their portfolio, lookup stock quotes, and buy or sell stock shares. However, this is where their similarity ends. The following table describes key high-level differences in the application design: Continue reading →
vFabric RabbitMQ, based on the open source RabbitMQ project, has significantly updated the underlying RabbitMQ version to 2.8.1 in the latest VMware vFabric Suite 5.1 update.
The new vFabric RabbitMQ release represents a significant upgrade from the open source RabbitMQ 2.4.1 that vFabric 5.0 was based on, and brings the commercially supported vFabric RabbitMQ to parity with recent open source RabbitMQ releases by rolling up all of the interim code changes. Furthermore, users now have the option of choosing to use the commercially bundled vFabric RabbitMQ from VMware or using the open source RabbitMQ distribution with a single license. Either way a user chooses, they can receive the same reliable support from VMware.
New internal message flow control to limit memory use and make performance more predictable if the server is overloaded
Recovery has been simplified, improving startup times when many exchanges or bindings exist
Better performance under high load and memory pressure
Improved inbound network performance
Improved routing performance
Significant performance improvements to connection creation, durable queues, message storage, and large numbers of consumers
In the 5.1 release of the vFabric Suite, we’ve added a new tool – vFabric Administration Server (VAS). VAS makes it significantly easier to administer some of the components of the vFabric Suite across small deployments with a handful of machines or something much larger. The first release of the Administration Server provides support for administering GemFire, RabbitMQ, and tc Server via a REST API. Some examples of VAS’s capabilities include installing GemFire, RabbitMQ, or tc Server, starting and stopping them, managing their configuration files, and deploying web applications to tc Server.
While there are more capabilities, here are three reasons why you need vFabric Administration Server:
Improved Deployment Consistency and Time Savings
Quick and Easy Scale Up and Down
Simplified and Centralized Administration
Improved Deployment Consistency and Time Savings
A common problem encountered by our customers is ensuring that their deployments are consistent when spread across multiple machines. It’s all too easy for a deployment that started out with identically configured machines to slowly diverge as changes are made inconsistently across the machines. vFabric Administration Server addresses this by building upon the concept of a single system image, i.e. a collection of systems that are used as a single system. In VAS, a single system image is known as a group, with each system in the group being called a node.