We are pleased to announce the release of RabbitMQ 3.2.0!
There are a number of enhancements and bug fixes for the server as well as many plugins, including management, LDAP, federation, Web-STOMP, AMQP 1.0, STOMP, MQTT, Java, .NET, and more. These are all noted in the release notes and summarized further below. The most significant new enhancement is for federated queues, but there are also significant advances with policies, authentication failure notification, connection blocks, and consumer priority.
About Federated Queues
There is a problem when messaging systems hit maximum capacity and throughput. Federated queues help solve the problem by increasing capacity.
While the Federation Plugin has existed for some time, federated queues allow you to have the same “logical” queue distributed over many brokers, even distributed across a WAN, and allows for higher capacity than a single queue. In work-queuing scenarios, messages are moved to where they can be consumed. A federated queue links to another queue, called an upstream queue, via AMQP. These queues do not have to be on the same broker or in the same cluster, nor do the upstream queues need to be reconfigured. Here are a few scenarios where federated queues apply:
- Generally, work is queued and scaled by connecting more consumers to an existing broker or queue. Companies often start by making sure each broker has more than enough consuming capacity, meaning you do waste some capacity. However, there are still times when this doesn’t solve the problem because the broker is operating at maximum capacity, and the broker itself is the bottleneck, not the consumers. By federating queues, other brokers and their consumers can help out. If there is enough capacity globally, you don’t have to have the original partition perfect in advance. The federated queue will grab unconsumed, “spare” messages from upstream. For example, you might have producers and consumers in multiple locations, and you want NYC to “help out” London or vice versa when one is busier than the other.
- While it might be more temporary, you could also add a new broker, federate a queue within the new broker, and add more consumers to the new queue until a more fully load balanced solution can be put in place. The federated queue will help with capacity in this case, but the original queue will still take the majority of the load.
- Federated queues can also expand capacity within a single broker. A single queue can only use a single core, and this puts a limit on how many messages/second a queue can handle. When you need more capacity than the core allows, you ultimately have to spread the load across multiple queues. With federated queues, you can basically add queues to a single broker by federating multiple queues within the broker. In doing so, the resulting logical queue gets spread across cores and increases capacity.
- Federated queues also become useful when a WAN network link goes down. Let’s say you have producers in multiple locations and the broker is located with all the consumers in one place. If a network link goes down, you can’t really publish to the broker. With queue federation, you could have a local broker in each location.
For more information on federated queues, read the following blog post or documentation. There are also important considerations to be aware of in the upgrade process.
RabbitMQ 3.2.0 Server Enhancements
Here is a list of key enhancements in this release:
- Support for federated queues
- Allow alternate and dead-letter exchanges, queue max length, expiry and message TTL to be controlled by policy as well as AMQP arguments
- Report client authentication errors during connection establishment explicitly using basic.close
- Inform clients when memory or disk alarms are set or cleared
- Allow policies to target queues or exchanges or both
- Make it harder to trigger the disk space alarm with default settings
- Offer greater control over threshold at which messages are paged to disk
- Allow missing exchanges & queues to be deleted and unbound without generating an AMQP error
- Implement consumer priorities [4]
- Backport OTP process supervision infrastructure improvements
- Relax type constraints of header exchanges
- Add support for specifying a SSL verify_fun name in the config file
For more information on RabbitMQ
- Case Studies: Real-time Telematics with RabbitMQ or Scaling with RabbitMQ @Soundcloud
- Messaging with JMS and RabbitMQ, all GoPivotal Blog articles on RabbitMQ, and all vFabric Blog articles on RabbitMQ
- Dialogue on the mailing list
- Read the documentation
- Read the Pivotal RabbitMQ Product Overview
