Teporingo ¹

Clojure RabbitMQ client with a focus on HA Configurations.

See teporingo/README.textile
for a more detailed description of the publisher and consumer APIs.

This project takes much inspiration from, and was heavily influenced by, Beetle, a HA solution for Ruby and RabbitMQ.

¹ http://en.wikipedia.org/wiki/Volcano_Rabbit

Default configuration is for exchanges, queues and bindings to be durable.

Publishing

Publishers are a list of broker configurations. Publishers are registered with logical names. Publishing is performed to all the brokers in the configuration. At least 1 broker must be published to or the client library will fail, throwing an exception. This ensures reliable message publishing.

Publish connections that encounter transient errors, a timeout or a broker failure, are passed off to an agent in order to be reconnected asynchronously from the code that is utilizing the publisher. In cases where a broker fails, this allows application code to continue publishing to the remaining brokers. When the connection is re-established, the publisher will begin using the connection again. The goal is to provide redundancy for message delivery as well as robustness (recovery) in the event a broker goes off line and comes back on-line.

Consuming

Consumers are registered as with a logical name. A consumer is an amqp endpoint configuration (vhost, exchange, queue, bindings) plus event handler functions that follow the RabbitMQ Java Consumer interface. Consumers can be started / stopped by invoking ‘add-consumer’ with a given registered name. This allows pools of consumers to be grown or shrunk interactively.

By default, consumers attempt to reconnect if they are shut down unexpectedly. Reconnect attempts will occur (default is 250ms) until the connection can be re-established or until they are explicitly shut down.

Getting Started

To run the example code, you’ll need to run a few services: RabbitMQ, Redis and HA Proxy. Each of these can be executed in a seperate terminal with a Rake task from the project’s root directory.

Clone the git repo:

   git clone [email protected]:kyleburton/teporingo.git

Execute each of the following in a separate terminal:

HA Proxy:

    rake teporingo:haproxy:run

The First RabbitMQ Broker:

    rake teporingo:rabbit:start_rabbit01

The Second RabbitMQ Broker:

    rake teporingo:rabbit:start_rabbit02

A Redis Master Server:

    rake teporingo:redis:master:run

Each of these will download and build the required services. Once these services are running you can start the consumers and then the publisher.

To run the consumers:

  rake teporingo:examples:consumer

For the publisher:

  rake teporingo:examples:publisher

With this executing you should see the publisher publishing message and the consumers processing (and deduplicating messages). At any time you can kill one of the RabbitMQ brokers (with a CTRL-C, followed by an ‘a’), publishing should continue unabated, and one of the consumers should continue to process messages. The other consumer will continually attempt to re-establish a connection to the failed broker until you bring it back up.

Bring the aborted broker back on-line and you should see the publisher and failed consumer recover and begin processing again.

License

Copyright (©) 2011 Kyle R. Burton <[email protected]>

Eclipse Public License 1.0 (http://opensource.org/licenses/eclipse-1.0.php)