Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adapters #13

Open
wants to merge 14 commits into
base: master
Choose a base branch
from
93 changes: 51 additions & 42 deletions README.mdown
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ Application A can publish an event

```ruby
# config
ResqueBus.redis = "192.168.1.1:6379"
Resque.redis = "192.168.1.1:6379"

# business logic
ResqueBus.publish("user_created", "id" => 42, "first_name" => "John", "last_name" => "Smith")
Expand All @@ -29,7 +29,7 @@ Application B is subscribed to events

```ruby
# config
ResqueBus.redis = "192.168.1.1:6379"
Resque.redis = "192.168.1.1:6379"

# initializer
ResqueBus.dispatch("app_b") do
Expand All @@ -38,21 +38,21 @@ ResqueBus.dispatch("app_b") do
subscribe "user_created" do |attributes|
NameCount.find_or_create_by_name(attributes["last_name"]).increment!
end

# processes event on app_b_critical queue
# critical is short-hand to subscribe to your 'critical' queue and this block with process events with the name "user_paid"
critical "user_paid" do |attributes|
CreditCard.charge!(attributes)
end

# you can pass any queue name you would like to process from as well IE: `banana "peeled" do |attributes|`

# and regexes work as well. note that with the above configuration along with this regex,
# the following as well as the corresponding block above would both be executed
subscribe /^user_/ do |attributes|
Metrics.record_user_action(attributes["bus_event_type"], attributes["id"])
end

# the above all filter on just the event_type, but you can filter on anything
# this would be _any_ event that has a user_id and the page value of homepage regardless of bus_event_type
subscribe "my_key", { "user_id" => :present, "page" => "homepage"} do
Expand All @@ -61,7 +61,7 @@ ResqueBus.dispatch("app_b") do
end
```

Applications can also subscribe within classes using the provided `Subscriber` module.
Applications can also subscribe within classes using the provided `Subscriber` module.

```ruby
class SimpleSubscriber
Expand Down Expand Up @@ -115,15 +115,43 @@ Each app needs to tell Redis about its subscriptions:
The subscription block is run inside a Resque worker which needs to be started for each app.

$ rake resquebus:setup resque:work

The incoming queue also needs to be processed on a dedicated or all the app servers.

$ rake resquebus:driver resque:work

If you want retry to work for subscribing apps, you should run resque-scheduler

$ rake resque:scheduler

### Adapters

ResqueBus now supports multiple adapters! By default ResqueBus uses Resque but you can now configure your application to use Sidekiq to drive and subscribe the bus.

First be sure to configure ResqueBus to use Sidekiq early in your applications' initialization cycle:
```
ResqueBus.adapter = 'Sidekiq'
```
You will be responsible for setting up the queues for your Sidekiq clients however you can get the appropriate queue names with the following tasks:
For driving applications:
```
$ rake resquebus:driver:sidekiq
```
For subscribing applications:
```
$ rake resquebus:setup:sidekiq
```
These tasks will provide the queue_names and some minimal suggestions for starting the client.

Your subscribing applications will still need to also use the appropriate rake task:
```
$ rake resquebus:subscribe:sidekiq
```

At the moment you are expected to include the Sidekiq gem in your own applications.

And yes we are planning on renaming and restructuring the project! Please contact the maintainer if you would like to add a different adapter.

### Heartbeat

We've found it useful to have the bus act like `cron`, triggering timed jobs throughout the system. Resque Bus calls this a heartbeat.
Expand Down Expand Up @@ -184,13 +212,13 @@ ResqueBus.dispatch("app_c") do
subscribe "once_an_hour", 'bus_event_type' => 'heartbeat_minutes', 'minute' => 20 do |attributes|
Sitemap.generate!
end

# runs every five minutes
subscribe "every_five_minutes", 'bus_event_type' => 'heartbeat_minutes' do |attributes|
next unless attributes["epoch_minutes"] % 5 == 0
HealthCheck.run!
end

# runs at 8am on the first of every month
subscribe "new_month_morning", 'bus_event_type' => 'heartbeat_minutes', 'day' => 1, hour' => 8, 'minute' => 0, do |attributes|
next unless attributes["epoch_minutes"] % 5 == 0
Expand All @@ -199,57 +227,38 @@ ResqueBus.dispatch("app_c") do
end
```

### Compatibility

ResqueBus can live along side another instance of Resque that points at a different Redis server.

```ruby
# config
Resque.redis = "192.168.1.0:6379"
ResqueBus.redis = "192.168.1.1:6379"
```

If no Redis instance is given specifically, ResqueBus will use the Resque one.

```ruby
# config
Resque.redis = "192.168.1.0:6379"
```

That will use the default (resque) namespace which can be helpful for using the tooling. Conflict with queue names are unlikely. You can change the namespace if you like though.

```ruby
# config
Resque.redis = "192.168.1.0:6379"
ResqueBus.redis.namespace = :get_on_the_bus
```

### Local Mode

For development, a local mode is also provided and is specified in the configuration.
For development, a local mode is provided and is specified in the configuration.

```ruby
# config
ResqueBus.local_mode = :standalone
or
or
ResqueBus.local_mode = :inline
```

Standalone mode does not require a separate resquebus:driver task to be running to process the
incoming queue. Simply publishing to the bus will distribute the incoming events
to the appropriate application specific queue. A separate resquebus:work task does
incoming queue. Simply publishing to the bus will distribute the incoming events
to the appropriate application specific queue. A separate resquebus:work task does
still need to be run to process these events

Inline mode skips queue processing entirely and directly dispatches the
event to the appropriate code block.
event to the appropriate code block.

You can also say `ResqueBus.local_mode = :suppress` to turn off publishing altogether.
This can be helpful inside some sort of migration, for example.

### TODO

* Sidekiq adapter
* Refactor rake tasks for resque/sidekiq
* Refactor to a storage adapter for Redis, so we can store subscription info in MySQL or something else
* Replace local modes with adapters
* There are a few spots in the code with TODO notes
* Make this not freak out in development without Redis or when Redis is down
* We might not actually need to publish in tests
* Add some rspec helpers for the apps to use: should_ post an event_publish or something along those lines
* Allow calling resquebus:setup and resquebus:driver together (append to ENV['QUEUES'], don't replace it)

Copyright (c) 2011 Brian Leonard, released under the MIT license
1 change: 0 additions & 1 deletion Rakefile
Original file line number Diff line number Diff line change
@@ -1,3 +1,2 @@
$LOAD_PATH.unshift 'lib'
require 'resque/tasks'
require 'resque_bus/tasks'
Loading