Effective Date: August 4, 2024
Please be aware that we are no longer maintaing the part related to ActiveRecord in this gem. We are dropping support for ActiveRecord in favor of Solid Queue.
In the meantime, we commit to continously support the Mongoid part of the gem.
For ActiveRecord users, we recommend transitioning to the Solid Queue gem, which provides enhanced functionality, improved performance, and better support for modern application requirements. Solid Queue is designed to seamlessly integrate with your existing infrastructure while offering robust features to handle your queuing needs efficiently.
The Outboxable Gem is tailored for Rails applications to implement the transactional outbox pattern. It supports both ActiveRecord and Mongoid.
Please take into consideration that this Gem is opinionated, meaning it expects you to follow a certain pattern and specific setting. If you don't like it, you can always fork it and change it.
- When using RabbitMQ, it only publishes events to a topic exchange.
- It assumes that you are using routing keys to publish to the topic exchange.
- It publishes events in a background job using Sidekiq. Therefore, you application must use Sidekiq.
- It implements the polling publisher pattern. For that, it uses sidekiq-cron to check the unpublished outboxes every 5 seconds after the initialization of the application.
Install the gem and add to the application's Gemfile by executing:
$ bundle add outboxableIf bundler is not being used to manage dependencies, install the gem by executing:
$ gem install outboxableFor use with ActiveRecord, run:
$ rails g outboxable:install --orm activerecordFor use with Mongoid, run:
$ rails g outboxable:install --orm mongoidThe command above will add a migration file and the Outbox model. You will need then to run the migrations (ActiveRecord only):
$ rails db:migrateThe installation command above will also add a configuration file to your initializer:
# This monkey patch allows you to customize the message format that you publish to your broker.
# By default, Outboxable publishes a CloudEvent message to your broker.
module Outboxable
module RabbitMq
class Publisher
# Override this method to customize the message format that you publish to your broker
# DO NOT CHANGE THE METHOD SIGNATURE
def to_envelope(resource:)
{
id: resource.id,
source: 'http://localhost:3000',
specversion: '1.0',
type: resource.routing_key,
datacontenttype: 'application/json',
data: resource.payload
}.to_json
end
end
end
end
Outboxable.configure do |config|
# Specify the ORM you are using. Supported values are :activerecord and :mongoid
config.orm = :activerecord
# Specify the message broker you are using. For now, only RabbitMQ is supported.
config.message_broker = :rabbitmq
# RabbitMQ configurations
config.rabbitmq_host = ENV.fetch('RABBITMQ__HOST')
config.rabbitmq_port = ENV.fetch('RABBITMQ__PORT', 5672)
config.rabbitmq_user = ENV.fetch('RABBITMQ__USERNAME')
config.rabbitmq_password = ENV.fetch('RABBITMQ__PASSWORD')
config.rabbitmq_vhost = ENV.fetch('RABBITMQ__VHOST')
config.rabbitmq_exchange_name = ENV.fetch('RABBITMQ__EXCHANGE_NAME')
endThe monkey patch in the code above is crucial in giving you a way to customize the format of the message that you will publish to the message broker. Be default, it follows the specs of the Cloud Native Events Specifications v1.0.2.
The Outboxable Gem provides a general-purpose polymorphic model for creating outboxes. In order to maintain a transactional outbox, the changes that occur to a resource and the associated outbox must happen within the same transaction:
ActiveRecord::Base.transaction do
book = Book.find(1)
book.update!(title: 'Patterns of Enterprise Application Architecture', author: 'Martin Fowler')
Outbox.create!(
routing_key: 'books.updated',
exchange: 'YOUR RABBITMQ EXCHANGE',
payload: book.as_json,
outboxable: book
)
endIf you are intending to create an outbox that is not necessarily associated with a resource, you can use the following configuration that sets the size attribute to :batch:
ActiveRecord::Base.transaction do
book = Book.find(1)
book.update!(status: :published)
Outbox.create!(
routing_key: 'notifications.publish',
exchange: 'YOUR EXCHANGE',
payload: {
channels: ['sms', 'email'],
message: "Hello, world!"
},
size: :batch
)
endThe Outboxable Gem provides an ActiveRecord Model Concern that you can reuse in your models, which in turn will take care of transactionally creating events in case of create or update of the resource. All you have to do is to include Outboxable in your model and implement a method by the name of outbox_configurations:
class Book < ApplicationRecord
include Outboxable
# Enums
enum status: { draft: 0, published: 1 }
def outbox_configurations
@outbox_configurations ||= {
base: {
routing_key: 'books'
},
run_on: {
create: {},
update: {
condition: proc { |book| book.published? },
routing_key: 'books.published'
}
}
}
end
endThe outbox_configurations method will be called and used by the Outboxable Gem to transactionally create an outbox and publish. In the code above, it will create an outbox when the book is created. For that purpose it will use the routing key books.created as a convention. It will also publish an event if the book is updated, using the routing key: books.published since it was specified in the hash.
Here's the schema of what could be passed to the outbox_configurations in JSON Schema format:
{
"type": "object",
"properties": {
"base": {
"type": "object",
"properties": {
"routing_key": { "type": "string" }
}
},
"run_on": {
"type": "object",
"properties": {
"create": {
"type": "object",
"properties": {
"condition": { "type": "Ruby Proc" },
"routing_key": { "type": "string" }
}
},
"update": {
"type": "object",
"properties": {
"condition": { "type": "Ruby Proc" },
"routing_key": { "type": "string" }
}
}
}
}
}
}The run_on key represents another hash that can have the keys create and update. If one of these keys are not supplied, the outbox will not be created for the unspecified operation; in other words, if you do not specify the configuration for update, for example, an outbox will NOT be created when the book is updated.
Each operation key such as create and update can also take a condition key, which represents a Ruby proc that must return a boolean expression. It can also take a routing_key option, which specifies that routing key with which the outbox will publish the event to the message broker. If you don't specify the routing_key, it will use the base`s routing key dotted bycreated for create operation and ``updated`` for update operation.
Last but not least, run sidekiq so that the Outboxable Gem can publish the events to the broker:
$ bundle exec sidekiqThe Outboxable gem works smoothly with Mongoid. It is to be noted that when used with Mongoid, Outboxable does not use the _id as the idempotency key. It creates a field called idempotency_key which is a UUID generated at the time of the insertion of the document.
After checking out the repo, run bin/setup to install dependencies. Then, run rake spec to run the tests. You can also run bin/console for an interactive prompt that will allow you to experiment.
To install this gem onto your local machine, run bundle exec rake install. To release a new version, update the version number in version.rb, and then run bundle exec rake release, which will create a git tag for the version, push git commits and the created tag, and push the .gem file to rubygems.org.
Bug reports and pull requests are welcome on GitHub at https://github.com/[USERNAME]/outboxable. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the code of conduct.
The gem is available as open source under the terms of the MIT License.
Everyone interacting in the Outboxable project's codebases, issue trackers, chat rooms and mailing lists is expected to follow the code of conduct.