Schema-less attribute storage plugin for Sequel.
Database schemas are great: they enforce data integrity and help ensure that your data always looks like you expect it to. Furthermore, indexing can dramatically speed up many types of complex queries with a schema.
However, schemas can also get in the way: dozens of columns get to be unwieldy with most database tools, and migrations to add and drop columns can be difficult on large data sets.
Consequently, schema-less storage can be an excellent complement to a rigid database schema: you can define a schema for the well-understood parts of your data model, but augment it with schema-less storage for the parts that are in flux or just awkward to fit into the relational model.
Add attr_pouch to your Gemfile:
gem 'attr_pouch'Then run bundle install.
AttrPouch allows you to designate a "pouch" database field (of type
jsonb, hstore, or json) that provides schema-less storage for
any Sequel::Model object. Within this pouch, you can define
sub-fields that behave largely like standard Sequel fields (i.e., as
if they were backed by their own database columns), but are all in
fact stored in the pouch:
class User < Sequel::Model
include AttrPouch
pouch(:preferences) do
field :theme
field :autoplay_videos?
end
end
bob = User.create(name: 'bob')
bob.theme = 'gothic'
bob.autoplay_videos = false
bob.save_changes
bob.update(theme: 'ponies!', autoplay_videos: false)Note that if a field name ends in a question mark, its setter is defined with the trailing question mark stripped.
Because there is no schema defined for these fields, changes to the pouch definition do not require a database migration. You can add and remove fields with just code changes.
Fields are required by default; attempting to read a field that has
not been set will raise AttrPouch::MissingRequiredFieldError. They
can be marked as optional by providing a default:
class User < Sequel::Model
include AttrPouch
pouch(:preferences) do
field :favorite_color, default: 'puce'
end
end
ursula = User.create(name: 'ursula')
ursula.favorite_color # 'puce'AttrPouch supports per-type serializers and deserializers to allow any Ruby objects to be handled transparently as field values (as long as they can be serialized and deserialized in some fashion).
AttrPouch comes with a number of built-in encoders and decoders for
some common data types, and these are specified either directly using
Ruby classes or via symbols representing an encoding mechanism. The
simple built-in types are String, Integer, Float, Time, and
:bool (since Ruby has no single class representing a boolean type,
attr_pouch uses a symbol to stand in as a "logical" type here).
class User < Sequel::Model
include AttrPouch
pouch(:preferences) do
field :favorite_color, type: String
field :lucky_number, type: Integer
field :gluten_free?, type: :bool
end
endYou can override these built-ins or register entirely new types:
AttrPouch.configure do |config|
config.encode(:obfuscated_string) do |field, value|
value.reverse
end
config.decode(:obfuscated_string) do |field, value|
value.reverse
end
endNote that your encoder and decoder do have access to the field
definition object, which includes name, type, and any options you've
configured in the field definition. Option names are not checked by
attr_vault, so custom decoder or encoder options are possible.
When an encoder or decoder is specified via symbol, it will only work for fields whose type is declared to be exactly that symbol. When specified via class, it will also be used to encode and decode any fields whose declared type is a subclass of the encoder/decoder class.
This can be illustrated via the last built-in codec, for
Sequel::Model objects:
class User < Sequel::Model
include AttrPouch
pouch(:preferences) do
field :bff, type: User
end
end
alonzo = User.create(name: 'alonzo')
alonzo.update(bff: User[name: 'ursula'])Even though the built-in encoder is specified for just Sequel::Model
(no custom encoder was specified for User here), it can handle the
bff field above with no additional configuration because User
descends from Sequel::Model.
If the field type is not specified, it is inferred from the field definition. The default mechanism only considers the field name and infers types as follows:
Integer: name starts withnum_or ends in_sizeor_countTime: name ends with_ator_by:bool: name ends with?String: anything else
If this is not suitable, you can register your own "type inference" mechanism instead:
AttrPouch.configure do |config|
config.infer_type { |field| String }
endThe above just considers every field without a declared type to be a
String.
Fields can be marked deletable, which will generate two deleter
methods for them:
class User < Sequel::Model
include AttrPouch
pouch(:preferences) do
field :proxy_address, deletable: true
end
end
karen = User.create(name: 'karen')
karen.update(proxy_address: '10.11.12.13:8001')
karen.delete_proxy_address # note this does not save the object
karen.reload
karen.proxy_address # still '10.11.12.13:8001'; we never saved
karen.delete_proxy_address! # or call #save or #save_changes after a delete
karen.proxy_address # now it's goneDeletable fields are automatically given a default of nil if no
other default is present; reading a deletable field does not raise an
error.
N.B.: You can still delete fields without this flag by deleting their
corresponding keys directly from the underlying storage column,
flagging the field as modified (to ensure Sequel knows the field has
changed, since the object reference it holds does not change when
the object itself is modified). You can also set the value to nil
explicitly which, while not semantically identical, can be sufficient.
Fields are mutable by default but can be flagged immutable to reject updates once an initial value has been set:
class User < Sequel::Model
include AttrPouch
pouch(:preferences) do
field :lucky?, mutable: false
end
end
abbas = User.create(name: 'abbas', lucky: true)
abbas.lucky? # true
abbas.lucky = false # raises ImmutableFieldUpdateErrorFields can be renamed by providing a previous name or an array of
previous names under the was option.
class User < Sequel::Model
include AttrPouch
pouch(:preferences) do
field :tls?, was: :ssl?
field :instabul?, was: %i[constantinople? byzantion?]
end
end
nils = User[name: 'nils'] # in db we have `{ ssl?: true, byzantion?: true }`
nils.tls? # true
nils.istanbul? # trueNote that no direct accessors are defined for the old names, and if the value is updated, it is written under the new name and any old values in the pouch are deleted:
nils.byzantion? # raises NoMethodError
nils.istanbul = false
nils.tls? # true
nils.instanbul? # false
nils.save_changes # now in db as `{ tls?: true, instanbul?: false }`Any field can be accessed directly, bypassing the encoder and decoder,
by specifying the raw_field option to provide the name of the setter
and getter that will be created to manipulate the underlying value
directly. Required fields are still required when read via
raw_field, and immutable fields are still immutable, but if a
default is set, the raw value will be nil, rather than the default
itself, to allow the user to distinguish between a field value equal
to the default and an absent field value deferring to the default:
class User < Sequel::Model
include AttrPouch
pouch(:preferences) do
field :bff, type: User, raw_field: :bff_id
field :arch_nemesis, type: User, raw_field: :nemesis_id,
default: User[name: 'donald']
end
end
alonzo = User.create(name: 'alonzo')
alonzo.update(bff: User[name: 'ursula'])
alonzo.bff_id # Ursula's user id
alonzo.arch_nemesis # the User object representing the 'donald' record
alonzo.nemesis_id # nilRaw fields also obey the was option for renames, as above. If the
raw field value is updated, values present under any of the was keys
will be deleted.
AttrPouch provides a dataset method to simplify querying pouch contents. The interface is similar to querying first-class columns:
class User < Sequel::Model
include AttrPouch
pouch(:preferences) do
field :bff, type: User
field :favorite_color
end
end
# match on multiple fields
User.where_pouch(bff: User[name: 'georgia'], favorite_color: 'goldenrod').all
# match on multiple values
User.where_pouch(favorite_color: [ 'gray', 'dark gray', 'black' ]).all
# check for nil value (or absence of key)
User.where_pouch(bff: nil).allNote that you can speed up your queries considerably by adding an
index. AttrPouch uses containment queries to implement dataset
querying, so you can use GIN indexes with both
hstore
and
jsonb:
Sequel.migration do
up do
execute <<-EOF
CREATE INDEX users_prefs_idx ON users USING gin (preferences);
EOF
end
down do
execute <<-EOF
DROP INDEX users_prefs_idx;
EOF
end
EOF
endN.B.: For simplicity, AttrPouch treats the absence of a key and the
presence of a key with a NULL value the same when querying, which
means queries looking for nil values do not use the indexes.
Dataset queries are not supported for fields backed by columns of
type json.
AttrPouch requires a new storage field for each pouch added to a
model. It is currently designed for and tested with jsonb, hstore,
and json (although json-format fields do not currently support
dataset querying, as per above). Consider using a single pouch per
model class unless you clearly need several distinct pouches.
Sequel.migration do
change do
alter_table(:users) do
add_column :preferences, :jsonb
end
end
endPatches are warmly welcome.
To run tests locally, you'll need a DATABASE_URL environment
variable pointing to a database AttrPouch can use for testing. E.g.,
$ createdb attr_pouch_test
$ DATABASE_URL=postgres:///attr_pouch_test bundle exec rspecPlease follow the project's general coding style and open issues beforehand to discuss any significant behavior or API changes.
A pull request is understood to mean you are offering your code to the project under the MIT License.
Copyright (c) 2015 AttrPouch Contributors
MIT License. See LICENSE for full text.