Hyperion

1 API, multiple database backends.

Hyperion provides you with a simple API for data persistence allowing you to delay the choice of database without delaying your development.

There are a few guiding principles for Hyperion.

key/value store. All Hyperion implementations, even for relational databases, conform to the simple key/value store API.
values are maps. Every 'value' the goes in or out of a Hyperion datastore is a hash.
:key and :kind. Every 'value' must have a :kind entry; a short string like "user" or "product". Persisted 'value's will have a :key entry; strings generated by the datastore.
Search with data. All searches are described by data. See find_by_kind below.

Hyperion Implementations:

memory - an in-memory datastore, ideal for testing, included in the hyperion-api gem
mysql - MySQL
postgres - PostgreSQL
sqlite - SQLite
riak - Riak

Installation

To use just the in-memory datastore...

gem 'hyperion-api'

To use a specific implementation...

gem 'hyperion-postgres'
# or
gem 'hyperion-mysql'
# or
gem 'hyperion-sqlite'

Usage

Instantiating datastores

Hyperion provides a convenient factory function for instantiating any datastore implementation

require 'hyperion'
Hyperion.new_datastore(:memory)
Hyperion.new_datastore(:mysql)
Hyperion.new_datastore(:postgres, options)
Hyperion.new_datastore(:sqlite, options)

This will require the file "hyperion/<impl>" and instantiate the class Hyperion::<impl camelcased>

Each implementation must accept a hash of options in it's initializer

module Hyperion
  class Memory
    def initialize(opts={})
    end
  end
end

Installing a datastore

Before you can use the Hyperion API, you must tell Hyperion which backend datastore you would like to use. There a couple of ways to go about this.

With elegance

This installs the datastore thread locally, and only within the given block.

# datastore not installed
Hyperion.with_datastore(:postgres, options) do
  # datastore is installed
  # some persistence code here
end
# datastore not installed

This method is useful for applications that have control of the main thread, such as webapps. The call to with_datastore can be placed inside of a middleware, such that the datastore is installed at the beginning of the request and uninstalled after the request. For example, this is a Rack middleware which instantiates a postgres datastore, opens a pooled connection, starts a transaction and calls the next middleware.

require 'hyperion'
require 'hyperion/sql'

class DatastoreMiddleware

  def initialize(app)
    @app = app
  end

  def call(env)
    connection_url = 'postgres://cspvswmv:bwTTUFRBRgnb@ec2-23-23-234-187.compute-1.amazonaws.com:5432/d1uh0jkh0n8j3l'
    Hyperion.with_datastore(:postgres, :connection_url => connection_url) do
      Hyperion::Sql.with_connection(connection_url) do
        Hyperion::Sql.transaction do
          @app.call(env)
        end
      end
    end
  end
end

Each of these calls could, of course, be their own separate middlewares.

With brute force

This installs the datastore across all threads.

Hyperion.datastore = Hyperion.new_datastore(:postgres, options)

To uninstall

Hyperion.datastore = nil

This method is useful for when you do not have control of the main thread, such as desktop GUI applications. If you can bind the datastore once at high level in your application, that's ideal. Otherwise use the brute force technique.

Saving a value:

Hyperion.save({:kind => :foo}) #=> {:kind=>"foo", :key=>"<generated key>"}
Hyperion.save({:kind => :foo}, :value => :bar) #=> {:kind=>"foo", :value=>:bar, :key=>"<generated key>"}

Updating a value:

record = Hyperion.save({:kind => :foo, :name => 'Sam'})
Hyperion.save(record, :name => 'John') #=> {:kind=>"foo", :name=>'John', :key=>"<generated key>"}

Loading a value:

# if you have a key...
Hyperion.find_by_key(my_key)

# otherwise
Hyperion.find_by_kind(:dog) # returns all records with :kind of \"dog\"
Hyperion.find_by_kind(:dog, :filters => [[:name, '=', "Fido"]]) # returns all dogs whose name is Fido
Hyperion.find_by_kind(:dog, :filters => [[:age, '>', 2], [:age, '<', 5]]) # returns all dogs between the age of 2 and 5 (exclusive)
Hyperion.find_by_kind(:dog, :sorts => [[:name, :asc]]) # returns all dogs in alphebetical order of their name
Hyperion.find_by_kind(:dog, :sorts => [[:age, :desc], [:name, :asc]]) # returns all dogs ordered from oldest to youngest, and gos of the same age ordered by name
Hyperion.find_by_kind(:dog, :limit => 10) # returns upto 10 dogs in undefined order
Hyperion.find_by_kind(:dog, :sorts => [[:name, :asc]], :limit => 10) # returns upto the first 10 dogs in alphebetical order of their name
Hyperion.find_by_kind(:dog, :sorts => [[:name, :asc]], :limit => 10, :offset => 10) # returns the second set of 10 dogs in alphebetical order of their name

Filter operations and acceptable syntax:

"=" "eq"
"<" "lt"
"<=" "lte"
">" "gt"
">=" "gte"
"!=" "not"
"contains?" "contains" "in?" "in"

Sort orders and acceptable syntax:

:asc "asc" :ascending "ascending"
:desc "desc" :descending "descending"

Deleting a value:

# if you have a key...
Hyperion.delete_by_key(my_key)

# otherwise
Hyperion.delete_by_kind(:dog) # deletes all records with :kind of "dog"
Hyperion.delete_by_kind(:dog, :filters => [[:name, "=", "Fido"]]) # deletes all dogs whose name is Fido
Hyperion.delete_by_kind(:dog, :filters => [[:age, ">", 2], [:age, "<", 5]]) # deletes all dogs between the age of 2 and 5 (exclusive)

Entities

An entity is, in essence, configuration. By defining an entity, you are telling Hyperion how to save ('pack') and load ('unpack') a specific kind. Defining entites is not required or necessary in some cases. Hyperion will work just fine without them. However, they offer some very clear advantages.

Whitelisting

Only fields defined in the entity are allowed into the datastore.

Hyperion.defentity(:whitelisted) do |kind|
  kind.field(:age)
end

Hyperion.save(kind: :whitelisted, age: 23, hair_color: 'brown')
#=> {:kind=>"whitelisted", :key=><generated_key>, :age=>23}

Defaults

Default values can be assigned to fields.

Hyperion.defentity(:defaulted) do |kind|
  kind.field(:name)
  kind.field(:age, :default => 25)
end

Hyperion.save(kind: :defaulted, name: 'Myles')
#=> {:kind=>"whitelisted", :key=><generated_key>, :name=>"Myles", :age=>25}

Packers

Packers tell Hyperion how to treat certain fields when they are being saved.

Hyperion.defentity(:packed) do |kind|
  kind.field(:age, :packer => lambda {|value| value.to_i})
end

Hyperion.save(kind: :packed, age: '25')
#=> {:kind=>"packed", :key=><generated_key>, :age=>25}

A packer must be callable, i.e. respond to 'call'. The current value of a field will be passed to the packer, and the packer returns a potentially transformed value.

Unpackers

Unpackers tell Hyperion how to treat certain fields when they are being loaded.

Hyperion.defentity(:unpacked) do |kind|
  kind.field(:age, :packer => lambda {|age| age.to_i}, :unpacker => lambda {|age| age.to_s})
end

Hyperion.save(kind: :unpacked, age: '25')
#=> {:kind=>"unpacked", :key=><generated_key>, :age=>"25"}

An unpacker must be callable, i.e. respond to 'call'. The current value of a field will be passed to the unpacker, and the unpacker returns a potentially transformed value.

Types

Types are an easy way to share packers and unpackers between fields.

def to_int(value)
  if value
    value.to_i
  end
end

def to_str(value)
  if value
    value.to_s
  end
end

Hyperion.pack(Integer) {|value| to_int(value)}
Hyperion.unpack(Integer) {|value| to_int(value)}
Hyperion.pack(String) {|value| to_str(value)}
Hyperion.unpack(String) {|value| to_str(value)}

Hyperion.defentity(:typed) do |kind|
  kind.field(:age, :type => Integer)
  kind.field(:name, :type => String)
end

Hyperion.save(kind: :typed, age: '25', name: :a_symbol)
 => {:kind=>"typed", :key=><generated_key>, :age=>25, :name=>"a_symbol"}

If a type is specified as well as a custom packer or unpacker, the custom packer has priority.

Hyperion.defentity(:typed_2) do |kind|
  kind.field(:age, :type => Integer, :packer => lambda {|value| 2})
end

Hyperion.save(kind: :typed_2, age: '25')
#=> {:kind=>"typed_2", :key=><generated_key>, :age=>2}

Foreign Keys

In a traditional SQL database, you may have a schema that looks like this:

users:

id
first_name
created_at
updated_at

profiles:

id
user_id
created_at
updated_at

Since Hyperion presents every underlying datastore as a key-value store, configuring Hyperion to use this schema is a little tricky, but certainly possible.

This is what the corresponding defentity notation would be:

Hyperion.defentity(:users) do |kind|
  kind.field(:first_name)
  kind.field(:created_at)
  kind.field(:updated_at)
end

Hyperion.defentity(:profiles) do |kind|
  kind.field(:user_key, :type => Hyperion::Types.foreign_key(:users), :db_name => :user_id)
  kind.field(:created_at)
  kind.field(:updated_at)
end

myles = Hyperion.save(kind: :users, :first_name => "Myles")
#=> {:kind=>"users", :key=>"46e5ebef8554422db90f7eb4d01be4c6", :email=>nil, :first_name=>"Myles", :last_name=>nil, :created_at=>2012-12-02 17:30:41 -0600, :updated_at=>nil}

# myles is stored in the users table as:
# | id | first_name | created_at | updated_at |
# | 1  | Myles      | <time>     | <time>     |


myles_profile = Hyperion.save(:kind => :profiles, :user_key => myles[:key])
#=> {:kind=>"profiles", :user_key=>"46e5ebef8554422db90f7eb4d01be4c6", :key=>"356c89696b0d40f6a5754f225c01f7a9"}

# myles' profile is stored in the profiles table as:
# | id | user_id | created_at | updated_at |
# | 1  | 1       | <time>     | <time>     |

Using the Hyperion::Types.foreign_key helper, our foreign key references are stored in whatever way is conventional to the underlying datastore. In this example, the user_key field will be packed as an integer id, as stored in the user_id column.

If your schema requires foreign keys, ALWAYS USE THE FOREIGN KEY TYPE. If you do not, you will be storing generated keys instead of actual database ids. DO NOT DO THIS. If Hyperion changes the way it generates keys, all of your foreign key data will be useless.

Contributing

Clone the master branch, and run all the tests:

git clone git@github.com:mylesmegyesi/hyperion-ruby.git
cd hyperion-ruby
rake

Make patches and submit them along with an issue (see below).

Issues

Post issues on the hyperion-ruby github project:

https://github.com/mylesmegyesi/hyperion-ruby/issues

License

Distributed under the Eclipse Public License

hyperion-sqlite

Development

Runtime