Frivol - Frivolously simple temporary storage backed by Redis¶ ↑

A really simple Redis-backed temporary storage mechanism intended to be used with ActiveRecord, but will work with other ORM’s or any classes really. I developed Frivol secifically for use in Mad Mimi (madmimi.com) to help with caching of data which requires fairly long running (multi-second) database queries, and also to aid with communication of status from background Resque jobs running on the workers to the front end web servers. Redis was chosen because we already had Resque, which is Redis-backed. Also, unlike memcached, Redis persists it’s data to disk, meaning there is far less warmup required when a hot system is restarted. Frivol’s design is such that it solves our problem, but I believe it is generic enough to be used in many Rails web projects and even in other types of projects altogether.

As of version 0.4.0, Frivol supports various backends, including Redis::Destributed and Riak. There’s a Multi backend which will migrate keys from an old backend to a new one, like from Redis to Redis::Destributed.

Configure Frivol in your configuration, for example in an initializer or in environment.rb

REDIS_CONFIG = {
  :host => "localhost",
  :port => 6379
}
Frivol::Config.backend = Frivol::Backend::Redis.new(REDIS_CONFIG)

Note: This configuration has changed in version 0.4.0 with the introduction of Backends

You can also use RedisDestributed or Riak backends. These are configured similarly. Finally there is a Multi which works to migrate data from an old backend to a new one. For example, we used a Multi backend to migrate from a single Redis instance to a RedisDistributed pair.

Now include Frivol in whichever classes you’d like to make use of temporary storage. You can optionally call the storage_expires_in(time) class method to set a default expiry. In your methods you can now call the store(keys_and_values) and retrieve(keys_and_defaults) methods. Defaults in the retrieve method can be symbols, in which case Frivol will check if the class respond_to? a method by that name to get the default.

The expire_storage(time) method can be used to set the expiry time in seconds of the temporary storage. The default is not to expire the storage, in which case it will live for as long as Redis keeps it. delete_storage, as the name suggests will immediately delete the storage, while clear_storage will clear the cache that Frivol keeps and force the next retrieve to return to Redis for the data.

class BigComplexCalcer
  include Frivol
  storage_expires_in 600 # temporary storage expires in 10 minutes.
  def initialize(key)
    @key = key
  end
  def storage_key(bucket = nil)
    "frivol-test-#{key}" # override the storage key because we don't respond_to? :id, and don't care about buckets
  end
  def big_complex_calc
    retrieve :complex => :do_big_complex_calc # do_big_complex_calc is the method to get the default from
  end
  def last_calc_done
    last = retrieve(:last => nil) # default is nil
    return "never" if last.nil?
    return "#{Time.now - Time.at(last)} seconds ago"
  end
  def do_big_complex_calc
    # Wee! Do some really hard work here...
    # ...still working...
    store :complex => result, :last => Time.now.to_i # ...and let's keep the result for at least 10 minutes, as well as the last timme we did it
  end
end

Since version 0.1.5 Frivol can create different storage buckets. Note that this introduces a breaking change to the storage_key method if you have overriden it. It now takes a bucket parameter.

Buckets can have their own expiry time and there are special counter buckets which simply keep an integer count.

storage_bucket :my_bucket, :expires_in => 5.minutes
storage_bucket :my_counter, :counter => true

Given the above, Frivol will create store_my_bucket and retrieve_my_bucket methods which work exactly like the standard store and retrieve methods. There will also be store_my_counter, retrieve_my_counter and increment_my_counter methods. The counter store and retrieve only take a integer (value and default, respectively) and the increment does not take a parameter. Since version 0.2.1 there is also increment_my_counter_by, decrement_my_counter and <tt>decrement_my_counter_by<tt>.

Fine grained control of storing and retrieving values from buckets can be controlled using the :condition and :else options.

storage_bucket :my_bucket,
               :condition => Proc.new{ |object, frivol_method, *args| ... },
               :else       => :your_method

For the above example, frivol execute the :condition proc and passes the instance of the current class, which method is being attempted (increment, increment_by, store, retrieve, etc.) and any arguments that may have been passed on to frivol. If the condition returns a truthy result, the frivol method is executed unimpeded, otherwise frivol moves on to :else. :else for the above example is a method on the instance, and that method must be able to receive the frivol method used (as a string) and any arguments passed to that method:

def your_method(frivol_method, *args)
  ...
end

The :condition and :else options can be specified as a proc, symbol, true or false. Frivol uses the storage_key method to create a base key for storage in Redis. The current implementation uses "#{self.class.name}-#{id}" so you’ll want to override that method if you have classes that don’t respond to id.

The frivolize method is similar to memoize except that it uses the backend to store the data which can thus be shared amongst multiple processes.

def long_running_result
  # do lots of hard work
end
frivolize :long_running_result

The frivolize also is also able to take options to make counters and expire.

def long_running_count
  # count something that takes a long time
end
frivolize :long_running_count, counter: true, expires_in: 10.minutes

These extensions allow the storing and retrieving of Time and ActiveSupport::TimeWithZone objects in Frivol. We now recommend that times are stored using #to_i, but the extensions are provided for legacy Frivol users. In order to use them you will need to have:

require 'frivol/extensions'

You can also create your own extensions to save other complex objects in Frivol. Your classes will need a #to_json(*a) to dump the object and a .json_create(o) class method to load the object. You will need to tell Frivol to allow the use of create extensions for your class with:

Frivol::Config.allow_json_create << Klass