Tokyo Cache Cow¶ ↑

Tokyo Cache Cow is MemCache protocol speaking cache server. It offers the ability to delete keys based on a substring as well as aggregate functions based on substrings.

Cache sweepers in rails do not currently operate with memcache because the memcache server itself does not support key matching removal operations. After reading www.igvita.com/2009/02/13/tokyo-cabinet-beyond-key-value-store/ and seeing the performance characteristics of that database, I thought I’d give it a go. Event Machine does the heavy lifting on the network end. Performance is currently comparable to memcached.

You’ll need the eventmachine gem installed. As well, you’ll have to install Tokyo Cabinet itself (available at tokyocabinet.sourceforge.net/index.html) and the Tokyo Cabinet Ruby bindings (available at tokyocabinet.sourceforge.net/rubypkg/)

Lets write four keys: other_key, test_key, test_key2 and test_key3.

>> Rails.cache.write('other_key', 'other_value')
=> true
>> Rails.cache.write('test_key', 'test_value')
=> true
>> Rails.cache.write('test_key2', 'test_value2')
=> true
>> Rails.cache.write('test_key3', 'test_value3')
=> true

Read back test_key and make sure life is still good.

>> Rails.cache.read('test_key2')
=> "test_value2"

But lets delete test_key2 for fun.

>> Rails.cache.delete('test_key2')
=> true

Confirm that test_key2 is really gone.

>> Rails.cache.read('test_key2')
=> nil

.. but our other keys (namely, test_key) are just fine, thank you.

>> Rails.cache.read('test_key')
=> "test_value"

.. lets nuke EVERYTHING with test_key in it though.

>> Rails.cache.delete_matched('test_key')
=> true

Now test_key and test_key3 are both nuked.

>> Rails.cache.read('test_key')
=> nil
>> Rails.cache.read('test_key3')
=> nil

But other_key is still peachy.

>> Rails.cache.read('other_key')
=> "other_value"

You can invoke aggregates by using the functions avg(), count(), sum(), min() and max().

>> Rails.cache.write('hello1', '1')
=> true
>> Rails.cache.write('hello2', '2')
=> true
>> Rails.cache.write('hello3', '3')
=> true
>> Rails.cache.read('avg(hello)')
=> "2.0"
>> Rails.cache.read('sum(hello)')
=> "6.0"

>> tokyo_cache_cow --help

Usage: tokyo_cache_cow [options]
Options:
    -p, --port[OPTIONAL]             Port (default: 11211)
    -a, --address[OPTIONAL]          Address (default: 0.0.0.0)
    -c, --class[OPTIONAL]            Cache provider class (default: TokyoCacheCow::Cache::TokyoCabinetMemcache)
    -r, --require[OPTIONAL]          require
    -f, --file[OPTIONAL]             File (default: /tmp/tcc-cache)
    -d, --daemonize[OPTIONAL]        Daemonize (default: false)
    -P, --pid[OPTIONAL]              Pid file (default: /tmp/tcc.pid)
    -m, --matcher[OPTIONAL]          Special flag for doing matched deletes (not enabled by default)
    -h, --help                       Show this help message.

Any standard memcache client will do, however, there is a special initialize for Rails to enable delete_matched functionality within the built-in memcache client there. To install this into rails:

script/plugin install git://github.com/joshbuddy/tokyo-cache-cow.git

Right now there is no compression on disk. Things could get big, but Tokyo Cabinet does support various compression schemes, so exposing that to the runner should be trivial. Potentially performance could degrade after time, but I have yet to seriously investigate if thats the case.

Feel the moo.