Store large objects in memcache or others by slicing them.
- uses read_multi for fast access
- returns nil if one slice is missing
- low performance overhead, only uses single read/write if data is below 1MB
Install
gem install large_object_storeUsage
Rails.cache.write("a", "a"*10_000_000) # => false -> oops too large
store = LargeObjectStore.wrap(Rails.cache)
store.write("a", "a"*10_000_000) # => true -> always!
store.read("a").size # => 10_000_000 using multi_get
store.read("b") # => nil
store.fetch("a"){ "something" } # => "something" executes block on miss
store.write("a" * 10_000_000, compress: true) # compress when greater than 16k
store.write("a" * 1000, compress: true, compress_limit: 100) # compress when greater than 100
store.write("a" * 1000, raw: true) # store as string to avoid marshaling overheadzstd
zstd compression, a modern improvement over the venerable zlib compression algorithm, is supported by passing the zstd flag when writing items:
store.write("a" * 10_000_000, compress: true, zstd: true)
For backwards compatibility and to enable safe roll-out of the change in working systems, the zstd flag defaults to false.
zstd decompression is used when the zstd magic number is detected at the beginning of compressed data, so zstd: true does not need to be passed when reading/fetching items.
Author
Ana Martinez
acemacu@gmail.com
Michael Grosser
michael@grosser.it
License: MIT
