zstandard-ruby

• Installation
• Usage
• Advanced Usage
• Configuration
• Docs
• Examples for installing libzstd
• Contributing
• License

This gem implements FFI based bindings to the Zstandard compression library libzstd. The Zstandard compression algorithm shines because it compresses data with the same or better ratio as Zlib but does this (much) faster, depending on the input. For the majority of cases it's faster and better then Zlib. It is tested to work with MRI from 1.9.3 to 2.6.x and with JRuby from >= 1.7.24, including 9.2.x.

It does not ship the actual libzstd library but expects some version to be present on your system.

This gem is activly maintained. The tests are updated regularly, to keep up with the latest zstd version.

Make sure you have libzstd installed on your system. In doubt, have a look at the examples for installing libzstd.

Add this line to your application's Gemfile:

gem "zstandard"

And then execute:

bundle

Or install it yourself as:

gem install zstandard

The gem provides an API which aims compatibility the with zlib gem. There are two module methods

• deflate(string, level)
• inflate(compressed_string)

The only difference between this and the zlib gem is the interpretation of the compression level. For zlib, this is a value between 1..9, whereas for Zstandard it's between 1..22.

For most use cases, you should try to keep the compression level (very) low for Zstandard, because often compression time increases without significant better compression ratios. If in doubt, do not specify a compression level at all, which will use the default compression level.

require "zstandard"

compressed_string = Zstandard.deflate(string)
decompressed_string = Zstandard.inflate(compressed_string)

This is not intended to be used by regular users.

Besides the high level API which targets compatibility with the well known zlib gem there are two additional layers you can interact with. There is a low-level API which tries to cover differences between various libzstd version, e.g. different frame header formats. You should only use this if you know, what you are doing.

require "zstandard"

compressed_string = Zstandard::API.simple_compress(string)
decompressed_string = Zstandard::API.simple_decompress(compressed_string)

The most low-level bindings are exposed via Zstandard::FFIBindings. If there is any reason for this, you can do the following.

require "zstandard"

zstd_library_version = Zstandard::FFIBindings.zstd_version_number

This gem can be configured by setting various environment variables. Please be carefull if you decide to change/overwrite any of these. The default values are carefully choosen and there should be no need to alter one of these for regular use cases.

If you have libzstd installed in some unusual location or if you want to explictly tell, which library to use, you can set ZSTANDARD_LIBRARY to the path of the library you want to use. This can be handy for example if you have the latest version compiled in /usr/local/lib, but your system has an old version in /usr/lib.

ZSTANDARD_LIBRARY=/usr/local/lib/libzstd.so bundle exec rspec

This specifies the maximum (decompressed) size of a string for which the simple decompression approach should be used. In order minimise memory consumption, if the expected decompressed size exceeds this limit, streaming decompression is used.

For streaming decompression, this specifies the size of the decompression bufffer.

Yard generated docs can be found at http://www.rubydoc.info/github/msievers/zstandard-ruby.

• Debian
• Fedora
• FreeBSD
• Mac
• NetBSD
• Ubuntu

The package is only included in sid, the unstable Debian version. There are guides describing how to install unstable packages into a stable Debian, for example at Linuxaria or serverfault.com.

# run as root

apt-get install zstd

sudo dnf install libzstd

# run as root

portsnap fetch && portsnap extract

cd /usr/ports/archivers/zstd
make install

brew install zstd

# run as root

# the following assumes you are running a x86_64 system with NetBSD 7.0.x

export PATH="/usr/pkg/sbin:$PATH"
export PKG_PATH="ftp://ftp.netbsd.org/pub/pkgsrc/packages/NetBSD/x86_64/7.0_current/All/"

pkg_add zstd

sudo apt-get install zstd

Bug reports and pull requests are welcome on GitHub at https://github.com/msievers/zstandard-ruby.

The gem is available as open source under the terms of the MIT License.