python-pickle.rb

Description

python-pickle is a modern Ruby implementation of the Python Pickle serialization format.

Features

Supports deserializing Python Pickle data into Ruby objects.
Optionally supports only parsing Python Pickle data streams for debugging purposes.
Supports Pickle protocol 0, protocol 1, protocol 2, protocol 3, protocol 4, and protocol 5.
- Can parse both Python 2 and Python 3 Pickled data.
Supports deserializing Python None, True, False, int, str, tuple, set, list, bytearray, and other objects.
Supports mapping Python extension codes to Ruby classes.
Supports mapping Python functions to Ruby methods.
Supports mapping Python classes to Ruby classes.
Supports out-of-band buffers.

TODO

Add support for writing Python Pickle data.
Add support for serializing Ruby objects to Python Pickle data.

Requirements

Ruby >= 3.0.0

Install

$ gem install python-pickle

gemspec

gem.add_dependency 'python-pickle', '~> 1.0'

Gemfile

gem 'python-pickle', '~> 1.0'

Examples

Load a Python Pickle string:

Python::Pickle.load("\x80\x04\x95\x10\x00\x00\x00\x00\x00\x00\x00}\x94\x8C\x03foo\x94\x8C\x03bar\x94s.")
# => {"foo"=>"bar"}

Load a Python Pickle stream:

Python::Pickle.load(io)
# => ...

Loading a Python Pickle file:

Python::Pickle.load_file('dict.pkl')
# => {"foo"=>"bar"}

Loading Python bytearray objects:

pickle = "\x80\x05\x95\x0E\x00\x00\x00\x00\x00\x00\x00\x96\x03\x00\x00\x00\x00\x00\x00\x00ABC\x94."

Python::Pickle.load(pickle)
# => #<Python::Pickle::ByteArray: "ABC">

Loading Python objects:

pickle = "\x80\x04\x95,\x00\x00\x00\x00\x00\x00\x00\x8C\b__main__\x94\x8C\aMyClass\x94\x93\x94)\x81\x94}\x94(\x8C\x01x\x94KA\x8C\x01y\x94KBub."

Python::Pickle.load(pickle)
# => 
# #<Python::Pickle::PyObject:0x00007f48c9ba7598                   
#  @attributes={"y"=>66, "x"=>65},                                
#  @init_args=[],                                                 
#  @init_kwargs={},                                               
#  @py_class=#<Python::Pickle::PyClass: __main__.MyClass>>

Mapping Python classes to Ruby classes:

class MyClass

  attr_reader :x
  attr_reader :y

  def __setstate__(attributes)
    @x = attributes['x']
    @y = attributes['y']
  end

end

pickle = "\x80\x04\x95,\x00\x00\x00\x00\x00\x00\x00\x8C\b__main__\x94\x8C\aMyClass\x94\x93\x94)\x81\x94}\x94(\x8C\x01x\x94KA\x8C\x01y\x94KBub."

Python::Pickle.load(pickle, constants: {
  '__main__' => {
    'MyClass' => MyClass
  }
})
# => #<MyClass:0x00007f48c5c28980 @x=65, @y=66>

Parsing and inspecting a pickle file:

require 'python/pickle'

Python::Pickle.parse(File.open('dict.pkl'))
# => 
# [#<Python::Pickle::Instructions::Mark: MARK>,
#  #<Python::Pickle::Instructions::Dict: DICT>,
#  #<Python::Pickle::Instructions::Put: PUT 0>,
#  #<Python::Pickle::Instructions::String: STRING "foo">,
#  #<Python::Pickle::Instructions::Put: PUT 1>,
#  #<Python::Pickle::Instructions::String: STRING "bar">,
#  #<Python::Pickle::Instructions::Put: PUT 2>,
#  #<Python::Pickle::Instructions::SetItem: SETITEM>,
#  #<Python::Pickle::Instructions::Stop: STOP>]

Copyright

See {file:LICENSE.txt} for details.

python-pickle

Development