Shale
Shale is a Ruby object mapper and serializer for JSON, YAML, TOML, CSV and XML. It allows you to parse JSON, YAML, TOML, CSV and XML data and convert it into Ruby data structures, as well as serialize data structures into JSON, YAML, TOML, CSV or XML.
Documentation with interactive examples is available at Shale website
Features
- Convert JSON, YAML, TOML, CSV and XML to Ruby data model
- Convert Ruby data model to JSON, YAML, TOML, CSV and XML
- Generate JSON and XML Schema from Ruby models
- Compile JSON and XML Schema into Ruby models
- Out of the box support for JSON, YAML, Tomlib, toml-rb, CSV, Nokogiri, REXML and Ox parsers
- Support for custom adapters
Installation
Shale supports Ruby (MRI) 3.0+
Add this line to your application's Gemfile:
gem 'shale'
And then execute:
$ bundle install
Or install it yourself as:
$ gem install shale
Contents
- Simple use case
- Creating objects
- Converting JSON to object
- Converting object to JSON
- Converting YAML to object
- Converting object to YAML
- Converting TOML to object
- Converting object to TOML
- Converting Hash to object
- Converting object to Hash
- Converting XML to object
- Converting object to XML
- Converting CSV to object
- Converting object to CSV
- Converting collections
- Mapping JSON keys to object attributes
- Mapping YAML keys to object attributes
- Mapping TOML keys to object attributes
- Mapping CSV columns to object attributes
- Mapping Hash keys to object attributes
- Mapping XML elements and attributes to object attributes
- Using XML namespaces
- Rendering nil values
- Using methods to extract and generate data
- Delegating fields to child attributes
- Additional options
- Overriding attribute methods
- Using custom models
- Supported types
- Writing your own type
- Adapters
- Generating JSON Schema
- Compiling JSON Schema into Shale model
- Generating XML Schema
- Compiling XML Schema into Shale model
Usage
Simple use case
require 'shale'
class Address < Shale::Mapper
attribute :city, :string
attribute :street, :string
attribute :zip, :string
end
class Person < Shale::Mapper
attribute :first_name, :string
attribute :last_name, :string
attribute :age, :integer
attribute :married, :boolean, default: -> { false }
attribute :hobbies, :string, collection: true
attribute :address, Address
end
-
default: -> { 'value' }
- add a default value to attribute (it must be a proc that returns value) -
collection: true
- indicates that a attribute is a collection
Creating objects
person = Person.new(
first_name: 'John',
last_name: 'Doe',
age: 50,
hobbies: ['Singing', 'Dancing'],
address: Address.new(city: 'London', street: 'Oxford Street', zip: 'E1 6AN'),
)
Converting JSON to object
person = Person.from_json(<<~DATA)
{
"first_name": "John",
"last_name": "Doe",
"age": 50,
"married": false,
"hobbies": ["Singing", "Dancing"],
"address": {
"city": "London",
"street": "Oxford Street",
"zip": "E1 6AN"
}
}
DATA
# =>
#
# #<Person:0x00007f9bc3086d60
# @address=
# #<Address:0x00007f9bc3086748
# @city="London",
# @street="Oxford Street",
# @zip="E1 6AN">,
# @age=50,
# @first_name="John",
# @hobbies=["Singing", "Dancing"],
# @last_name="Doe",
# @married=false>
Converting object to JSON
person.to_json
# =>
#
# {
# "first_name": "John",
# "last_name": "Doe",
# "age": 50,
# "married": false,
# "hobbies": ["Singing", "Dancing"],
# "address": {
# "city": "London",
# "street": "Oxford Street",
# "zip": "E1 6AN"
# }
# }
Converting YAML to object
person = Person.from_yaml(<<~DATA)
first_name: John
last_name: Doe
age: 50
married: false
hobbies:
- Singing
- Dancing
address:
city: London
street: Oxford Street
zip: E1 6AN
DATA
Converting object to YAML
person.to_yaml
# =>
#
# ---
# first_name: John
# last_name: Doe
# age: 50
# married: false
# hobbies:
# - Singing
# - Dancing
# address:
# city: London
# street: Oxford Street
# zip: E1 6AN
Converting TOML to object
To use TOML with Shale you have to set adapter you want to use. It comes with adapters for Tomlib and toml-rb. For details see Adapters section.
To set it, first make sure Tomlib gem is installed:
$ gem install tomlib
then setup adapter:
require 'sahle/adapter/tomlib'
Shale.toml_adapter = Shale::Adapter::Tomlib
# Alternatively if you'd like to use toml-rb, use:
require 'shale/adapter/toml_rb'
Shale.toml_adapter = Shale::Adapter::TomlRB
Now you can use TOML with Shale:
person = Person.from_toml(<<~DATA)
first_name = "John"
last_name = "Doe"
age = 50
married = false
hobbies = ["Singing", "Dancing"]
[address]
city = "London"
street = "Oxford Street"
zip = "E1 6AN"
DATA
Converting object to TOML
person.to_toml
# =>
#
# first_name = "John"
# last_name = "Doe"
# age = 50
# married = false
# hobbies = [ "Singing", "Dancing" ]
#
# [address]
# city = "London"
# street = "Oxford Street"
# zip = "E1 6AN"
Converting Hash to object
person = Person.from_hash(
'first_name' => 'John',
'last_name' => 'Doe',
'age' => 50,
'married' => false,
'hobbies' => ['Singing', 'Dancing'],
'address' => {
'city'=>'London',
'street'=>'Oxford Street',
'zip'=>'E1 6AN'
},
)
Converting object to Hash
person.to_hash
# =>
#
# {
# "first_name"=>"John",
# "last_name"=>"Doe",
# "age"=>50,
# "married"=>false,
# "hobbies"=>["Singing", "Dancing"],
# "address"=>{"city"=>"London", "street"=>"Oxford Street", "zip"=>"E1 6AN"}
# }
Converting XML to object
To use XML with Shale you have to set adapter you want to use. Shale comes with adapters for REXML, Nokogiri and OX parsers. For details see Adapters section.
require 'shale/adapter/rexml'
Shale.xml_adapter = Shale::Adapter::REXML
Now you can use XML with Shale:
person = Person.from_xml(<<~DATA)
<person>
<first_name>John</first_name>
<last_name>Doe</last_name>
<age>50</age>
<married>false</married>
<hobbies>Singing</hobbies>
<hobbies>Dancing</hobbies>
<address>
<city>London</city>
<street>Oxford Street</street>
<zip>E1 6AN</zip>
</address>
</person>
DATA
Converting object to XML
person.to_xml
# =>
#
# <person>
# <first_name>John</first_name>
# <last_name>Doe</last_name>
# <age>50</age>
# <married>false</married>
# <hobbies>Singing</hobbies>
# <hobbies>Dancing</hobbies>
# <address>
# <city>London</city>
# <street>Oxford Street</street>
# <zip>E1 6AN</zip>
# </address>
# </person>
Converting CSV to object
To use CSV with Shale you have to set adapter. Shale comes with adapter for csv. For details see Adapters section.
To set it, first make sure CSV gem is installed:
$ gem install csv
then setup adapter:
require 'shale/adapter/csv'
Shale.csv_adapter = Shale::Adapter::CSV
Now you can use CSV with Shale.
CSV represents a flat data structure, so you can't map properties to complex types directly, but you can use methods to map properties to complex types (see Using methods to extract and generate data section).
.from_csv
method allways returns an array of records.
people = Person.from_csv(<<~DATA)
John,Doe,50,false
DATA
Converting object to CSV
people[0].to_csv # or Person.to_csv(people) if you want to convert a collection
# =>
#
# John,Doe,50,false
Converting collections
Shale allows converting collections for formats that support it (JSON, YAML and CSV). To convert Ruby array to JSON:
person1 = Person.new(name: 'John Doe')
person2 = Person.new(name: 'Joe Sixpack')
Person.to_json([person1, person2], pretty: true)
# or Person.to_yaml([person1, person2])
# or Person.to_csv([person1, person2])
# =>
#
# [
# { "name": "John Doe" },
# { "name": "Joe Sixpack" }
# ]
To convert JSON array to Ruby:
Person.from_json(<<~JSON)
[
{ "name": "John Doe" },
{ "name": "Joe Sixpack" }
]
JSON
# =>
#
# [
# #<Person:0x00000001033dbce8 @name="John Doe">,
# #<Person:0x00000001033db4c8 @name="Joe Sixpack">
# ]
Mapping JSON keys to object attributes
By default keys are named the same as attributes. To use custom keys use:
⚠️ Declaring custom mapping removes default mapping for given format!
class Person < Shale::Mapper
attribute :first_name, :string
attribute :last_name, :string
json do
map 'firstName', to: :first_name
map 'lastName', to: :last_name
end
end
Mapping YAML keys to object attributes
class Person < Shale::Mapper
attribute :first_name, :string
attribute :last_name, :string
yaml do
map 'firstName', to: :first_name
map 'lastName', to: :last_name
end
end
Mapping TOML keys to object attributes
class Person < Shale::Mapper
attribute :first_name, :string
attribute :last_name, :string
toml do
map 'firstName', to: :first_name
map 'lastName', to: :last_name
end
end
Mapping CSV columns to object attributes
For CSV the order of mapping matters, the first argument in the map
method is only
used as a label in header row. So, in the example below the first column will be mapped
to :first_name
attribute and the second column to :last_name
.
class Person < Shale::Mapper
attribute :first_name, :string
attribute :last_name, :string
csv do
map 'firstName', to: :first_name
map 'lastName', to: :last_name
end
end
Mapping Hash keys to object attributes
class Person < Shale::Mapper
attribute :first_name, :string
attribute :last_name, :string
hsh do
map 'firstName', to: :first_name
map 'lastName', to: :last_name
end
end
Mapping XML elements and attributes to object attributes
XML is more complicated format than JSON or YAML. To map elements, attributes and content use:
class Address < Shale::Mapper
attribute :street, :string
attribute :city, :string
attribute :zip, :string
xml do
map_content to: :street
map_element 'City', to: :city
map_element 'ZIP', to: :zip
end
end
class Person < Shale::Mapper
attribute :first_name, :string
attribute :last_name, :string
attribute :age, :integer
attribute :hobbies, :string, collection: true
attribute :address, Address
xml do
root 'Person'
map_attribute 'age', to: :age
map_element 'FirstName', to: :first_name
map_element 'LastName', to: :last_name
map_element 'Hobby', to: :hobbies
map_element 'Address', to: :address
end
end
person = Person.from_xml(<<~DATA)
<Person age="50">
<FirstName>John</FirstName>
<LastName>Doe</LastName>
<Hobby>Singing</Hobby>
<Hobby>Dancing</Hobby>
<Address>
Oxford Street
<City>London</City>
<ZIP>E1 6AN</ZIP>
</Address>
</person>
DATA
-
root
- name of the root element -
map_element
- map content of element to attribute -
map_attribute
- map element's attribute to attribute -
map_content
- map first text node to attribute
You can use cdata: true
option on map_element
and map_content
to handle CDATA nodes:
class Address < Shale::Mapper
attribute :content, :string
xml do
map_content to: :content, cdata: true
end
end
class Person < Shale::Mapper
attribute :first_name, :string
attribute :address, Address
xml do
root 'Person'
map_element 'FirstName', to: :first_name, cdata: true
map_element 'Address', to: :address
end
end
person = Person.from_xml(<<~DATA)
<Person>
<FirstName><![CDATA[John]]></FirstName>
<Address><![CDATA[Oxford Street]]></Address>
</person>
DATA
Using XML namespaces
To map namespaced elements and attributes use namespace
and prefix
properties on
map_element
and map_attribute
class Person < Shale::Mapper
attribute :first_name, :string
attribute :last_name, :string
attribute :age, :integer
xml do
root 'person'
map_element 'first_name', to: :first_name, namespace: 'http://ns1.com', prefix: 'ns1'
map_element 'last_name', to: :last_name, namespace: 'http://ns2.com', prefix: 'ns2'
map_attribute 'age', to: :age, namespace: 'http://ns2.com', prefix: 'ns2'
end
end
person = Person.from_xml(<<~DATA)
<person xmlns:ns1="http://ns1.com" xmlns:ns2="http://ns2.com" ns2:age="50">
<ns1:first_name>John</ns1:first_name>
<ns2:last_name>Doe</ns2:last_name>
</person>
DATA
To define default namespace for all elements use namespace
declaration
(this will define namespace on elements only, if you want to define namespace on an attribute
explicitly declare it on map_attribute
).
class Person < Shale::Mapper
attribute :first_name, :string
attribute :middle_name, :string
attribute :last_name, :string
attribute :age, :integer
attribute :hobby, :string
xml do
root 'person'
namespace 'http://ns1.com', 'ns1'
map_element 'first_name', to: :first_name
# undeclare namespace on 'middle_name' element
map_element 'middle_name', to: :middle_name, namespace: nil, prefix: nil
# overwrite default namespace
map_element 'last_name', to: :last_name, namespace: 'http://ns2.com', prefix: 'ns2'
map_attribute 'age', to: :age
map_attribute 'hobby', to: :hobby, namespace: 'http://ns1.com', prefix: 'ns1'
end
end
person = Person.from_xml(<<~DATA)
<ns1:person xmlns:ns1="http://ns1.com" xmlns:ns2="http://ns2.com" age="50" ns1:hobby="running">
<ns1:first_name>John</ns1:first_name>
<middle_name>Joe</middle_name>
<ns2:last_name>Doe</ns2:last_name>
</ns1:person>
DATA
Rendering nil values
For JSON, YAML, TOML and XML by default, elements with nil
value are not rendered.
You can change this behavior by using render_nil: true
on a mapping.
For CSV the default is to render nil
elements.
class Person < Shale::Mapper
attribute :first_name, :string
attribute :last_name, :string
attribute :age, :integer
json do
map 'first_name', to: :first_name, render_nil: true
map 'last_name', to: :last_name, render_nil: false
map 'age', to: :age, render_nil: true
end
xml do
root 'person'
map_element 'first_name', to: :first_name, render_nil: true
map_element 'last_name', to: :last_name, render_nil: false
map_attribute 'age', to: :age, render_nil: true
end
end
person = Person.new(first_name: nil, last_name: nil, age: nil)
puts person.to_json(pretty: true)
# =>
#
# {
# "first_name": null,
# "age": "null"
# }
puts person.to_xml(pretty: true)
# =>
#
# <person age="">
# <first_name/>
# </person>
If you want to change how nil values are rendered for all mappings you can use render_nil
method:
class Base < Shale::Mapper
json do
# change render_nil default for all JSON mappings inheriting from Base class
render_nil true
end
end
class Person < Base
attribute :first_name, :string
attribute :last_name, :string
attribute :age, :integer
json do
# override default from Base class
render_nil false
map 'first_name', to: :first_name
map 'last_name', to: :last_name
map 'age', to: :age, render_nil: true # override default
end
end
⚠️ The default affects only the mappings declared after setting the default value e.g.
class Person < Base
attribute :first_name, :string
attribute :last_name, :string
json do
render_nil false
map 'first_name', to: :first_name # render_nil will be false for this mapping
render_nil true
map 'last_name', to: :last_name # render_nil will be true for this mapping
end
end
Using methods to extract and generate data
If you need full controll over extracting and generating data from/to document, you can use methods to do so:
class Person < Shale::Mapper
attribute :hobbies, :string, collection: true
attribute :street, :string
attribute :city, :string
json do
map 'hobbies', using: { from: :hobbies_from_json, to: :hobbies_to_json }
map 'address', using: { from: :address_from_json, to: :address_to_json }
end
xml do
root 'Person'
map_attribute 'hobbies', using: { from: :hobbies_from_xml, to: :hobbies_to_xml }
map_element 'Address', using: { from: :address_from_xml, to: :address_to_xml }
end
def hobbies_from_json(model, value)
model.hobbies = value.split(',').map(&:strip)
end
def hobbies_to_json(model, doc)
doc['hobbies'] = model.hobbies.join(', ')
end
def address_from_json(model, value)
model.street = value['street']
model.city = value['city']
end
def address_to_json(model, doc)
doc['address'] = { 'street' => model.street, 'city' => model.city }
end
def hobbies_from_xml(model, value)
model.hobbies = value.split(',').map(&:strip)
end
def hobbies_to_xml(model, element, doc)
doc.add_attribute(element, 'hobbies', model.hobbies.join(', '))
end
def address_from_xml(model, node)
model.street = node.children.find { |e| e.name == 'Street' }.text
model.city = node.children.find { |e| e.name == 'City' }.text
end
def address_to_xml(model, parent, doc)
street_element = doc.create_element('Street')
doc.add_text(street_element, model.street.to_s)
city_element = doc.create_element('City')
doc.add_text(city_element, model.city.to_s)
address_element = doc.create_element('Address')
doc.add_element(address_element, street_element)
doc.add_element(address_element, city_element)
doc.add_element(parent, address_element)
end
end
person = Person.from_json(<<~DATA)
{
"hobbies": "Singing, Dancing, Running",
"address": {
"street": "Oxford Street",
"city": "London"
}
}
DATA
person = Person.from_xml(<<~DATA)
<Person hobbies="Singing, Dancing, Running">
<Address>
<Street>Oxford Street</Street>
<City>London</City>
</Address>
</Person>
DATA
# =>
#
# #<Person:0x00007f9bc3086d60
# @hobbies=["Singing", "Dancing", "Running"],
# @street="Oxford Street",
# @city="London">
You can also pass a context
object that will be available in extractor/generator methods:
class Person < Shale::Mapper
attribute :password, :string
json do
map 'password', using: { from: :password_from_json, to: :password_to_json }
end
def password_from_json(model, value, context)
if context.admin?
model.password = value
else
model.password = '*****'
end
end
def password_to_json(model, doc, context)
if context.admin?
doc['password'] = model.password
else
doc['password'] = '*****'
end
end
end
Person.new(password: 'secret').to_json(context: current_user)
If you want to work on multiple elements at a time you can group them using group
block:
class Person < Shale::Mapper
attribute :name, :string
json do
group from: :name_from_json, to: :name_to_json do
map 'first_name'
map 'last_name'
end
end
xml do
group from: :name_from_xml, to: :name_to_xml do
map_content
map_element 'first_name'
map_attribute 'last_name'
end
end
def name_from_json(model, value)
model.name = "#{value['first_name']} #{value['last_name']}"
end
def name_to_json(model, doc)
doc['first_name'] = model.name.split(' ')[0]
doc['last_name'] = model.name.split(' ')[1]
end
def name_from_xml(model, value)
# value => { content: ..., attributes: {}, elements: {} }
end
def name_to_xml(model, element, doc)
# ...
end
end
Person.from_json(<<~DATA)
{
"first_name": "John",
"last_name": "Doe"
}
DATA
# => #<Person:0x00007f9bc3086d60 @name="John Doe">
Delegating fields to child attributes
To delegate fields to child complex types you can use receiver: :child
declaration:
class Address < Shale::Mapper
attribute :city, :string
attribute :street, :string
end
class Person < Shale::Mapper
attribute :name, :string
attribute :address, Address
json do
map 'name', to: :name
map 'city', to: :city, receiver: :address
map 'street', to: :street, receiver: :address
end
end
person = Person.from_json(<<~DATA)
{
"name": "John Doe",
"city": "London",
"street": "Oxford Street"
}
DATA
# =>
#
# #<Person:0x00007f9bc3086d60
# @name="John Doe",
# @address=#<Address:0x0000000102cbd218 @city="London", @street="Oxford Street">>
Additional options
You can control which attributes to render and parse by
using only: []
and except: []
parameters.
# e.g. if you have this model graph:
person = Person.new(
first_name: 'John'
last_name: 'Doe',
address: Address.new(city: 'London', street: 'Oxford Street')
)
# if you want to render only `first_name` and `address.city` do:
person.to_json(only: [:first_name, address: [:city]], pretty: true)
# =>
#
# {
# "first_name": "John",
# "address": {
# "city": "London"
# }
# }
# and if you don't need an address you can do:
person.to_json(except: [:address], pretty: true)
# =>
#
# {
# "first_name": "John",
# "last_name": "Doe"
# }
It works the same for parsing:
# e.g. if you want to parse only `address.city` do:
Person.from_json(doc, only: [address: [:city]])
# =>
#
# #<Person:0x0000000113d7a488
# @first_name=nil,
# @last_name=nil,
# @address=#<Address:0x0000000113d7a140 @street=nil, @city="London">>
# and if you don't need an `address`:
Person.from_json(doc, except: [:address])
# =>
#
# #<Person:0x0000000113d7a488
# @first_name="John",
# @last_name="Doe",
# @address=nil>
If you need formatted output you can pass pretty: true
parameter to #to_json
and #to_xml
person.to_json(pretty: true)
# =>
#
# {
# "name": "John Doe",
# "address": {
# "city": "London"
# }
# }
You can also add an XML declaration by passing declaration: true
and encoding: true
or if you want to spcify version: declaration: '1.1'
and encoding: 'ASCII'
to #to_xml
person.to_xml(pretty: true, declaration: true, encoding: true)
# =>
#
# <?xml version="1.0" encoding="UTF-8"?>
# <Person>
# <Address city="London"/>
# </Person>
For CSV you can pass headers: true
to indicate that the first row contains column
names and shouldn't be included in the returned collection. It also accepts all the options that
CSV parser accepts.
class Person
attribute :first_name, :string
attribute :last_name, :string
end
people = Person.from_csv(<<~DATA, headers: true, col_sep: '|')
first_name|last_name
John|Doe
James|Sixpack
DATA
# =>
#
# [
# #<Person:0x0000000113d7a488 @first_name="John", @last_name="Doe">,
# #<Person:0x0000000113d7a488 @first_name="James", @last_name="Sixpack">
# ]
Person.to_csv(people, headers: true, col_sep: '|')
# =>
#
# first_name|last_name
# John|Doe
# James|Sixpack
Most adapters accept options specific to them. Eg. if you want to be able to work with NaN values in JSON:
class Person
attribute :age, :float
end
person = Person.from_json('{"age": NaN}', allow_nan: true)
# =>
#
# #<Person:0x0000000113d7a488 @age=Float::NAN>
Person.to_json(person, allow_nan: true)
# =>
#
# {
# "age": NaN
# }
Overriding attribute methods
It's possible to override an attribute method to change its output:
class Person < Shale::Mapper
attribute :gender, :string
def gender
if super == 'm'
'male'
else
'female'
end
end
end
puts Person.from_json('{"gender":"m"}')
# =>
#
# #<Person:0x00007f9bc3086d60
# @gender="male">
Be conscious that the original attribute value will be lost after its transformation though:
puts User.from_json('{"gender":"m"}').to_json
# => {"gender":"male"}
It'll no longer return gender m
.
Using custom models
By default Shale combines mapper and model into one class. If you want to use your own classes
as models you can do it by using model
directive on the mapper:
class Address
attr_accessor :street, :city
end
class Person
attr_accessor :first_name, :last_name, :address
end
class AddressMapper < Shale::Mapper
model Address
attribute :street, :string
attribute :city, :string
end
class PersonMapper < Shale::Mapper
model Person
attribute :first_name, :string
attribute :last_name, :string
attribute :address, AddressMapper
end
person = PersonMapper.from_json(<<~DATA)
{
"first_name": "John",
"last_name": "Doe",
"address": {
"street": "Oxford Street",
"city": "London"
}
}
DATA
# =>
#
# #<Person:0x0000000113d7a488
# @first_name="John",
# @last_name="Doe",
# @address=#<Address:0x0000000113d7a140 @street="Oxford Street", @city="London">>
PersonMapper.to_json(person, pretty: true)
# =>
#
# {
# "first_name": "John",
# "last_name": "Doe",
# "address": {
# "street": "Oxford Street",
# "city": "London"
# }
# }
Supported types
Shale supports these types out of the box:
-
:boolean
(Shale::Type::Boolean
) -
:date
(Shale::Type::Date
) -
:float
(Shale::Type::Float
) -
:integer
(Shale::Type::Integer
) -
:string
(Shale::Type::String
) -
:time
(Shale::Type::Time
)
The symbol type alias and the type class are interchangeable:
class Person < Shale::Mapper
attribute :age, Shale::Type::Integer
# attribute :age, :integer
end
Writing your own type
To add your own type extend it from Shale::Type::Value
and implement .cast
class method.
require 'shale/type/value'
class MyIntegerType < Shale::Type::Value
def self.cast(value)
value.to_i
end
end
Then you can use it in your model:
class Person < Shale::Mapper
attribute :age, MyIntegerType
end
You can also register your own type with a symbol alias if you intend to use it often.
require 'shale/type'
Shale::Type.register(:my_integer, MyIntegerType)
Then you can use it like this:
class Person < Shale::Mapper
attribute :age, :my_integer
end
Adapters
Shale uses adapters for parsing and generating documents. By default Ruby's standard JSON and YAML parsers are used for handling JSON and YAML documents.
You can change it by providing your own adapter. For JSON, YAML, TOML and CSV adapter must
implement .load
and .dump
class methods.
require 'shale'
require 'multi_json'
Shale.json_adapter = MultiJson
Shale.yaml_adapter = MyYamlAdapter
To handle TOML documents you have to set TOML adapter. Out of the box Tomlib
is supported.
Shale also provides adapter for toml-rb
parser:
require 'shale'
# if you want to use Tomlib
require 'tomlib'
Shale.toml_adapter = Tomlib
# if you want to use toml-rb
require 'shale/adapter/toml_rb'
Shale.toml_adapter = Shale::Adapter::TomlRB
To handle CSV documents you have to set CSV adapter. Shale provides adapter for csv
parser:
require 'shale'
require 'shale/adapter/csv'
Shale.csv_adapter = Shale::Adapter::CSV
To handle XML documents you have to explicitly set XML adapter. Shale provides adapters for most popular Ruby XML parsers:
⚠️ Ox doesn't support XML namespaces
require 'shale'
# if you want to use REXML:
require 'shale/adapter/rexml'
Shale.xml_adapter = Shale::Adapter::REXML
# if you want to use Nokogiri:
require 'shale/adapter/nokogiri'
Shale.xml_adapter = Shale::Adapter::Nokogiri
# or if you want to use Ox:
require 'shale/adapter/ox'
Shale.xml_adapter = Shale::Adapter::Ox
Generating JSON Schema
⚠️ Only Draft 2020-12 JSON Schema is supported
To generate JSON Schema from your Shale data model use:
require 'shale/schema'
Shale::Schema.to_json(
Person,
id: 'http://foo.bar/schema/person',
description: 'My description',
pretty: true
)
# =>
#
# {
# "$schema": "https://json-schema.org/draft/2020-12/schema",
# "$id": "http://foo.bar/schema/person",
# "description": "My description",
# "$ref": "#/$defs/Person",
# "$defs": {
# "Address": {
# "type": [
# "object",
# "null"
# ],
# "properties": {
# "city": {
# "type": [
# "string",
# "null"
# ]
# }
# }
# },
# "Person": {
# "type": "object",
# "properties": {
# "name": {
# "type": [
# "string",
# "null"
# ]
# },
# "address": {
# "$ref": "#/$defs/Address"
# }
# }
# }
# }
# }
You can also use a command line tool to do it:
$ shaleb -i data_model.rb -r Person -p
If you want to convert your own types to JSON Schema types use:
require 'shale'
require 'shale/schema'
class MyEmailType < Shale::Type::Value
...
end
class MyEmailJSONType < Shale::Schema::JSONGenerator::Base
def as_type
{ 'type' => 'string', 'format' => 'email' }
end
end
Shale::Schema::JSONGenerator.register_json_type(MyEmailType, MyEmailJSONType)
To add validation keywords to the schema, you can use a custom model and do this:
require 'shale/schema'
class PersonMapper < Shale::Mapper
model Person
attribute :first_name, :string
attribute :last_name, :string
attribute :address, :string
attribute :age, :integer
json do
properties max_properties: 5, additional_properties: false
map "first_name", to: :first_name, schema: { required: true }
map "last_name", to: :last_name, schema: { required: true }
map "address", to: :address, schema: { max_length: 128, description: "Street, home number, city and country" }
map "age", to: :age, schema: { minimum: 1, maximum: 150, description: "Person age" }
end
end
Shale::Schema.to_json(
PersonMapper,
pretty: true
)
# =>
#
# {
# "$schema": "https://json-schema.org/draft/2020-12/schema",
# "description": "My description",
# "$ref": "#/$defs/Person",
# "$defs": {
# "Person": {
# "type": "object",
# "properties": {
# "first_name": {
# "type": "string"
# },
# "last_name": {
# "type": "string"
# },
# "address": {
# "type": [
# "string",
# "null"
# ],
# "maxLength": 128,
# "description": "Street, home number, city and country"
# },
# "age": {
# "type": [
# "integer",
# "null"
# ],
# "minimum": 1,
# "maximum": 150,
# "description": "Person age"
# }
# },
# "required": [
# "first_name",
# "last_name"
# ],
# "maxProperties": 5,
# "additionalProperties": false
# }
# }
# }
Validation keywords are supported for all types, only the global enum
and const
types are not supported.
Compiling JSON Schema into Shale model
⚠️ Only Draft 2020-12 JSON Schema is supported
To generate Shale data model from JSON Schema use Shale::Schema.from_json
.
You can pass root_name: 'Foobar'
to change the name of the root type and
namespace_mapping: {}
to map schemas to Ruby modules:
require 'shale/schema'
schema = <<~SCHEMA
{
"type": "object",
"properties": {
"firstName": { "type": "string" },
"lastName": { "type": "string" },
"address": { "$ref": "http://bar.com" }
},
"$defs": {
"Address": {
"$id": "http://bar.com",
"type": "object",
"properties": {
"street": { "type": "string" },
"city": { "type": "string" }
}
}
}
}
SCHEMA
Shale::Schema.from_json(
[schema],
root_name: 'Person',
namespace_mapping: {
nil => 'Api::Foo', # default schema (without ID)
'http://bar.com' => 'Api::Bar',
}
)
# =>
#
# {
# "api/bar/address" => "
# require 'shale'
#
# module Api
# module Bar
# class Address < Shale::Mapper
# attribute :street, Shale::Type::String
# attribute :city, Shale::Type::String
#
# json do
# map 'street', to: :street
# map 'city', to: :city
# end
# end
# end
# end
# ",
# "api/foo/person" => "
# require 'shale'
#
# require_relative '../bar/address'
#
# module Api
# module Foo
# class Person < Shale::Mapper
# attribute :first_name, Shale::Type::String
# attribute :last_name, Shale::Type::String
# attribute :address, Api::Bar::Address
#
# json do
# map 'firstName', to: :first_name
# map 'lastName', to: :last_name
# map 'address', to: :address
# end
# end
# end
# end
# "
# }
You can also use a command line tool to do it:
$ shaleb -c -i schema.json -r Person -m http://bar.com=Api::Bar,=Api::Foo
Generating XML Schema
To generate XML Schema from your Shale data model use:
require 'shale/schema'
Shale::Schema.to_xml(Person, pretty: true)
# =>
#
# {
# 'schema0.xsd' => '
# <xs:schema
# elementFormDefault="qualified"
# attributeFormDefault="qualified"
# xmlns:xs="http://www.w3.org/2001/XMLSchema"
# xmlns:foo="http://foo.com"
# >
# <xs:import namespace="http://foo.com" schemaLocation="schema1.xsd"/>
# <xs:element name="person" type="Person"/>
# <xs:complexType name="Person">
# <xs:sequence>
# <xs:element name="name" type="xs:string" minOccurs="0"/>
# <xs:element ref="foo:address" minOccurs="0"/>
# </xs:sequence>
# </xs:complexType>
# </xs:schema>',
#
# 'schema1.xsd' => '
# <xs:schema
# elementFormDefault="qualified"
# attributeFormDefault="qualified"
# targetNamespace="http://foo.com"
# xmlns:xs="http://www.w3.org/2001/XMLSchema"
# xmlns:foo="http://foo.com"
# >
# <xs:element name="address" type="foo:Address"/>
# <xs:complexType name="Address">
# <xs:sequence>
# <xs:element name="city" type="xs:string" minOccurs="0"/>
# </xs:sequence>
# </xs:complexType>
# </xs:schema>'
# }
You can also use a command line tool to do it:
$ shaleb -i data_model.rb -r Person -p -f xml
If you want to convert your own types to XML Schema types use:
require 'shale'
require 'shale/schema'
class MyEmailType < Shale::Type::Value
...
end
Shale::Schema::XMLGenerator.register_xml_type(MyEmailType, 'myEmailXMLType')
Compiling XML Schema into Shale model
To generate Shale data model from XML Schema use Shale::Schema.from_xml
.
You can pass namespace_mapping: {}
to map XML namespaces to Ruby modules:
require 'shale/schema'
schema1 = <<~SCHEMA
<xs:schema
xmlns:xs="http://www.w3.org/2001/XMLSchema"
xmlns:bar="http://bar.com"
elementFormDefault="qualified"
>
<xs:import namespace="http://bar.com" />
<xs:element name="Person" type="Person" />
<xs:complexType name="Person">
<xs:sequence>
<xs:element name="Name" type="xs:string" />
<xs:element ref="bar:Address" />
</xs:sequence>
</xs:complexType>
</xs:schema>
SCHEMA
schema2 = <<~SCHEMA
<xs:schema
xmlns:xs="http://www.w3.org/2001/XMLSchema"
xmlns:bar="http://bar.com"
targetNamespace="http://bar.com"
elementFormDefault="qualified"
>
<xs:element name="Address" type="bar:Address" />
<xs:complexType name="Address">
<xs:sequence>
<xs:element name="Street" type="xs:string" />
<xs:element name="City" type="xs:string" />
</xs:sequence>
</xs:complexType>
</xs:schema>
SCHEMA
Shale::Schema.from_xml(
[schema1, schema2],
namespace_mapping: {
nil => 'Api::Foo', # no namespace
'http://bar.com' => 'Api::Bar',
}
)
# =>
#
# {
# "api/bar/address" => "
# require 'shale'
#
# module Api
# module Bar
# class Address < Shale::Mapper
# attribute :street, Shale::Type::String
# attribute :city, Shale::Type::String
#
# xml do
# root 'Address'
# namespace 'http://bar.com', 'bar'
#
# map_element 'Street', to: :street
# map_element 'City', to: :city
# end
# end
# end
# end
# ",
# "api/foo/person" => "
# require 'shale'
#
# require_relative '../bar/address'
#
# module Api
# module Foo
# class Person < Shale::Mapper
# attribute :name, Shale::Type::String
# attribute :address, Api::Bar::Address
#
# xml do
# root 'Person'
#
# map_element 'Name', to: :name
# map_element 'Address', to: :address, prefix: 'bar', namespace: 'http://bar.com'
# end
# end
# end
# end
# "
# }
You can also use a command line tool to do it:
$ shaleb -c -f xml -i schema.xml -m http://bar.com=Api::Bar,=Api::Foo
Contributing
Bug reports and pull requests are welcome on GitHub at https://github.com/kgiszczak/shale.
License
The gem is available as open source under the terms of the MIT License.