Datomic Flare for Ruby

A Ruby gem for interacting with Datomic through Datomic Flare.

This is not an official Datomic project or documentation and it is not affiliated with Datomic in any way.

TL;DR and Quick Start

gem 'datomic-flare', '~> 1.0.1'

require 'datomic-flare'

client = Flare.new(credentials: { address: 'http://localhost:3042' })

client.dsl.transact_schema!(
  {
    book: {
      title: { type: :string, doc: 'The title of the book.' },
      genre: { type: :string, doc: 'The genre of the book.' }
    }
  }
)

client.dsl.assert_into!(
  :book,
  {
    title: 'The Tell-Tale Heart',
    genre: 'Horror'
  }
)

client.dsl.query(
  datalog: <<~EDN
    [:find ?e ?title ?genre
     :where [?e :book/title ?title]
            [?e :book/genre ?genre]]
  EDN
)

[[4611681620380877802, 'The Tell-Tale Heart', 'Horror']]

TL;DR and Quick Start
Flare
- Creating a Client
- Meta
Flare DSL
- Creating a Database
- Deleting a Database
- Listing Databases
- Transacting Schema
- Checking Schema
- Asserting Facts
- Reading Data by Entity
- Reading Data by Querying
- Accumulating Facts
- Retracting Facts
Flare API
- Creating a Database
- Deleting a Database
- Listing Databases
- Transacting Schema
- Checking Schema
- Asserting Facts
- Reading Data by Entity
- Reading Data by Querying
- Accumulating Facts
- Retracting Facts
Development
- Publish to RubyGems
- Setup for Tests and Documentation
- Running Tests
- Updating the README

Flare

Creating a Client

require 'datomic-flare'

client = Flare.new(credentials: { address: 'http://localhost:3042' })

Flare DSL

It provides a Ruby-familiar approach to working with Datomic. It brings Ruby’s conventions and idioms while preserving Datomic’s data-first principles and terminology.

This approach should be cozy to those who are familiar with Ruby.

Learn more about Ruby and The Rails Doctrine:

Creating a Database

client.dsl.create_database!('radioactive')

true

Deleting a Database

client.dsl.destroy_database!('radioactive')

true

Listing Databases

client.dsl.databases

['my-datomic-database']

Transacting Schema

Like CREATE TABLE in SQL databases or defining document or record structures in other databases.

client.dsl.transact_schema!(
  {
    book: {
      title: { type: :string, doc: 'The title of the book.' },
      genre: { type: :string, doc: 'The genre of the book.' },
      published_at_year: { type: :long, doc: 'The year the book was first published.' }
    }
  }
)

true

Checking Schema

Like SHOW COLUMNS FROM in SQL databases or checking document or record structures in other databases.

client.dsl.schema

{
  book: {
    published_at_year: {
      type: :long,
      cardinality: :one,
      doc: 'The year the book was first published.',
      unique: false,
      index: false,
      history: true
    },
    title: {
      type: :string,
      cardinality: :one,
      doc: 'The title of the book.',
      unique: false,
      index: false,
      history: true
    },
    genre: {
      type: :string,
      cardinality: :one,
      doc: 'The genre of the book.',
      unique: false,
      index: false,
      history: true
    }
  }
}

Asserting Facts

Like INSERT INTO in SQL databases or creating a new document or record in other databases.

client.dsl.assert_into!(
  :book,
  {
    title: 'Pride and Prejudice',
    genre: 'Romance',
    published_at_year: 1813
  }
)

4611681620380877802

client.dsl.assert_into!(
  :book,
  [{
    title: 'Near to the Wild Heart',
    genre: 'Novel',
    published_at_year: 1943
  },
   {
     title: 'A Study in Scarlet',
     genre: 'Detective',
     published_at_year: 1887
   },
   {
     title: 'The Tell-Tale Heart',
     genre: 'Horror',
     published_at_year: 1843
   }]
)

[4611681620380877804, 4611681620380877805, 4611681620380877806]

Reading Data by Entity

Like SELECT in SQL databases or querying documents or records in other databases.

client.dsl.find_by_entity_id(4611681620380877804)

{
  book: {
    title: 'Near to the Wild Heart',
    genre: 'Novel',
    published_at_year: 1943,
    _id: 4611681620380877804
  }
}

Reading Data by Querying

Like SELECT in SQL databases or querying documents or records in other databases.

client.dsl.query(
  datalog: <<~EDN
    [:find ?e ?title ?genre ?year
     :where [?e :book/title ?title]
            [?e :book/genre ?genre]
            [?e :book/published_at_year ?year]]
  EDN
)

[[4611681620380877805, 'A Study in Scarlet', 'Detective', 1887],
 [4611681620380877804, 'Near to the Wild Heart', 'Novel', 1943],
 [4611681620380877806, 'The Tell-Tale Heart', 'Horror', 1843],
 [4611681620380877802, 'Pride and Prejudice', 'Romance', 1813]]

client.dsl.query(
  params: ['The Tell-Tale Heart'],
  datalog: <<~EDN
    [:find ?e ?title ?genre ?year
     :in $ ?title
     :where [?e :book/title ?title]
            [?e :book/genre ?genre]
            [?e :book/published_at_year ?year]]
  EDN
)

[[4611681620380877806, 'The Tell-Tale Heart', 'Horror', 1843]]

Accumulating Facts

Like UPDATE in SQL databases or updating documents or records in other databases. However, Datomic never updates data. It is an immutable database that only accumulates new facts or retracts past facts.

client.dsl.assert_into!(
  :book, { _id: 4611681620380877806, genre: 'Gothic' }
)

4611681620380877806

Retracting Facts

Like DELETE in SQL databases or deleting documents or records in other databases. However, Datomic never deletes data. It is an immutable database that only accumulates new facts or retracts past facts.

Retract the value of an attribute:

client.dsl.retract_from!(
  :book, { _id: 4611681620380877806, genre: 'Gothic' }
)

true

Retract an attribute:

client.dsl.retract_from!(
  :book, { _id: 4611681620380877804, genre: nil }
)

true

Retract an entity:

client.dsl.retract_from!(
  :book, { _id: 4611681620380877805 }
)

true

Flare API

It provides methods that mirror Datomic's APIs. Most interactions use EDN, closely following Datomic’s documentation.

This approach should be familiar to those who know Datomic concepts and APIs.

Learn more about Clojure and Datomic:

Creating a Database

client.api.create_database!({ name: 'fireball' })['data']

true

Deleting a Database

client.api.delete_database!({ name: 'fireball' })['data']

true

Listing Databases

# Flare on Peer Mode
client.api.get_database_names['data']

# Flare on Client Mode
client.api.list_databases['data']

['my-datomic-database']

Transacting Schema

client.api.transact!(
  { data: <<~EDN
    [{:db/ident       :book/title
      :db/valueType   :db.type/string
      :db/cardinality :db.cardinality/one
      :db/doc         "The title of the book."}

     {:db/ident       :book/genre
      :db/valueType   :db.type/string
      :db/cardinality :db.cardinality/one
      :db/doc         "The genre of the book."}

     {:db/ident       :book/published_at_year
      :db/valueType   :db.type/long
      :db/cardinality :db.cardinality/one
      :db/doc         "The year the book was first published."}]
  EDN
  }
)['data']

{
  'db-before' => 'datomic.db.Db@72351777',
  'db-after' => 'datomic.db.Db@34ed3129',
  'tx-data' =>
  [[13194139534312, 50, '2024-09-29T14:29:21.908Z', 13194139534312, true],
   [72, 10, ':book/title', 13194139534312, true],
   [72, 40, 23, 13194139534312, true],
   [72, 41, 35, 13194139534312, true],
   [72, 62, 'The title of the book.', 13194139534312, true],
   [73, 10, ':book/genre', 13194139534312, true],
   [73, 40, 23, 13194139534312, true],
   [73, 41, 35, 13194139534312, true],
   [73, 62, 'The genre of the book.', 13194139534312, true],
   [74, 10, ':book/published_at_year', 13194139534312, true],
   [74, 40, 22, 13194139534312, true],
   [74, 41, 35, 13194139534312, true],
   [74, 62, 'The year the book was first published.', 13194139534312, true],
   [0, 13, 72, 13194139534312, true],
   [0, 13, 73, 13194139534312, true],
   [0, 13, 74, 13194139534312, true]],
  'tempids' =>
  {
    '-9223300668110558584' => 72,
    '-9223300668110558583' => 73,
    '-9223300668110558582' => 74
  }
}

Checking Schema

client.api.q(
  {
    inputs: [{ database: { latest: true } }],
    query: <<~EDN
      [:find
          ?e ?ident ?value_type ?cardinality ?doc
          ?unique ?index ?no_history
       :in $
       :where
         [?e :db/ident ?ident]

         [?e :db/valueType ?value_type_id]
         [?value_type_id :db/ident ?value_type]

         [?e :db/cardinality ?cardinality_id]
         [?cardinality_id :db/ident ?cardinality]

         [(get-else $ ?e :db/doc "") ?doc]

         [(get-else $ ?e :db/unique -1) ?unique_id]
         [(get-else $ ?unique_id :db/ident false) ?unique]

         [(get-else $ ?e :db/index false) ?index]
         [(get-else $ ?e :db/noHistory false) ?no_history]]
    EDN
  }
)['data'].filter do |datom|
  !%w[
    db
    db.alter db.attr db.bootstrap db.cardinality db.entity db.excise
    db.fn db.install db.lang db.part db.sys db.type db.unique
    fressian
  ].include?(datom[1].split('/').first)
end

[[74,
  'book/published_at_year',
  'db.type/long',
  'db.cardinality/one',
  'The year the book was first published.',
  false,
  false,
  false],
 [72,
  'book/title',
  'db.type/string',
  'db.cardinality/one',
  'The title of the book.',
  false,
  false,
  false],
 [73,
  'book/genre',
  'db.type/string',
  'db.cardinality/one',
  'The genre of the book.',
  false,
  false,
  false]]

Asserting Facts

client.api.transact!(
  { data: <<~EDN
    [{:db/id      -1
      :book/title "Pride and Prejudice"
      :book/genre "Romance"
      :book/published_at_year 1813}]
  EDN
  }
)['data']

{
  'db-before' => 'datomic.db.Db@6674710',
  'db-after' => 'datomic.db.Db@2d37f88f',
  'tx-data' =>
  [[13194139534313, 50, '2024-09-29T14:29:22.020Z', 13194139534313, true],
   [4611681620380877802, 72, 'Pride and Prejudice', 13194139534313, true],
   [4611681620380877802, 73, 'Romance', 13194139534313, true],
   [4611681620380877802, 74, 1813, 13194139534313, true]],
  'tempids' => { '-1' => 4611681620380877802 }
}

client.api.transact!(
  { data: <<~EDN
    [{:db/id      -1
      :book/title "Near to the Wild Heart"
      :book/genre "Novel"
      :book/published_at_year 1943}
     {:db/id      -2
      :book/title "A Study in Scarlet"
      :book/genre "Detective"
      :book/published_at_year 1887}
     {:db/id      -3
      :book/title "The Tell-Tale Heart"
      :book/genre "Horror"
      :book/published_at_year 1843}]
  EDN
  }
)['data']

{
  'db-before' => 'datomic.db.Db@23f01359',
  'db-after' => 'datomic.db.Db@27cf93ff',
  'tx-data' =>
  [[13194139534315, 50, '2024-09-29T14:29:22.072Z', 13194139534315, true],
   [4611681620380877804, 72, 'Near to the Wild Heart', 13194139534315, true],
   [4611681620380877804, 73, 'Novel', 13194139534315, true],
   [4611681620380877804, 74, 1943, 13194139534315, true],
   [4611681620380877805, 72, 'A Study in Scarlet', 13194139534315, true],
   [4611681620380877805, 73, 'Detective', 13194139534315, true],
   [4611681620380877805, 74, 1887, 13194139534315, true],
   [4611681620380877806, 72, 'The Tell-Tale Heart', 13194139534315, true],
   [4611681620380877806, 73, 'Horror', 13194139534315, true],
   [4611681620380877806, 74, 1843, 13194139534315, true]],
  'tempids' =>
  {
    '-1' => 4611681620380877804,
    '-2' => 4611681620380877805,
    '-3' => 4611681620380877806
  }
}

Reading Data by Entity

client.api.entity(
  {
    database: { latest: true },
    id: 4611681620380877804
  }
)['data']

{
  ':book/title' => 'Near to the Wild Heart',
  ':book/genre' => 'Novel',
  ':book/published_at_year' => 1943,
  ':db/id' => 4611681620380877804
}

Reading Data by Querying

client.api.q(
  {
    inputs: [{ database: { latest: true } }],
    query: <<~EDN
      [:find ?e ?title ?genre ?year
       :where [?e :book/title ?title]
              [?e :book/genre ?genre]
              [?e :book/published_at_year ?year]]
    EDN
  }
)['data']

[[4611681620380877805, 'A Study in Scarlet', 'Detective', 1887],
 [4611681620380877804, 'Near to the Wild Heart', 'Novel', 1943],
 [4611681620380877806, 'The Tell-Tale Heart', 'Horror', 1843],
 [4611681620380877802, 'Pride and Prejudice', 'Romance', 1813]]

client.api.q(
  {
    inputs: [
      { database: { latest: true } },
      'The Tell-Tale Heart'
    ],
    query: <<~EDN
      [:find ?e ?title ?genre ?year
       :in $ ?title
       :where [?e :book/title ?title]
              [?e :book/genre ?genre]
              [?e :book/published_at_year ?year]]
    EDN
  }
)['data']

[[4611681620380877806, 'The Tell-Tale Heart', 'Horror', 1843]]

Accumulating Facts

client.api.transact!(
  { data: <<~EDN
    [{:db/id 4611681620380877806 :book/genre "Gothic"}]
  EDN
  }
)['data']

{
  'db-before' => 'datomic.db.Db@62cd1962',
  'db-after' => 'datomic.db.Db@702c6634',
  'tx-data' =>
  [[13194139534319, 50, '2024-09-29T14:29:22.286Z', 13194139534319, true],
   [4611681620380877806, 73, 'Gothic', 13194139534319, true],
   [4611681620380877806, 73, 'Horror', 13194139534319, false]],
  'tempids' => {}
}

Retracting Facts

Retract the value of an attribute:

client.api.transact!(
  { data: <<~EDN
    [[:db/retract 4611681620380877806 :book/genre "Gothic"]]
  EDN
  }
)['data']

{
  'db-before' => 'datomic.db.Db@271bc3fa',
  'db-after' => 'datomic.db.Db@7d4ff6c1',
  'tx-data' =>
  [[13194139534320, 50, '2024-09-29T14:29:22.337Z', 13194139534320, true],
   [4611681620380877806, 73, 'Gothic', 13194139534320, false]],
  'tempids' => {}
}

Retract an attribute:

client.api.transact!(
  { data: <<~EDN
    [[:db/retract 4611681620380877804 :book/genre]]
  EDN
  }
)['data']

{
  'db-before' => 'datomic.db.Db@20f6e438',
  'db-after' => 'datomic.db.Db@1c1568f1',
  'tx-data' =>
  [[13194139534321, 50, '2024-09-29T14:29:22.377Z', 13194139534321, true],
   [4611681620380877804, 73, 'Novel', 13194139534321, false]],
  'tempids' => {}
}

Retract an entity:

client.api.transact!(
  { data: <<~EDN
    [[:db/retractEntity 4611681620380877805]]
  EDN
  }
)['data']

{
  'db-before' => 'datomic.db.Db@65687c9a',
  'db-after' => 'datomic.db.Db@67937708',
  'tx-data' =>
  [[13194139534322, 50, '2024-09-29T14:29:22.416Z', 13194139534322, true],
   [4611681620380877805, 72, 'A Study in Scarlet', 13194139534322, false],
   [4611681620380877805, 73, 'Detective', 13194139534322, false],
   [4611681620380877805, 74, 1887, 13194139534322, false]],
  'tempids' => {}
}

Development

bundle
rubocop -A

Publish to RubyGems

gem build datomic-flare.gemspec

gem signin

gem push datomic-flare-1.0.1.gem

Setup for Tests and Documentation

Tests run against real Datomic databases, and documentation (README) is generated by interacting with real Datomic databases.

To accomplish that, we need to have Datomic and Flare running.

TL;DR:

git clone https://github.com/gbaptista/datomic-pro-docker.git

cd datomic-pro-docker

cp compose/datomic-flare-dev.yml docker-compose.yml

docker compose up -d datomic-storage

docker compose run datomic-tools psql \
  -f bin/sql/postgres-table.sql \
  -h datomic-storage \
  -U datomic-user \
  -d my-datomic-storage

docker compose up -d datomic-transactor

docker compose run datomic-tools clojure -M -e "$(cat <<'CLOJURE'
  (require '[datomic.api :as d])
  
  (d/create-database "datomic:sql://my-datomic-database?jdbc:postgresql://datomic-storage:5432/my-datomic-storage?user=datomic-user&password=unsafe")

  (d/create-database "datomic:sql://my-datomic-database-test?jdbc:postgresql://datomic-storage:5432/my-datomic-storage?user=datomic-user&password=unsafe")

  (d/create-database "datomic:sql://my-datomic-database-test-green?jdbc:postgresql://datomic-storage:5432/my-datomic-storage?user=datomic-user&password=unsafe")

  (System/exit 0)
CLOJURE
)"

docker compose up -d datomic-peer-server

docker compose up -d datomic-flare-peer datomic-flare-client

curl -s http://localhost:3042/meta \
  -X GET \
  -H "Content-Type: application/json"  \
| jq

{
  "data": {
    "mode": "peer"
  }
}

curl -s http://localhost:3043/meta \
  -X GET \
  -H "Content-Type: application/json"  \
| jq

{
  "data": {
    "mode": "client"
  }
}

You are ready to run tests and generate documentation.

Detailed instructions:

Clone the datomic-pro-docker repository and copy the Docker Compose template:

git clone https://github.com/gbaptista/datomic-pro-docker.git

cd datomic-pro-docker

cp compose/datomic-flare-dev.yml docker-compose.yml

Start PostgreSQL as Datomic's storage service:

docker compose up -d datomic-storage

docker compose logs -f datomic-storage

Create the table for Datomic databases:

docker compose run datomic-tools psql \
  -f bin/sql/postgres-table.sql \
  -h datomic-storage \
  -U datomic-user \
  -d my-datomic-storage

You will be prompted for a password, which is unsafe.

Start the Datomic Transactor:

docker compose up -d datomic-transactor

docker compose logs -f datomic-transactor

Create the following databases:

my-datomic-database
my-datomic-database-test
my-datomic-database-test-green

docker compose run datomic-tools clojure -M -e "$(cat <<'CLOJURE'
  (require '[datomic.api :as d])
  
  (d/create-database "datomic:sql://my-datomic-database?jdbc:postgresql://datomic-storage:5432/my-datomic-storage?user=datomic-user&password=unsafe")

  (d/create-database "datomic:sql://my-datomic-database-test?jdbc:postgresql://datomic-storage:5432/my-datomic-storage?user=datomic-user&password=unsafe")

  (d/create-database "datomic:sql://my-datomic-database-test-green?jdbc:postgresql://datomic-storage:5432/my-datomic-storage?user=datomic-user&password=unsafe")

  (System/exit 0)
CLOJURE
)"

Start the Peer Server:

docker compose up -d datomic-peer-server

docker compose logs -f datomic-peer-server

Start 2 instances of Flare, one in Peer Mode and another in Client Mode:

docker compose up -d datomic-flare-peer datomic-flare-client

docker compose logs -f datomic-flare-peer
docker compose logs -f datomic-flare-client

You should be able to request both:

Datomic Flare in Peer Mode:

curl -s http://localhost:3042/meta \
  -X GET \
  -H "Content-Type: application/json"  \
| jq

{
  "data": {
    "mode": "peer"
  }
}

Datomic Flare in Client Mode:

curl -s http://localhost:3043/meta \
  -X GET \
  -H "Content-Type: application/json"  \
| jq

{
  "data": {
    "mode": "client"
  }
}

You are ready to run tests and generate documentation.

Running Tests

Tests run against real Datomic databases, so complete the Setup for Tests and Documentation first.

cp .env.example .env

bundle exec rspec

Updating the README

Documentation (README) is generated by interacting with real Datomic databases, so complete the Setup for Tests and Documentation first.

Update the docs/templates/*.md files, and then:

cp .env.example .env

bundle exec ruby ports/cli.rb docs:generate

Trick for automatically updating the README.md when docs/templates/*.md files change:

sudo pacman -S inotify-tools # Arch / Manjaro
sudo apt-get install inotify-tools # Debian / Ubuntu / Raspberry Pi OS
sudo dnf install inotify-tools # Fedora / CentOS / RHEL

while inotifywait -e modify docs/templates/*; \
  do bundle exec ruby ports/cli.rb docs:generate; \
  done

Trick for Markdown Live Preview:

pip install -U markdown_live_preview

mlp README.md -p 8042 --no-follow

datomic-flare

Runtime

Datomic Flare for Ruby

TL;DR and Quick Start

Flare

Creating a Client

Meta

Flare DSL

Creating a Database

Deleting a Database

Listing Databases

Transacting Schema

Checking Schema

Asserting Facts

Reading Data by Entity

Reading Data by Querying

Accumulating Facts

Retracting Facts

Flare API

Creating a Database

Deleting a Database

Listing Databases

Transacting Schema

Checking Schema

Asserting Facts

Reading Data by Entity

Reading Data by Querying

Accumulating Facts

Retracting Facts

Development

Publish to RubyGems

Setup for Tests and Documentation

Running Tests

Updating the README