esse - ElasticSearch and OpenSearch Ruby Client

This gem is a Ruby simple and extremely flexible client for ElasticSearch and OpenSearch based on official clients such as elasticsearch-ruby and opensearch-ruby. It's a pure Ruby implementation, framework agnostic, and due to its modular design, it's easy to extend and adapt to your needs. Esse extensions are available as separate gems. A few examples:

• esse-pagy - Pagy pagination support
• esse-kaminari - Kaminari pagination support
• esse-will_paginate - WillPaginate pagination support
• esse-rails - Ruby on Rails integration
• esse-active_record - ActiveRecord integration
• WIP esse-sequel - Sequel integration
• esse-rspec - RSpec helpers and matchers
• esse-jbuilder - Build Search requests using Jbuilder templates
• esse-redis_storage - Add-on for the esse gem to be used with Redis as a storage backend
• esse-async_indexing - Esse extension to allow async indexing using Faktory or Sidekiq.

The main idea of the gem is to be compatible with any type of datasource. It means that you can use it with ActiveRecord, Sequel, HTTP APIs, or any other data source. The gem is divided into three main components:

• Index: The index is the main component. It's responsible for defining the index settings, mappings, and other index-level configurations. It also provides a DSL to define the next two components.
• Repository: The repository is responsible for loading the data. One index may have more than one repo. Each repo must implement a collection. Collection is an Enumerable that iterate over the data in chunks. It may receive a given context for filtering the data that can be used in the next component.
• Document: Each repo's collection must have a respective Document. The document is responsible for defining the document itself. It's a simple Ruby object that can be used to define the document id, routing, doc attributes.

This architecture provides a powerful ETL (Extract, Transform, Load) solution for your elasticsearch/opensearch indices.

And to help you to build and interact with the index, the gem provides a CLI tool called esse. You can use it to generate index boilerplate, and to interact with index operations and elasticsearch/opensearch cluster

Bellow is an image that shows the main components and how they interact with each other.

Add this line to your application's Gemfile:

gem 'elasticsearch' # or gem 'opensearch' with specific version according to your needs
gem 'esse'

And then execute:

❯ bundle install

Or install it yourself as:

❯ gem install esse

To get started, you need to create the configuration file with the elasticsearch/opensearch cluster connection information. You can use the CLI tool to generate the configuration file:

❯ esse install

The CLI generates the configuration file in the config/esse.rb directory. Make sure you require application dependencies in the configuration file if you need to access them in your index classes.

require_relative "../environment" unless defined?(Rails)

Esse.configure do |config|
  conf.cluster(:default) do |cluster|
    cluster.client = Elasticsearch::Client.new
  end
end

For more information about the configuration options, check out the Configuration page.

Now you need to create an index class. You can use the CLI tool to generate the boilerplate:

❯ esse generate index PostsIndex

For more information about the index class, check out the Index page.

After editing the index class, you can create the index in the elasticsearch/opensearch cluster and start indexing the data. There are several ways to do that. But the easiest way is to use the index reset command from the CLI tool:

❯ esse index reset PostsIndex

Reset will create the index with the defined settings and mappings, and then it will start indexing data using the bulk API, and finally, it will point the alias to the created index name. I recommend checking out the CLI page for more information about the CLI tool.

In the wiki you can find more information about the gem and how to use it. If you have any questions, feel free to open an issue or contact me on twitter.

Bug reports and pull requests are welcome on GitHub at https://github.com/marcosgz/esse.

The gem is available as open source under the terms of the MIT License.