EagleSearch

EagleSearch is a ruby gem that integrates Rails ActiveRecord to Elasticsearch. It handles the Elasticsearch internals by itself, and in most cases minimal (none) configuration is needed.

Installation

First of all, you should have Elasticsearch installed.

Using Homebrew:

brew install elasticsearch

Add eagle_search to Gemfile:

gem 'eagle_search'

Get Started

Add EagleSearch module and call eagle_search class method in your ActiveRecord:

class Article < ActiveRecord::Base
  include EagleSearch
  eagle_search

  def index_data
    as_json only: [:title, :body, :active]
  end
end

Notice that EagleSearch will use the #index_data method to know what will be indexed.

As the model was configured, you should populate the records into Elasticsearch index:

Article.reindex

EagleSearch will automatically handle the mapping based on model column types unless you explicit set a custom mapping.

Article.search "programming language"

Searching

Basic

The following code will return all articles:

articles = Article.search "*"

You can iterate over the records (will hit the database):

articles.each do |article_record|
  ...
end

To avoid the database, you can access the hits directly:

articles.hits.each do |article_hit|
  ...
end

Filtering

Get only active articles:

Article.search "*", filters: { active: true }

If you want to combine conditions, you should add the operator:

Product.search "*", filters: {
  and: {
    active: true,
    in_stock: true
  }
}

You can go deep, mixing AND and OR operators:

Product.search "*", filters: {
  and: {
    active: true,
    or: [
      {
        and: {
          price: { gte: 543.50 },
          available_stock: 300
        }
      },
      {
        available_stock: 500
      },
      {
        not: { expired: false }
      }
    ]
  }
}

Filtering by string (only for not_analyzed strings):

Product.search "*", filters: {
  name: "Book: The Hidden Child"
}

If you want to map a string field as an exact value, you need to set it, as documented here.

Filtering by ranges:

Product.search "*", filters: {
  available_stock: (10..150)
}

or equivalent:

Product.search "*", filters: {
  available_stock: {
    gte: 10,
    lte: 150
  }
}

Available options are gte, gt, lt, lte.

Complex queries

As queries in some cases might to get very complex, you can make your query by yourself, in the next code, EagleSearch will replace ONLY the query part of filtered query, which will let you to use EagleSearch filter:

Product.search "*", custom_query: {
  ...
}, filters: { price: { gt: 50 } }

Similarly, you can make the filter part by yourself:

Product.search "*", custom_filter: {
  ...
}

If you need to make the whole query by yourself, set a custom payload:

Product.search "*", custom_payload: {
  ...
}

Relevance (boost)

If you want to consider some fields more important:

Product.search "*", fields: ["name^5", "description"]

The number 5 is the factor that the field will be boosted, see here.

Pagination (tested with Kaminari)

Product.search "*", page: 2, per_page: 20

Defaults: page: 1 and per_page: 10

Typoes and misspellings

Product.search "neighhbour" #matches documents containing whether 'neighbor' or 'neighbour'

Highlight

products = Product.search "book", highlight: { fields: [:name], tags: ["<span>"] }
products.hits.each do |hit|
  hit["highlight"]["name"] # "<span>Book</span>: The Hidden Child"
end

Autocomplete

Product.search "fri" # will match docs with fields like: "fries, friendship..."

You can disable this behavior:

Product.search "fri", autocomplete: false # won't match docs with fields like: "fries, friendship..."

Aggregations

Official documentation

To aggregate data, set the aggregations option:

products = Product.search "*", aggregations: :category
products.aggregations

Response:

{
  "category"=>{
    ...
    "buckets"=>[
      { "key"=>"Book", "doc_count"=>2 },
      { "key"=>"Vesture", "doc_count"=>1 }
    ]
    ...
  }
}

By default, when an aggregation is a symbol or a string, it will be interpreted as a terms aggregation.

Multiple aggregations:

products = Product.search "*", aggregations: [:category, :country]
products.aggregations

Response:

{
  "category"=>{
    ...
    "buckets"=>[
      { "key"=>"Book", "doc_count"=>2 },
      { "key"=>"Vesture", "doc_count"=>1 }
    ]
    ...
  },
  "country"=>{
    ...
    "buckets"=>[
      { "key"=>"Brazil", "doc_count"=>1 },
      { "key"=>"USA", "doc_count"=>1 },
      { "key"=>"Spain", "doc_count"=>1 }
    ]
    ...
  }
}

Nesting aggregations:

products = Product.search "*", aggregations: { category: :country }
products.aggregations

Response:

{
  "category"=>{
    ...
    "buckets"=>[
      {
        "key"=>"Book"
        "doc_count"=>2,
        "country"=>{
          "buckets"=>[
            { "key"=>"Brazil", "doc_count"=>1 },
            { "key"=>"USA", "doc_count"=>1 }
          ]
        }
      },
      {
        "key"=>"Vesture"
        "doc_count"=>1,
        "country"=>{
          "buckets"=>[
            { "key"=>"Spain", "doc_count"=>1 }
          ]
        }
      }
    ]
    ...
  }
}

Stats aggregations:

products = Product.search "*", aggregations: {
  available_stock: { type: "stats" }
}
products.aggregations

Response:

{
  "available_stock"=>{
    "count"=>4,
    "min"=>20.0,
    "max"=>400.0,
    "avg"=>185.0,
    "sum"=>740.0
  }
}

Mixing terms and stats aggregations:

products = Product.search "*", aggregations: {
  country: {
    available_stock: { type: "stats" }
  }
}
products.aggregations

Response:

{
  "country"=>{
    ...
    "buckets"=>[
      {
        "key"=>"Brazil",
        "doc_count"=>1,
        "available_stock"=>{
          "count"=>4,
          "min"=>20.0,
          "max"=>400.0,
          "avg"=>185.0,
          "sum"=>740.0
        }
      },
      ...
    ]
  }
}

Ranges aggregations:

products = Product.search "*", aggregations: {
  available_stock: {
    ranges: [
      (0..30),
      (30..60),
      { from: 60, to: 90 },
      { from: 90 }
    ]
  }
}
products.aggregations

Response:

{
  "available_stock"=>{
    ...
    "buckets"=>[
      {
        ...
        "key"=>"0-30",
        "doc_count"=>4
      },
      {
        ...
        "key"=>"30-60",
        "doc_count"=>2
      },
      {
        ...
        "key"=>"60-90",
        "doc_count"=>13
      },
      {
        ...
        "key"=>"90-*",
        "doc_count"=>37
      }
    ]
  }
}

Remember that you can go as deep as you want nesting and mixing terms and stats aggregations.

Settings

Exact string fields

By default, EagleSearch (even Elasticsearch) will map string field as analyzed, which won't let you to filter these fields.

If you have an exact value for string, and want to search by its exact value, you explicitly need to declare it, for example:

class Product < ActiveRecord::Base
  include EagleSearch
  eagle_search exact_match_fields: [:code]
end

It will let you to filter string fields:

Product.search "*", filters: {
  code: "JUR123-A"
}

Unsearchable fields

You can disable the search and filter on certain fields:

class Product < ActiveRecord::Base
  include EagleSearch
  eagle_search unsearchable_fields: [:code]
end

Synonyms

Official documentation

You can explicitly declare your synonyms in your class:

class Product < ActiveRecord::Base
  include EagleSearch
  eagle_search synonyms: [
    "dog, canine",
    "cup, glass, chalice"
  ]
end

So, whether searching by cup, glass or chalice will match documents containing these words, as well as dog or canine.

Putting synonyms in a separated file:

class Product < ActiveRecord::Base
  include EagleSearch
  eagle_search synonyms: {
    format: "wordnet", #could be solr format
    synonyms_path: "synonyms.txt" #relative to elasticsearch config location
  }
end

Using WordNet format files:

class Product < ActiveRecord::Base
  include EagleSearch
  eagle_search synonyms: {
    format: "wordnet",
    synonyms_path: "[WORDNET_FILE_LOCATION]"
  }
end

Using Solr format files:

class Product < ActiveRecord::Base
  include EagleSearch
  eagle_search synonyms: {
    format: "solr",
    synonyms_path: "[SOLR_FILE_LOCATION]"
  }
end

Custom Mapping

You can declare the index mapping by yourself:

class Product < ActiveRecord::Base
  include EagleSearch
  eagle_search mappings: {
    type_name: {
      properties: {
        name: {
          index: "no",
          type: "string"
        }
      }
    }
  }
end

Custom index name

class Product < ActiveRecord::Base
  include EagleSearch
  eagle_search index_name: "product"
end

Language

You can set the language of your index (default is english):

class Product < ActiveRecord::Base
  include EagleSearch
  eagle_search language: "portuguese"
end

Available languages are here

IMPORTANT all of the settings above require index to be reindexed:

Product.reindex

Auto reindex

As a record is created or changed, EagleSearch automatically reindex the record by default.

You can disable the auto reindex:

class Product < ActiveRecord::Base
  include EagleSearch
  eagle_search reindex: false
end

Roadmap

Suggestions
Elasticsearch 2.x

eagle_search

Development

Runtime

EagleSearch

Installation

Get Started

Searching

Basic

Filtering

Complex queries

Relevance (boost)

Pagination (tested with Kaminari)

Typoes and misspellings

Highlight

Autocomplete

Aggregations

Settings

Exact string fields

Unsearchable fields

Synonyms

Custom Mapping

Custom index name

Language

Auto reindex

Roadmap