Observed::Logstash

Allows you to use the number of results returned by an Elasticsearch query as an Observed healthcheck.

Expected use case is searching server logs stored in Logstash format.

You provide an Elasticsearch query and a timespan, and the plugin will search for logs that match your query. If there are too few hits, or too many, it will record an error event.

Example use cases

If your web server returned more than X "500 Internal Server Error" responses in the last few minutes, it's probably unhealthy.
If it returned fewer than Y "200 OK" responses in the last few minutes, it's probably unhealthy.

Installation

Add this line to your application's Gemfile:

gem 'observed-logstash'

And then execute:

$ bundle

Or install it yourself as:

$ gem install observed-logstash

Test

Some tests expect an Elasticsearch instance to be running on Localhost.

$ git clone https://github.com/cb372/observed-logstash.git
$ cd observed-logstash
$ bundle install
$ elasticsearch
$ bundle exec rspec

Usage

Configuration parameters

Name	Required?	Default value	Description
host	No	localhost:9200	ES server hostname and port
index_name_format	No	logstash-%Y.%m.%d (Logstash daily format)	Naming format of ES indices
query	Yes		A hash representing an ES query, e.g. { :term => { :status => 404 } }
timespan_in_seconds	Yes		Search for logs from the last N seconds
max_hits	No	1000000	Maximum number of matching logs in the last N seconds. If there are more than these, an error will be recorded.
min_hits	No	0

Example configuration

observe 'myapp.404', via: 'logstash', with: {
    host: 'localhost:9200',
    index_name_format: 'observed-logstash-test-%Y.%m.%d',
    query: { :term => { :status => 404 } },
    timespan_in_seconds: 3600,
    max_hits: 10
}

Example reporting

report /myapp.404/, via: 'stdout', with: {
    format: -> tag, time, data {
      case data[:status]
      when :success
        "Looks OK! #{data[:message]}"
      else
        "Oh noes! #{data[:message]}"
      end
    }
}