Marlowe, Your Request Sleuth¶ ↑

code

github.com/KineticCafe/marlowe/

issues

github.com/KineticCafe/marlowe/issues

docs

www.rubydoc.info/github/KineticCafe/marlowe/master

continuous integration

<img src=“https://travis-ci.org/KineticCafe/marlowe.svg?branch=master” alt=“Build Status” />

Marlowe is a Rack middleware that extracts or creates a request ID using a pre-defined header, permitting request correlation across multiple services.

When using Rails, Marlowe automatically adds itself to the middleware before Rails::Rack::Logger.

As of Marlowe 3.0, a Faraday middleware is provided (require 'marlowe/faraday').

In Marlowe 2.0, configuration was entirely specified in Rails configuration or in the Rack use clause. With the addition of a Faraday middleware for use with Marlowe, centralization of the configuration is advisable.

The existing configuration mechanisms will work (and create instance configurations if provided), but it is instead recommended to use the global configuration:

# Compatibility with Marlowe 1.0
Marlowe.configure do |config|
  config.header = 'Correlation-ID'
  config.handler = :simple
  config.return = false
end

Configuration is static for Marlowe::Middleware or Marlowe::Faraday instances and changes to Marlowe configuration only affect new instances. If the configuration options are provided in the middleware use clause, they will override the configured values.

Marlowe.configure do |config|
  config.header = 'Correlation-ID'
  config.handler = :simple
  config.return = false
end
use Marlowe::Middleware, handler: :clean
# The handler will be the :clean handler for the used middleware.

In Marlowe 1.0, the correlation header was called Correlation-Id; since then, Rails 5 and other tools and frameworks (such as Phoenix) have standardized on the header X-Request-Id. Marlowe 2.0 changes to this header by default.

To keep complete compatibility with Marlowe 1.0, the following should be used:

# Rails
config.marlowe_header = 'Correlation-Id'
config.marlowe_handler = :simple
config.marlowe_return = false

# Rack
use Marlowe::Middleware, header: 'Correlation-Id', handler: :simple, return: false

Marlowe has three main configuration options: the request ID header, the request ID handler, and the request ID return. The options may be provided to the Rack use command as a keyword option, or set in a corresponding marlowe_option configuration variable in Rails.

Specifies the header to be used for the request correlation ID. Defaults to X-Request-Id.

# Rails
config.marlowe_header = 'Correlation-Id'
# OR: config.marlowe_correlation_header = 'Correlation-Id'

# Rack
use Marlowe::Middleware, header: 'Correlation-Id'
# OR: use Marlowe::Middleware, correlation_header: 'Correlation-Id'

Marlowe will convert this to an appropriate HTTP header (in the Rack env parameter, the above header would be represented as env['HTTP_CORRELATION_ID']).

Specifies the method for sanitizing or generating the request correlation ID. Values can be :clean (the default, which limits incoming correlation IDs to 255 alphanumeric-or-dash characters), :simple (does not limit incoming correlation IDs), or a proc to transform or generate a correlation ID.

In all cases, if a correlation request ID is not handled, a UUID will be generated.

# Rails
config.marlowe_handler = :simple
config.marlowe_handler = ->(req_id) {
  req_id.try(:reverse) || SecureRandom.uuid
}

# Rack
use Marlowe::Middleware, handler: :simple
use Marlowe::Middleware, handler: ->(req_id) {
  req_id.try(:reverse) || SecureRandom.uuid
}

If true (the default), the request correlation ID will be returned to the client in the same header that it was provided in.

# Rails
config.marlowe_return = false

# Rack
use Marlowe::Middleware, return: false

Rails 5 includes the ActionDispatch::RequestId middleware, reducing the need for Marlowe. Marlowe is more configurable than the Rails 5 default, so set marlowe_replace_action_dispatch_request_id to true to have Marlowe::Middleware will replace ActionDispatch::RequestId:

# Rails only
config.marlowe_replace_action_dispatch_request_id = true

The correlation id can be accessed throughout the application by accessing the RequestStore storage.

RequestStore[:correlation_id]

For a Rails application, you simply need to change the log formatter to one of the provided ones. Correlated versions of both the SimpleFormatter and Formatter are included.

# config/environments/development.rb
Rails.application.configure do
  config.log_formatter = Marlowe::SimpleFormatter.new
end

To create your own formatter, you’ll need to access the RequestStore storage. You can use this pattern if you’ve rolled your own logger/formatter:

# lib/correlated_formatter.rb
require 'request_store'

class CorrelatedSimpleFormatter < ActiveSupport::Logger::SimpleFormatter
  def call(severity, timestamp, progname, msg)
    "[#{RequestStore.store[:correlation_id]}] #{super}"
  end
end

As lograge supplies its own formatter, you will need to do something a little different:

# config/application.rb

class Application < Rails::Application
  config.before_initialize do
    ...
    # use lograge for all request logs
    config.lograge.enabled = true
    config.lograge.custom_options = lambda do |event|
      { correlation_id: RequestStore[:correlation_id] }
    end
  end
end

As semantic_logger provides its own formatters this should be added as a tag. The best way that I can see to do this is to capture the correlation_id in an on_log event:

# config/initializers/semantic_logger.rb

SemanticLogger.on_log do |log|
  if RequestStore[:correlation_id]
    log.named_tags[:correlation_id] = RequestStore[:correlation_id]
  end
end

Catching and creating the correlation ID is a great all on its own, but to really take advantage of the correlation in a service based architecture you’ll need to pass the request ID to the next service in the change.

Here’s an example with Faraday:

require 'faraday'
require 'faraday_middleware'
require 'marlowe/faraday'

conn = Faraday.new(url: 'https://example.org/') do |conn|
  conn.request :marlowe
  conn.request :json

  conn.response :json
  conn.adapter Faraday.default_adapter
end

Add Marlowe to your Gemfile:

gem 'marlowe', '~> 2.0'

Or manually install:

$ gem install marlowe

Marlowe uses a Semantic Versioning scheme with one significant change:

• When PATCH is zero (0), it will be omitted from version references.

Additionally, the major version will generally be reserved for plug-in infrastructure changes.

Marlowe welcomes your contributions as described in Contributing.md. This project, like all Kinetic Cafe open source projects, is under the Kinetic Cafe Open Source Code of Conduct.