Wrappi

Making APIs fun again!

Wrappi is a Framework to create API clients. The intention is to bring the best practices and standardize how API clients behave. It allows to create API clients in a declarative way improving readability and unifying the behavior. It abstracts complex operations like caching, retries, background requests and error handling.

Enjoy!

Add this line to your application's Gemfile:

gem 'wrappi'

And then execute:

$ bundle

Or install it yourself as:

$ gem install wrappi

Github example:

You can test this examples running bin/console

module GithubCLI
  class Client < Wrappi::Client
    setup do |config|
      config.domain = 'https://api.github.com'
      config.headers = {
        'Content-Type' => 'application/json',
        'Accept' => 'application/vnd.github.v3+json',
      }
    end
  end

  class User < Wrappi::Endpoint
    client Client
    verb :get
    path "users/:username"
  end
end

user = GithubCLI::User.new(username: 'arturictus')
user.success? # => true
user.error? # => false
user.status_code # => 200
user.body # => {"login"=>"arturictus", "id"=>1930175, ...}

The next behaviours are using #success? method. You can override by redefining your own success?

The current #success? is defined like this:

wrappi/endpoint.rb

  def self.success?(request)
    request.code < 300 && request.code >= 200
  end

Overrride your own in Endpoint

  class User < Wrappi::Endpoint
    client Client
    verb :get
    path "users/:username"

    def self.success?(request)
      request.status == 200
    end
  end

GithubCLI::User.new(username: 'arturictus')
               .on_success do |inst|
                 inst.status_code # => 200
                 inst.body # => {"login"=>"arturictus", "id"=>1930175, ...}
                 # do something useful
               end.on_error do |inst|
                 puts "Error retrieving use"
               end

If you just need to retrieve the body and handle the error response on your side

GithubCLI::User.body(username: 'arturictus') # => {"login"=>"arturictus", "id"=>1930175, ...}

GithubCLI::User.body(username: 'sdfsdfasdjfojaspdjfpsajdpfoijsapdofijsadf')
# => {"message"=>"Not Found", "documentation_url"=>"https://developer.github.com/v3/users/#get-a-single-user"}

returns false if unsuccessful and instance if successful

if req = GithubCLI::User.call(username: 'arturictus')
  req.body # => {"login"=>"arturictus", "id"=>1930175, ...}
else
  # Handle error
end

Raises error if unsuccessful, returns instance if successful

begin
  req = GithubCLI::User.call!(username: 'arturictus')
  req.body # => {"login"=>"arturictus", "id"=>1930175, ...}
rescue => Wrappi::UnsuccessfulResponse
  # Handle error or not
end

The error:

GithubCLI::User.call!(username: 'sdfsdfasdjfojaspdjfpsajdpfoijsapdofijsadf')
# Wrappi::UnsuccessfulResponse ()
#     raw_body: {"message":"Not Found","documentation_url":"https://developer.github.com/v3/users/#get-a-single-user"}
#     code: 404
#     uri: https://api.github.com/users/sdfsdfasdjfojaspdjfpsajdpfoijsapdofijsadf
#     success: false

Wrappi comes with a background Job out of the box. If you are in a Rails app the #async method will queue a new job (< ActiveJob::Base) that will make the request and trigger the async callback after the request is made.

example:

class User < Wrappi::Endpoint
  client Client
  verb :get
  path "users/:username"
  async_callback do |opts|
    # this will be called in background after the request is made
    if success?
      if opts[:create]
        CreateUserService.call(body)
      elsif opts[:update]
        UpdateUserService.call(body)
      end
    end
  end
end
# This will execute the request in a background job
Github::User.new(username: 'arturictus').async(create: true)

If you need to send options to your Job (the ::set method) you can pass the key set to the options.

Github::User.new(username: 'arturictus').async(create: true, set: { wait: 10.minutes })

You can enable cache per endpoint. It depends on ::success? method to determine if it will be cached or nor.

Set the cache Handler in your client. It must behave like Rails.cache and respond to:

• read([key])
• write([key, value, options])

class Client < Wrappi::Client
  setup do |config|
    config.domain = 'https://api.github.com'
    config.cache = Rails.cache
  end
end

Enable cache in your endpoint.

class User < Wrappi::Endpoint
  cache true # enable for endpoint
  client Client
  verb :get
  path "users/:username"
end

user = User.new(username: 'arturictus')
user.response.class # => Wrappi::Response
user.flush
user.response.class # => Wrappi::CachedResponse
user.success? # => true
user.body # => {"login"=>"arturictus", "id"=>1930175, ...}

When cached the response will be a Wrappi::CachedResponse. Wrappi::CachedResponse behaves like Wrappi::Response that means you can use the endpoint in the same way as it was a non cached. See cache_options to fine tune your cache with expiration and other cache options.

You can use options to cache a single request.

class User < Wrappi::Endpoint
  client Client
  verb :get
  path "users/:username"
end
User.new({username: 'arturictus'}, cache: true)
user.response.class # => Wrappi::Response
user.flush
user.response.class # => Wrappi::CachedResponse
user.success? # => true
user.body # => {"login"=>"arturictus", "id"=>1930175, ...}

Sometimes you want to retry if certain conditions affected your request.

This will retry if status code is not 200

  class User < Wrappi::Endpoint
    client Client
    verb :get
    path "users/:username"
    retry_if do |response|
      response.code != 200
    end
  end

Check more configuration options and examples for retry_if and retry_options below.

options:

Pass a second argument with options.

params = { username: 'arturictus' }
options = { options_in_my_instance: "yeah!" }

User.new(params, options)

Dynamic configurations:

All the configs in Endpoint are evaluated at instance level except: around_request and retry_if because of their nature. That allows you to fine tune the configuration at a instance level.

example:

Right now the default for cache config is: proc { options[:cache] }.

  class User < Wrappi::Endpoint
    client Client
    verb :get
    path "users/:username"
    cache do
      if input_params[:username] == 'arturictus'
        false
      else
        options[:cache]          
      end
    end
  end

endpoint is a ruby class: 😮

  class User < Wrappi::Endpoint
    client Client
    verb :get
    path "users/:username"
    cache do
      cache?
    end

    def cache?
      if input_params[:username] == 'arturictus'
        false
      else
        options[:cache]          
      end
    end

    def parsed_response
      @parsed_response ||= MyParser.new(body)
    end
  end

inheritance: All the configs will be inherited

class UserDetail < User
  path "users/:username/detail"
end

Is the main configuration for your service.

It holds the common configuration for all the endpoints (Wrappi::Endpoint).

• domain: Yep, you know.

  config.domain = 'https://api.github.com'

• params: Set global params for all the Endpoints. This is a great place to put the api_key.

  config.params = { "api_key" => "asdfasdfoerkwlejrwer" }

  default: {}

• logger: Set your logger.

  default: Logger.new(STDOUT)

  config.logger = Rails.logger

• headers: Headers for all the endpoints. Format, Authentication.

  default:

  { 'Content-Type' => 'application/json', 'Accept' => 'application/json' }

  config.headers = {
    "Content-Type" => "application/json",
    "Accept' => 'application/json",
    "Auth-Token" => "verysecret"
  }

• async_handler: If you are not in Rails app or you have another background mechanism in place you can configure here how the requests will be send to the background. When #async is called on an Endpoint instance the async_handler const will be called with: current endpoint instance (self) and the options passed to the async method.

  class MyAsyncHandler
    def self.call(endpoint, opts)
      # send to background
    end
  end
  class Client < Wrappi::Client
    setup do |config|
      config.domain = 'https://api.github.com'
      config.async_handler = MyAsyncHandler
    end
  end
  endpoint_inst.async(this_opts_are_for_the_handler: true)

• timeout: Set your specific timout. When you set timeout it will be merged with defaults.

  default: { write: 9, connect: 9, read: 9 }

    class Client < Wrappi::Client
      setup do |config|
        config.domain = 'https://api.github.com'
        config.timeout = { read: 3 }
      end
    end
    Client.timeout # => { write: 9, connect: 9, read: 3 }

• use_ssl_context: It has to be set to true for using the ssl_context

  default: false

• ssl_context: If you need to set an ssl_context.

  default: nil

  config.ssl_context = OpenSSL::SSL::SSLContext.new.tap do |ctx|
                         ctx.verify_mode = OpenSSL::SSL::VERIFY_NONE
                       end

• client: Wrappi::Client class

    client MyClient

• path: The path to the resource. You can use doted notation and they will be interpolated with the params

    class MyEndpoint < Wrappi::Endpoint
      client MyClient
      verb :get
      path "/users/:id"
    end
    endpoint = MyEndpoint.new(id: "the_id", other: "foo")
    endpoint.url_with_params #=> "http://domain.com/users/the_id?other=foo"
    endpoint.url #=> "http://domain.com/users/the_id"
    endpoint.consummated_params #=> {"other"=>"foo"}

  Notice how interpolated params are removed from the query or the body

• verb:

  default: :get

  • :get
  • :post
  • :delete
  • :put

• default_params: Default params for the request. This params will be added to all the instances unless you override them.

  default: {}

  class MyEndpoint < Wrappi::Endpoint
    client MyClient
    verb :get
    path "/users/:id"
    default_params do
      { other: "bar", foo: "foo" }
    end
  end
  endpoint = MyEndpoint.new(id: "the_id", other: "foo")
  endpoint.consummated_params #=> {"other"=>"foo","foo" => "foo" }

• headers: You can modify the client headers here. Notice that if you want to use the client headers as well you will have to merge them.

  default: proc { client.headers }

  class MyEndpoint < Wrappi::Endpoint
    client MyClient
    verb :get
    path "/users"
    headers do
      client.headers #=> { 'Content-Type' => 'application/json', 'Accept' => 'application/json' }
      client.headers.merge('Agent' => 'wrappi')
    end
  end
  endpoint = MyEndpoint.new()
  endpoint.headers #=> { 'Agent' => 'wrappi', 'Content-Type' => 'application/json', 'Accept' => 'application/json'}

• basic_auth: If your endpoint requires basic_auth here is the place. keys have to be: user and pass.

  default: nil

    basic_auth do
      { user: 'wrappi', pass: 'secret'}
    end

• follow_redirects: If the request responds with a redirection it will follow them.

  default: true

• body_type: Body type.

  default: :json

  • :json
  • :form
  • :body (Binary data)

• async_callback: When request is executed in the background with #async(opts = {}) this callback will be called with this opts as and argument in the block. The block is executed in the endpoint instance. You can access to all the methods in Endpoint.

  default: proc {}

  async_callback do |opts|
    if success?
      MyCreationService.call(body) if opts[:create]
    end
  end
  MyEndpoint.new().async(create: true)

This configs allows you fine tune your request adding middleware, retries and cache. The are executed in this nested stack:

  cache
    |- retry
      |- around_request

Check specs for more examples.

• cache: Cache the request if successful.

  default: proc { options[:cache] }

• cache_options: Options for the cache to receive on write

  cache_options do
    { expires_in: 12, another_opt: true }
  end

default: {}

• retry_if: Block to evaluate if request has to be retried. In the block are yielded Response instance. If the block returns true the request will be retried.

    retry_if do |response|
      response.status != 200 # => true or false
    end

  Use case:

  We have a service that returns an aggregation of hotels available to book for a city. The service will start the aggregation in the background and will return 200 if the aggregation is completed if the aggregation is not completed will return 201 making us know that we should call again to retrieve all the data. This behavior only occurs if we pass the param: onlyIfComplete.

    retry_if do |response, endpoint|
      endpoint.consummated_params["onlyIfComplete"] &&
        response.status_code == 201
    end

  Notice that this block will never be executed if an error occur (like timeouts). For retrying on errors use the retry_options

• retry_options: We are using the great gem retryable to accomplish this behavior. Check the documentation for fine tuning. I just paste some examples for convenience.

  retry_options do
    { tries: 5, on: [ArgumentError, Wrappi::TimeoutError] } # or
    { tries: :infinite, sleep: 0 }
  end

• around_request: This block is executed surrounding the request. The request will only get executed if you call request.call.

  around_request do |request, endpoint|
    endpoint.logger.info("making a request to #{endpoint.url} with params: #{endpoint.consummated_params}")
    request.call # IMPORTANT
    endpoint.logger.info("response status is: #{request.status_code}")
  end

Wrappi is designed to be able to build HTTP client gems with it.

module GithubCLI
  class Client < Wrappi::Client
    setup do |config|
      config.domain = 'https://api.github.com'
      config.headers = {
        'Content-Type' => 'application/json',
        'Accept' => 'application/vnd.github.v3+json',
      }
    end

    class << self
      attr_accessor :my_custom_config
    end
  end

  def self.setup
    yield(Client)
  end

  class Endpoint < Wrappi::Endpoint
    client Client
  end

  class User < Endpoint
    verb :get
    path "users/:username"
  end

  def self.user(params, opts = {})
    User.new(params, opts)
  end
end

user = GithubCLI.user(username: 'arturictus')
user.success?

Once you created a gem Wrappi allows to parent projects to customize endpoints without having to change the gem's code.

example customizing GithubCLI::User

GithubCLI::User.setup do
  cache true
  async_callback do |opts|
    if success?
      # do something
    end
  end
end

Example customizing all the Endpoints, adding loging to all the requests and changing client depending of enviroment:

GithubCLI::Endpoint.setup do
  client do
    if ENV['production']
      GithubCLI::Client
    else
      GithubCLI::MyStagingClient
    end
  end

  around_request do |request, endpoint|
    endpoint.logger.info("making a request to #{endpoint.url} with params: #{endpoint.consummated_params}")
    request.call # IMPORTANT
    endpoint.logger.info("response status is: #{request.status_code}")
  end
end

In ruby there are many ruby clients an everyone has an opinion of which one is the best. Every new API client that you install in your project will install a different HTTP client adding redundant and unnecessary dependencies in your project. That's why Wrappi is designed to be HTTP client agnostic. Right now is implemented with HTTP gem (my favorite) but all the logic is decoupled from the HTTP client.

All the configuration, metadata and logic to build the request is hold by an instance of Endpoint. Allowing to create adapters that translates this processed metadata to the target HTTP client.

Tests are HTTP client agnostic. To help the development of these adapters and probe the reliability of the gem most of the test are run against a Rails application. All the tests that probe an HTTP call are running this HTTP call against a local server making all test End To End and again, HTTP client agnostic.

Right now is not designed the system to change HTTP clients via configuration but if you are interested to implement one let me know and we will figure out the way.

After checking out the repo, run bin/setup to install dependencies.

bin/dev_server

This will run a rails server. The test are running against it.

bundle exec rspec

You can also run bin/console for an interactive prompt that will allow you to experiment.

Run dummy server with docker:

docker build -t wrappi/dummy -f spec/dummy/Dockerfile .
docker run -d -p 127.0.0.1:9873:9873 wrappy/dummy /bin/sh -c "bin/rails server -b 0.0.0.0 -p 9873"

Try:

curl 127.0.0.1:9873 #=> {"controller":"pages","action":"show_body"}

Bug reports and pull requests are welcome on GitHub at https://github.com/arturictus/wrappi. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the Contributor Covenant code of conduct.

The gem is available as open source under the terms of the MIT License.