Rack::Prerender

This is a fork and drop-in replacement for prerender_rails with some improvements, most notably improved performance for non-crawlers.

Add it to your gemfile or run

gem install rack-prerender

The public interface is fully compatible with prerender_rails, so see its readme.

The middleware class name (Rack::Prerender) and options are the same.

Only additive changes will be made to the interface, see Improvements below.

If you need to require the gem manually: require 'rack/prerender'

About 0.07ms instead of 0.5ms lost per request in my case. See benchmark.rb.

You can call the service manually:

• Rack::Prerender.fetch(my_url) (uses token etc. from middleware setup or ENV)
• or Rack::Prerender.fetch(my_url, prerender_token: token, ...)
• you can also pass a Request or env or ActiveRecord/Model to #fetch:
• Rack::Prerender.fetch(User.last) (assuming there is a route for user)

You can use your own constraint or fetcher:

• rack_prerender_instance.constraint = MyBotConstraint.new

Recaching / cache-busting functionality:

• Rack::Prerender.recache_later(my_url) (async, requires ActiveJob or Sidekiq)
• Rack::Prerender.recache_now(my_url) (sync)
• both use token from middleware setup or ENV and default API URL, to override:
• Rack::Prerender.recache_now(my_url, prerender_recache_url: my_api_url, ...)

More options for constraints:

• you can pass a single Regexp as whitelist or blacklist
• (prerender_rails supports only a String, or an Array of Strings or Regexps)
• you can pass Regexp(s) for crawler_user_agents and extensions_to_ignore

Directly works with the param-based URL of the prerender node app:

• e.g. PRERENDER_SERVICE_URL=https://my-service.com/render?url=

• correct Content-Length for compressed pages with multibyte chars
• removes Transfer-Encoding header after result decompression
• supports recent (non-vulnerable) versions of Rack
• works without undeclared dependency activesupport

Pull requests and suggestions are welcome.