GOV.UK Sidekiq
Provides a unified set of configurations and behaviours when using Sidekiq in GOV.UK applications.
Features
What does govuk_sidekiq
do for you?
- Makes sure Sidekiq can connect to Redis correctly, using the default environment variables (these are set in govuk-puppet).
- Makes sure that the correct HTTP headers are passed on to gds-api-adapters.
This means that for each request a unique ID (
govuk_request_id
) will be passed on to downstream applications. Read more about request tracing. - Makes sure that we use JSON logging, so that Sidekiq logs will end up properly in Logit and searchable through Kibana.
Installation (Rails only)
1. Add the gem
# Gemfile
gem "govuk_sidekiq"
2. Add a Sidekiq config file
# config/sidekiq.yml
---
:concurrency: 2
This file also allows you to configure queues with priority. See the Sidekiq wiki for available options.
3. Configure environment variables in EKS
For each environment (integration, staging and production), add a REDIS_URL
environment variable for your application with a value of redis://shared-redis-govuk.eks.{environment}.govuk-internal.digital
.
Additionally, set the value of workerEnabled
to true
for your application. This will result in a worker
process running alongside the web application. The queue length and max delay can be monitored using the Sidekiq Grafana dashboard, once the Sidekiq worker is initialised.
There's no step-by-step guide for this, but you can copy the changes made when Sidekiq was added to the release application. You may also want to resize resource requests for the app depending on the predicted request rate.
# your-application/config/deploy.rb
after "deploy:restart", "deploy:restart_procfile_worker"
4. Create some jobs
You can use normal Sidekiq jobs:
# app/workers/hard_worker.rb
class HardWorker
include Sidekiq::Worker
def perform(name, count)
# do something
end
end
Note that the convention is to use app/workers
as the directory for your workers.
Testing
See Sidekiq testing documentation on how to test Sidekiq workers.
Because of the way we use middleware, you may see errors that indicate that
your job is called with the wrong number of arguments. To set up testing
correctly, replace require 'sidekiq/testing'
with:
require 'govuk_sidekiq/testing'