sidekiq_utils
Sidekiq powers our background processing needs at AngelList. As we introduced Sidekiq in a legacy codebase and it started to handle significant job throughput, we developed a number of utilities to make working with Sidekiq easier that we'd love to share.
Additional Serialization
Adds support for automatically serializing and deserializing additional argument types for your workers:
Class-
Symbol(both by themselves and as hash keys) ActiveSupport::HashWithIndifferentAccess
To use this utility, you need to include the client and server middlewares. Be sure to include the client middleware on the server as well.
Configuration
Sidekiq.configure_server do |config|
config.server_middleware do |chain|
chain.add ::SidekiqUtils::Middleware::Server::AdditionalSerialization
end
config.client_middleware do |chain|
chain.add ::SidekiqUtils::Middleware::Client::AdditionalSerialization
end
end
Sidekiq.configure_client do |config|
config.client_middleware do |chain|
chain.add ::SidekiqUtils::Middleware::Client::AdditionalSerialization
end
end
A note of caution for users of the sidekiq-unique-jobs gem
sidekiq-unique-jobs will fail to properly determine job uniqueness if you don't hook the middlewares in the correct order. If you use both sidekiq-unique-jobs and the additional serialization utility, be sure to hook your middlewares in the following way:
Sidekiq.configure_server do |config|
# unwrap arguments first
config.server_middleware do |chain|
chain.add ::SidekiqUtils::Middleware::Server::AdditionalSerialization
end
# then determine unique jobs
SidekiqUniqueJobs.configure_server_middleware
# first determine uniqueness
SidekiqUniqueJobs.configure_client_middleware
# then wrap arguments
config.client_middleware do |chain|
chain.add ::SidekiqUtils::Middleware::Client::AdditionalSerialization
end
end
Sidekiq.configure_client do |config|
# first determine uniqueness
SidekiqUniqueJobs.configure_client_middleware
# then wrap arguments
config.client_middleware do |chain|
chain.add ::SidekiqUtils::Middleware::Client::AdditionalSerialization
end
end
Deprioritize
Adds an easy ability to divert jobs added within a block to a lower-priority queue. This is useful, for example, in cron jobs. All jobs added within the block will be added to the low queue instead of their default queue.
Configuration
Sidekiq.configure_server do |config|
config.client_middleware do |chain|
chain.add SidekiqUtils::Middleware::Client::Deprioritize
end
end
Sidekiq.configure_client do |config|
config.client_middleware do |chain|
client_middlewares.each do |middleware|
chain.add SidekiqUtils::Middleware::Client::Deprioritize
end
end
end
Usage
SidekiqUtils::Deprioritize.workers(SolrIndexWorker) do
10_000.times do
# these will all go to the `low` queue
SolrIndexWorker.perform_async(User, rand(10_000))
end
end
Enqueued jobs helper
A simple tool to inspect and manipulate Sidekiq queues from the console:
> SidekiqUtils::EnqueuedJobsHelper.counts
=> {"default"=>{}, "high"=>{"AlgoliaIndexWorker[JobProfile]"=>1}, "low"=>{"AlgoliaIndexWorker[JobProfile]"=>1}}
> SidekiqUtils::EnqueuedJobsHelper.delete(queue: 'low', job_class: 'AlgoliaIndexWorker', first_argument: JobProfile)
# first_argument is optional
Find optional
Lots of jobs become moot when a record cannot be found; however, because Sidekiq is very fast and it's easy to forget to enqueue jobs in after_commit hooks, sometimes the record isn't found simply because the transaction in which it was inserted has not been committed yet. This will retry a find_optional call exactly once after 30 seconds if the record cannot be found.
Configuration
Sidekiq.configure_server do |config|
config.server_middleware do |chain|
chain.add SidekiqUtils::Middleware::Server::FindOptional
end
end
Usage
class SolrIndexWorker
include Sidekiq::Worker
def perform(user_id)
user = find_optional(User, user_id)
end
end
Latency monitor
This will monitor queue latency and report to Slack channels if the latency exceeds the configured threshold.
Configuration
Create a config/sidekiq_utils.yml file in your project:
repeat_alert_every: 60 # repeat identical alerts every x minutes
alert_thresholds: # in minutes
default: 10
high: 5
low: 60
slack:
username: 'Sidekiq alerts'
icon: 'alarm_clock'
team: 'foobar'
token: 'xxx'
channels_to_alert:
- "#devops-alerts"
- "#sidekiq-alerts"
Usage
Simply call SidekiqUtils::LatencyAlert.check! at regular intervals.
Throughput monitor
This will keep track of how many jobs of which worker class have run in the past week, as well as when it was last run.
Configuration
Sidekiq.configure_server do |config|
config.server_middleware do |chain|
chain.add SidekiqUtils::Middleware::Server::ThroughputMonitor
end
end
Sidekiq::Web.register SidekiqUtils::WebExtensions::ThroughputMonitor
Sidekiq::Web.tabs["Throughput"] = "throughput"
Usage
This will add a "Throughput" tab to your Sidekiq admin which will display job throughput information.