Resque::Plugins::SerialQueues
This gem is tested on the latest version from Resque 1.x.
Declaring resque queues as serial. Jobs from a serial queue won't be processed more than one at a time event if you have multiple workers / servers. There are similar solutions but none of them worked for me. To lock jobs from running in parallel I used Redis#setnx
. This implementation has a few sensitive things:
-
Resque::Job.reserve
is overriden (here the queue locking is done) - Added
Object#after_perform_unlock_queue_if_serial
(here the queue is unlocked) - Added
Object#on_failure_unlock_queue_if_serial
(here the queue is unlocked)
I think serial queues are not a scalable. You shouldn't have to have serial queues. This should be a temporary drop-in solution without changing the deployment process. If you really need serial job processing you should start a worker for each serial queue that will listen for jobs only on that queue or migrate to a background processing / messaging tool that provides serial processing out of the box.
Installation
Add this line to your application's Gemfile:
gem 'resque-serial-queues'
And then execute:
$ bundle
Or install it yourself as:
$ gem install resque-serial-queues
Usage (Rails)
Create a .rb
file in config/initializers
Resque::Plugins::SerialQueues.configure do |config|
config.serial_queues = [:serial_jobs]
config.lock_timeout = 120 #default 3600 seconds
end
and use the declared serial queues in your jobs:
class MyJob
@queue = :serial_jobs
def perform(*)
end
end
Contributing
- Fork it ( https://github.com/cristianbica/resque-serial-queues/fork )
- Create your feature branch (
git checkout -b my-new-feature
) - Commit your changes (
git commit -am 'Add some feature'
) - Push to the branch (
git push origin my-new-feature
) - Create a new Pull Request