LineUp
With LineUp you can enqueue Resque jobs in arbitrary Redis namespaces.
Resque uses a Set and a List to keep track of all Queues and their Jobs.
- The Set is usually located at
resque:queues
and contains a list of (lower-cased, underscored) Strings, each representing a queue name - Each queue is a List located at
resque:queue:my_job
(withmy_job
as queue name forMyJob
-jobs in this example) - Each job inside of a queue is a JSON or Marshal'ed Hash with the keys
class
andargs
, for example{ class: 'MyJob', args: [123, some: thing] }.to_json
Depending on how you configure the Redis backend for Resque, you will end up in a different namespace:
- If you use
Resque.redis = Redis::Namespace.new(:bob, ...)
, Resque detects that you passed in anRedis::Namespace
object and will not add any additional namespace. So the queue Set in this example will be located atbob:queues
- Any other
Redis.new
-compatible object will get theresque
-namespace added. SoResque.redis = Redis.new(...)
will cause the queue Set to be located atresque:queues
If you use multiple applications, you should make sure that each of them has its own namespace. You would normally achieve that with Resque.redis = Redis::Namespace.new('myapp:resque', ...)
so that the queue Set would be located at myapp:resque:queues
.
So far so good, but there is no way to enqueue a Job for an application from inside another namespace, say otherapp:resque:queues
, without maintaining an additional connection to Redis in the other app's namespace. So far, the only solution has been to share the resque:queues
namespace between all applications and have separate queue-names, such as myapp-myjob
and otherapp-myjob
, but that is not really separating namespaces.
That's where LineUp comes in, it doesn't even need Resque. It goes right into Redis (scary huh?), just as Resque does internally.
Examples
Setup
If you use the Raidis
gem, you do not need any setup. Otherwise, a manual setup would look like this:
redis = Redis::Namespace.new 'myapp:resque', redis: Redis.new(...)
Resque.redis = redis
LineUp.redis = redis
Usage
With the setup above, Resque lies in the myapp:resque
-namespace. So you can enqeueue jobs to the very same application by using Resque.enqueue(...)
.
This is how you can enqueue a job for another applications:
LineUp.push :otherApp, :SomeJob, 12345, some: thing
This will enqueue to other_app:resque:some_job
with arguments [12345, { 'some' => 'thing' }]
and make sure that the other_app:resque:queues
Set references the queue List.
Gotchas
-
Resque.redis
MUST respond to a method called#namespace
which takes a block and yields a new namespace. See this commit. Currently LineUp requires a non-rubygems fork ofRedis::Namespace
in order to be able to use this cutting-edge method. - Currently the jobs are encoded using
MultiJson
only, notMarshal
, feel free to commit a patch if you need the latter - You cannot share the
resque
root namespace. LineUp defaults to theapplication:resque
namespace (because that's the only scenario I can think of that would make you want to use LineUp in the first place :)