A work queue is designed to coordinate work between a producer and a pool of worker threads. When some task needs to be performed, the producer adds an object containing the task routine to the work queue. If the work queue is full, the producer will block until a worker thread removes an object from the queue. Eventually, one of the worker threads removes the object from the work queue and executes the routine. If the work queue is empty, a worker thread will block until an object is made available by the producer.

Work queues are useful for several reasons:

• To easily perform tasks asynchronously and concurrently in your application;

• To let you focus on the work you actually want to perform without having to worry about the thread creation and management;

• To minimize overhead, by reusing previously constructed threads rather than creating new ones;

• To bound the resources used, by setting a limit on the maximum number of simultaneously executing threads;

Install the gem:

gem install work_queue

Run the code:

require 'rubygems'
require 'work_queue'
wq = WorkQueue.new
wq.enqueue_b { puts "Hello from the WorkQueue" }
wq.join

It’s generally recommended to bound the resources used:

# Limit the maximum number of simultaneous worker threads
WorkQueue.new 10, nil
# Limit the maximum number of queued tasks
WorkQueue.new nil, 20

Download multiple files in parallel:

require 'open-uri'
require 'rubygems'
require 'work_queue'

wq = WorkQueue.new 5

(1..6605).each do |number|
  wq.enqueue_b do
    open("rfc#{number}.txt", "wb") do |file|
      file.write open("http://www.rfc-editor.org/rfc/rfc#{number}.txt").read
      puts "rfc#{number} downloaded"
    end
  end
end

wq.join

Released under the MIT license.