0.0
Repository is archived
No release in over 3 years
Low commit activity in last 3 years
A library to iterate over entire Mongo collections or large queries exposing an API to control things like batch size, order and limit.
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
2017
2018
2019
2020
2021
2022
2023
2024
 Dependencies

Development

~> 1.10
~> 5.0
~> 10.0
~> 3.3
 Project Readme

THIS PROJECT IS DEPRECATED

mongo-batch is not maintained anymore.

MongoBatch Build Status

A Ruby library to run Mongoid queries on large collections in batches.

Supported Ruby versions: 2.1, 2.2
Supported Mongoid versions: 4.0, 5.0

Installation

gem install mongo_batch -v '~> 0.1'

Or, in your Gemfile:

gem 'mongo_batch', '~> 0.1'

Usage

Extend your Mongoid models with MongoBatch to be able to call find_in_batches on your models. The method will yield each batch of records to the given block.

require 'mongo_batch'

class Post
  extend MongoBatch
end

Post.find_in_batches do |batch|
  batch.each do |post|
    post.update(body: 'Hello world!')
  end
end

If you do not pass a block to find_in_batches, the method will return an Enumerator.

Post.find_in_batches.with_index.each do |batch, index|
  batch.each do |post|
    post.update(body: "Hello world! #{index}")
  end
end

The default batch size is 1,000 records, but find_in_batches accepts an option to configure a different batch size, as well as options to limit the records to process, sorting criteria and an initial offset.

Post
  .find_in_batches(batch_size: 500, to: 2_000, offset: 100, order_by: { _id: :desc })
  .each do |batch|
    batch.each do |post|
      post.update(body: "Hola mundo!")
    end
end

If you have more complex queries or prefer not to extend your models with MongoBatch, you can use MongoBatch.in_batches and supply the query you want to batch.

MongoBatch
  .in_batches(Post.where(:body.exists => true).no_timeout)
  .each do |batch|
    batch.each do |post|
      post.update(body: 'Hello world!')
    end
end

MongoBatch.in_batches also accepts values to configure the batch size, limit of records to process, sorting criteria or an initial offset.

MongoBatch
  .in_batches(Post.where(:body.exists => true).no_timeout,
              to: 2_000, offset: 100, order_by: { _id::desc })
  .each do |batch|
     batch.each do |post|
       post.update(body: 'Hi mum!')
     end
  end

License

Copyright © 2015 Altmetric LLP

Distributed under the MIT License.