mongar¶ ↑

Replicate data from MySQL, PostgreSQL, SQLite, etc to MongoDB!

Mongar will replicate data from ActiveRecord (or any data represented by a Ruby class) to MongoDB. For example, if you have MySQL data represented by ActiveRecord, you can easily replicate it to MongoDB.

Adding to a rails project¶ ↑

For now, you’ll have to manually add Mongar to your Rails project. We’ll work on a Railtie for Rails 3 soon.

Put your Mongar config into config/mongar.rb (See Example Config below)

Put the following code into your Rakefile

require 'mongar'

namespace :mongar do
  desc "Run mongar replication"
  task :run => :environment do
    mongar_config = File.join(Rails.root, 'config', 'mongar.rb')
    mongar = eval(File.read(mongar_config))
    mongar.run
  end
end

Run it with

rake mongar:run

Assumptions¶ ↑

The default configuration assumes that you are using ActiveRecord and that are you using a plugin similar to acts_as_paranoid that never deletes records from the database but only marks them as deleted with a deleted_at column.

You can customize the finder methods that Mongar uses to accommodate any data model or your own custom scopes.

Minimal example config¶ ↑

Given an ActiveRecord model call Domain with two attributes name and client_id:

Mongar.configure do
  mongo :default do
    database 'mydb'
  end

  replicate Domain do
    column :name do
      primary_index
    end

    column :client_id
  end
end

Full example Config¶ ↑

Mongar.configure do
  # Currently we only log to STDOUT, we have plans to use a
  # plans to use a configurable logger.
  # Valid log levels are :info and :debug
  log_level :debug

  mongo :default do
    database 'mydb'
  end

  mongo :otherdb do
    database 'myotherdb'
    user 'mongouser'
    password 'password'
    host '127.0.0.1'
    port 27017

    # By default Mongar uses the 'statuses' collection to
    # keep track of the last replicated times, you can specify
    # a custom collection name
    status_collection :stati
  end

  # Minimal replica config to replicate the Domain model
  # to the 'domains' mongo collection on the mongo db defined
  # in the :default config above

  replicate Domain do
    column :name do
      primary_index
    end

    column :client_id
  end

  # Customized replica config to replicate the Client
  # model to the 'customers' mongo collection on the db
  # defined in the :otherdb config above as well as the
  # default database
  replicate Client => [{ :otherdb => 'clients' }, 'clients'] do
    # By default, Mongar will try to determine the time on the
    # backend database. The supported ActiveRecord database adapters
    # are MysqlAdapter, Mysql2Adapter, and SQLServerAdapter.
    # You can use a custom function to determine the time on Client's
    # backend database
    db_time_selector do
      # this will run Client#get_db_time
      get_db_time
    end

    # Items are never deleted
    no_deleted_finder

    # Customize your updated record finder.  The block is
    # run in the source class context
    set_updated_finder do |last_replicated_date|
      find(:all, :conditions => ['something > ?', last_replicated_date])
    end

    # Custom created item finder
    set_created_finder do |last_replicated_date|
      created_scope(last_replicated_date)
    end

    # Run a full refresh of all items in the database based
    # a condition returned by the Proc
    full_refresh :if => Proc.new do |last_replicated_date|
      # class eval'ed code
      any_changes_since?(last_replicated_date)
    end

    # You can also refresh every N seconds
    # full_refresh :every => 3600

    # Define your columns
    column :id do
      # The 'primary index' column is used to find items in the mongo collection
      primary_index
    end

    column :tag do
      # Run the procedure :downcase on the value of Client#tag
      transform :downcase
    end

    column :employee_count do
      # Run a block transformation on the value of Client#employee_count
      transform do |value|
        value.nil? ? 0 : value
      end
    end
  end
end

Contributing to mongar¶ ↑

Find this project on Pivotal Tracker here: www.pivotaltracker.com/projects/434475

Check out the latest master to make sure the feature hasn’t been implemented or the bug hasn’t been fixed yet
Check out the issue tracker to make sure someone already hasn’t requested it and/or contributed it
Fork the project
Start a feature/bugfix branch
Commit and push until you are happy with your contribution
Make sure to add tests for it. This is important so I don’t break it in a future version unintentionally.
Please try not to mess with the Rakefile, version, or history. If you want to have your own version, or is otherwise necessary, that is fine, but please isolate to its own commit so I can cherry-pick around it.

mongar

Development