CopyBot

A gem that provides a rake task for importing a database dump into the local database, as well as for running any additional scripts afterwards.

This can be useful in situations when you want to sync your staging and production data so that you can debug issues in production without having to access the production environment itself. Also, it is useful for populating a database with production data so development and testing can be performed in circumstances that are as close to production as possible. It is advised that the production data is anonymized beforehand.

The gem assumes that the database dump that is to be restored to the local database has already been prepared and made available either on an AWS S3 bucket or on a local path.

• aws-sdk-ruby
• rails

• PostgreSQL

Add this line to your application's Gemfile:

gem 'copy_bot'

And then execute:

$ bundle install

Or install it yourself as:

$ gem install copy_bot

Add config/initializers/copy_bot.rb

CopyBot.setup do |config|
  config.logger = Rails.logger
  config.permitted_environments = ['development', 'staging']
end

It is necessary to define steps that should be run in a .yml file:

steps:
  download_remote_db_dump:
    s3_credentials:
      access_key_id: <%= Rails.application.secrets.aws_access_key_id %>
      access_key: <%= Rails.application.secrets.aws_secret_access_key %>
      region: <%= Rails.application.secrets.aws_region %>
      bucket: <%= Rails.application.secrets.aws_s3_bucket %>
    source_file_path: '/staging_db_dump.sql'
    destination_file_path: './tmp/downloaded_db_dump.sql'
  create_local_db_backup:
    destination_file_path: './tmp/development_backup.sql'
  drop_local_db_tables:
  import_remote_db_to_local_db:
    remote_db_dump_file_path: './tmp/downloaded_db_dump.sql'
  run_migrations_on_local_db:
  delete_remote_db_dump:
    remote_db_dump_file_path: './tmp/downloaded_db_dump.sql'
  execute_custom_command:
    command: 'RAILS_ENV=staging bundle exec rake some_task'

The example above contains all possible steps in the exact order in which they are run. Not all steps are mandatory. If a certain step doesn't need to be run, just omit it from the steps definition file. For example, you may not want to download the database dump from an S3 bucket, but you have acquired it in some other way and you have it available locally. In that case, omit the download_remote_db_dump step and add the relevant remote_db_dump_file_path to the import_remote_db_to_local_db step.

IMPORTANT: These steps include some destructive actions, e.g. drop_local_db_tables will drop all tables in the local database so make sure you don't use this gem in an environment where you do not want to lose data.

To use the gem, simply run:

$ bundle exec rake copy_bot STEP_DEFINITIONS_FILE_PATH=config/copy_bot/step_definitions.yml

Please note that the step definitions file is passed to the rake task as an environment variable, not an argument.

After checking out the repo, run bin/setup to install dependencies. Then, run rake spec to run the tests. You can also run bin/console for an interactive prompt that will allow you to experiment.

To install this gem onto your local machine, run bundle exec rake install. To release a new version, update the version number in version.rb, and then run bundle exec rake release, which will create a git tag for the version, push git commits and the created tag, and push the .gem file to rubygems.org.

Bug reports and pull requests are welcome on GitHub at https://github.com/infinum/copy_bot. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the code of conduct.

The gem is available as open source under the terms of the MIT License.

Everyone interacting in the CopyBot project's codebases, issue trackers, chat rooms and mailing lists is expected to follow the code of conduct.