communicator¶ ↑

A simple communication layer to keep database records in sync between two Rails apps.

The idea is that when you have a central application in a private network and a facade in a public network, the facade should not need to access the central application directly. Rather, the central application should manage communications for both applications. That is what Communicator does for you.

Install in Rails apps by adding this to your Gemfile:

gem 'communicator'

Import rake tasks in your Rails.root/Rakefile:

require 'communicator/tasks'

Set up the database tables for the messages by importing the corresponding migrations from the gem:

$ rake communicator:update_migrations
$ rake db:migrate

After doing these steps, you can proceed to configure either the server or client. On both sides, you’ll have to configure the client and server. You can do so by placing a configuration file in Rails.root/config/communicator.yml that looks somewhat like this:

defaults: &defaults
  username: communicator
  password: verysecret
  base_uri: localhost:3001 # Can be ommitted for server config

test:
  <<: *defaults

development:
  <<: *defaults

production:
  <<: *defaults
  password: Uber!Secret
  base_uri: 192.168.0.104:4000 # Can be ommitted for server config

The configuration for the current ‘Rails.env` will then be automatically loaded when you `require ’communicator’‘.

You can also skip the YAML configuration and set up the credentials programmatically:

Communicator.name = 'facade'
Communicator.username = 'foo'
Communicator.password = 'bar'

Communicator::Client.base_uri 'localhost:1234'

On the server side, you can mount the server component in a Rails 2.3 app as a rack middleware inside environment.rb:

require 'communicator'
config.middleware.use 'Communicator::Server'

It will then be mounted at /messages.json for GET and POST requests, requesting HTTP Basic Auth with the configured credentials.

Tell the client the server url and port as well as the auth credentials inside your environment.rb:

require 'communicator'

When everything’s fine, you should be able to push and pull using the client:

Communicator::Client.push
Communicator::Client.pull

To keep models in sync, specify the model class (in underscored type) the local class receives updates from like this:

class Post < ActiveRecord::Base
  receives_from :post
end

If you want to skip some remote attributes and not let them progapate into local instances, use ‘receives_from :post, :except => [:user_id]`.

Attributes that are not physically present in the local database will be skipped automatically.

Use the automatically added ‘publish` instance method on your models to push changes to the other side. They will be enqueued in the local `outbound_messages` table and will arrive at the other side when a client pulls or pushes.

When an update is received from a remote side, the message will be stored in ‘inbound_messages` and the received changes will be applied to the local record, either updating existing or creating new records.

Use a cron job or whatever else you use for scheduling tasks in your system. There is a rake task that does what you usually need:

rake communicator:communicate

Frequency of execution depends on your application’s needs, of course. For us, every other minute in production (and every minute in acceptance testing) works just fine.

When the structure of one of the models whose data you transfer changes or when Communicator itself changes, the following steps should provide a safe path forward:

1. Unschedule the rake task.

2. Stop your applications, both the facade and the central application. This ensures that no new messages get added to your inbound and outbound queues.

3. Manually execute push and pull until no further pending messages exist. Now we have reached a point where the system is temporarily quiescent and (message-)stable.

4. Optionally remove all messages from the queues. You might wish not to leave any messages around whose format cannot be processed automatically any more. On the other hand, you might wish to keep them available for reference. Your call.

5. Upgrade Communicator.

6. Excute the rake task for copying migrations (if any) in both of your applications: rake communicator:update_migrations

7. Upgrade your applications. If any migrations were copied in step 6, execute them.

8. Reschedule the rake task for automation.

9. Start up your applications.

This is actually a lot less complicated than it looks at first glance. The point is that if any of the underlying structures changes, you cannot have any messages in flight, because you would not be able to process them any more after the change. So you need to make sure all pending messages have been processed and no new ones are being generated.

Communicator is currently only proven to work on Rails 2 apps using either REE, 1.8.7, 1.9.1 or 1.9.2.

• Fork the project.

• Make your feature addition or bug fix.

• Add tests for it. This is important so I don’t break it in a future version unintentionally.

• Commit, do not mess with rakefile, version, or history. (if you want to have your own version, that is fine but bump version in a commit by itself I can ignore when I pull)

• Send me a pull request. Bonus points for topic branches.

Copyright © 2010 Capita Unternehmensberatung GmbH. See LICENSE for details.