CronoTrigger
Asynchronous Job Scheduler for Rails.
The purpose of this gem is to integrate job schedule into Service Domain.
Because of it, this gem uses ActiveRecord model as definition of job schedule.
Installation
Add this line to your application's Gemfile:
gem 'crono_trigger'
And then execute:
$ bundle
Or install it yourself as:
$ gem install crono_trigger
Breaking Changes
Update from v0.6.x
In previous version, now is 2023-06-07T18:00:00+00:00
, a cron definition is 0 1 * * * *
, and started_at
is 2023-06-08T1:00:00+00:00
.
In this case, next_execute_at
was 2023-06-09T1:00:00+00:00
From v0.7.0, if the cron definition and started_at
match, include the time of started_at
as the next_execute_at
.
For example, now is 2023-06-07T18:00:00+00:00
, a cron definition is 0 1 * * * *
, and started_at
is 2023-06-08T1:00:00+00:00
.
In this case, next_execute_at
is 2023-06-08T1:00:00+00:00
.
If the current time is past started_at
, the next_execute_at
is based on the current time.
Update from v0.3.x
Create crono_trigger system tables
$ rails g crono_trigger:install # => create migrations
$ rake db:migrate
Add locked_by:string
column to CronoTrigger::Schedulable model
$ rails g migration add_locked_by_column_to_your_model
$ rake db:migrate
class AddLockedByColumnToYourModel < ActiveRecord::Migration[5.2]
def change
add_column :your_models, :locked_by, :string
end
end
Usage
Execute crono_trigger:model
generator.
$ rails g crono_trigger:model mail_notification
create db/migrate/20170619064928_create_mail_notifications.rb
create app/models/mail_notification.rb
# ...
Migration sample
class CreateMailNotifications < ActiveRecord::Migration
def change
create_table :mail_notifications do |t|
# columns for CronoTrigger::Schedulable
t.string :cron
t.datetime :next_execute_at
t.datetime :last_executed_at
t.integer :execute_lock, limit: 8, default: 0, null: false
t.datetime :started_at
t.datetime :finished_at
t.string :last_error_name
t.string :last_error_reason
t.datetime :last_error_time
t.integer :retry_count, default: 0, null: false
t.timestamps
end
add_index :mail_notifications, [:next_execute_at, :execute_lock, :started_at, :finished_at], name: "crono_trigger_index_on_mail_notifications"
end
end
Implement #execute
method
class MailNotification < ActiveRecord::Base
include CronoTrigger::Schedulable
self.crono_trigger_options = {
retry_limit: 5,
retry_interval: 10,
exponential_backoff: true,
execute_lock_timeout: 300,
}
# `execute`, `retry` callback is defined
# can use `before_execute`, `after_execute`, `around_execute`
# `before_retry`, `after_retry`, `around_retry`
# If execute method raise Exception, worker retry task until reach `retry_limit`
# If `retry_count` reaches `retry_limit`, task schedule is reset.
#
# If record has cron value, reset process set next execution time by cron definition
# If record has no cron value, reset process clear next execution time
def execute
send_mail
throw :retry # break execution and retry task
throw :abort # break execution
throw :ok # break execution and handle task as success
throw :ok_without_reset # break execution and handle task as success but without schedule reseting and unlocking
end
end
# one time schedule
MailNotification.create.activate_schedule!(at: Time.current.since(5.minutes))
# cron schedule
MailNotification.create(cron: "0 12 * * *").activate_schedule!
# or
MailNotification.new(cron: "0 12 * * *").activate_schedule!.save
# if update cron column or timezone column
# update next_execute_at automatically by before_update callback
mail = MailNotification.create(cron: "0 12 * * *").activate_schedule!
mail.next_execute_at # => next 12:00 with Time.zone
mail.update(cron: "0 13 * * *")
mail.next_execute_at # => next 13:00 with Time.zone
mail.update(timezone: "Asia/Tokyo")
mail.next_execute_at # => next 13:00 with Asia/Japan
Run Worker
use crono_trigger
command.
crono_trigger
command accepts model class names.
For example,
$ crono_trigger MailNotification
And other options is following.
$ crono_trigger --help
Usage: crono_trigger [options] MODEL [MODEL..]
-f, --config-file=CONFIG Config file (ex. ./crono_trigger.rb)
-e, --environment=ENV Set environment name (ex. development, production)
-p, --polling-thread=SIZE Polling thread size (Default: 1)
-i, --polling-interval=SECOND Polling interval seconds (Default: 5)
-c, --concurrency=SIZE Execute thread size (Default: 25)
-r, --fetch-records=SIZE Record count fetched by polling thread (Default: concurrency * 3)
-l, --log=LOGFILE Set log output destination (Default: STDOUT or ./crono_trigger.log if daemonize is true)
--log-level=LEVEL Set log level (Default: info)
-d, --daemonize Daemon mode
--pid=PIDFILE Set pid file
-h, --help Prints this help
Handle errors
This gem provides the following options to handle errors:
CronoTrigger.configure do |config|
# These handlers are called when the execute method fails even after retries.
config.error_handlers << proc do |exception, record|
ActiveRecord::Base.logger.error("Failed to process #{record.class}##{record.id}: #{exception} (#{exception.class})")
end
# These handlers are called when an exception occurs outside of the execute method.
config.global_error_handlers << proc do |exception|
ActiveRecord::Base.logger.error("#{exception} (#{exception.class})\n#{exception.backtrace.join("\n")}")
end
# db_error_retriable_options are passed to `Retriable.retriable` used in the idempotent code that accesses the database
# except for the code processing each record.
# Here is the default value.
config.db_error_retriable_options = {
on: {
ActiveRecord::ConnectionNotEstablished => nil,
},
}
end
For example, if you would like to reconnect to the database before retry for some reason, you can do so using on_retry
option as follows:
CronoTrigger.configure do |config|
config.db_error_retriable_options = {
on: {
ActiveRecord::ConnectionNotEstablished => nil,
Mysql2::Error::ConnectionError => nil,
ActiveRecord::StatementInvalid => /MySQL server is running with the --read-only option/,
},
on_retry: proc do |exception, try, elapsed_time, interval|
ActiveRecord::Base.logger.info("#{try}th: Retry on #{exception.class}: #{exception}")
next unless exception.is_a?(ActiveRecord::StatementInvalid)
ActiveRecord::Base.logger.info("#{try}th: Reconnect to MySQL on #{exception.class}: #{exception}")
# NOTE: The connection acquired here might be different from the one used in the code that raised the error,
# but we don't have a way to get the latter connection.
ActiveRecord::Base.connection_pool.with_connection(&:reconnect!)
end,
}
end
Specification
Columns
name | type | required | rename | description |
---|---|---|---|---|
cron | string | no | no | Recurring schedule formatted by cron style |
next_execute_at | datetime | yes | yes | Timestamp of next execution. Worker executes task if this column <= now |
last_executed_at | datetime | no | yes | Timestamp of last execution |
timezone | datetime | no | yes | Timezone name (Parsed by tzinfo) |
execute_lock | integer | yes | yes | Timestamp of fetching record in order to hide record from other transaction during execute lock timeout. when execution complete this column is reset to 0 |
started_at | datetime | no | yes | Timestamp of schedule activated |
finished_at | datetime | no | yes | Timestamp of schedule deactivated |
last_error_name | string | no | no | Class name of last error |
last_error_reason | string | no | no | Error message of last error |
last_error_time | datetime | no | no | Timestamp of last error occured |
retry_count | integer | no | no | Retry count. If execution succeed retry_count is reset to 0 |
current_cycle_id | string | no | yes | UUID that is updated when the schedule is resetted successfully |
You can rename some columns.
ex. crono_trigger_options[:next_execute_at_column_name] = "next_time"
Admin Web
Standalone mode
$ crono_trigger-web --rails
Mount as Rack app
# config/routes.rb
require "crono_trigger/web"
mount CronoTrigger::Web => '/crono_trigger'
Rollbar integration
This gem has rollbar plugin.
If crono_trigger/rollbar
is required, Add Rollbar logging process to CronoTrigger.config.error_handlers
Active Support Instrumentation Events
This gem provides the following events for Active Support Instrumentation.
monitor.crono_trigger
This event is triggered every 20 seconds by the first active worker in worker_id order, so note that other workers don't receive the event.
Key | Value |
---|---|
model_name | The model name |
executable_count | The number of executable records |
max_lock_duration_sec | The maximum amount of time since locked records started being processed |
max_latency_sec | The maximum amount of time since executable records got ready to be processed |
process_record.crono_trigger
This event is triggered every time a record finishes being processed.
Key | Value |
---|---|
record | The processed record |
Development
After checking out the repo, run bin/setup
to install dependencies. Then, run rake spec
to run the tests. You can also run bin/console
for an interactive prompt that will allow you to experiment.
To install this gem onto your local machine, run bundle exec rake install
. To release a new version, update the version number in version.rb
, and then run bundle exec rake release
, which will create a git tag for the version, push git commits and tags, and push the .gem
file to rubygems.org.
Contributing
Bug reports and pull requests are welcome on GitHub at https://github.com/joker1007/crono_trigger.
License
The gem is available as open source under the terms of the MIT License.