Scripted

Scripted is a framework for organizing scripts.

Among its features are:

• A convenient DSL to determine how and when to run scripts
• Determine which scripts in parallel with each other
• Manage the exit status of your scripts
• Conditionally run certain commands
• A variaty of output formatters, including one that exports the output of the scripts via websockets!
• Specify groups of tasks
• Integration with Rake

See a video of scripted running with websockets.

It is considered good practice to bundle all the tasks you need to do in one script. This can be a setup script that installs your application, or a all test scripts combined for your CI to use.

While it is very easy to make this with plain Bash scripts, I found myself writing a lot of boiler code over and over again. I wanted to keep track of the runtimes of each commands. Or I wanted to run certain scripts in parallel, but still wait for them to finish.

This gem exists because I wanted to simply define which commands to run, and not deal with all the boilerplate code every time.

There are a number of examples included in the project. You can find them in the examples directory.

• Clone the project
• Install via ./install
• See which examples are avaibale: rake -T examples
• Run an example: rake examples:websockets

You'll need to create a configuration file for scripted to run. By default this file is called scripted.rb, but you can name it whatever you like.

After making the configuration, you can run it with the scripted executable.

Run scripted --help to get an overview of all the options.

Also, have a look at the Cucumber features for more information.

You can define "commands" via the run-method. For instance:

run "rspec"
run "cucumber"

The first argument to the run-method is the name of the command. If you don't specify anything else, this will be the shell command run. You can change the command further by supplying a block.

run "fast unit specs" do
  `rspec spec/unit`
end

run "slow integration specs" do
  `rspec spec/integration`
end

You can also specify Rake tasks and Ruby commands to run:

run "migrate the database" do
  rake "db:migrate"
end

run "some ruby code" do
  ruby { 1 + 1 }
end

Keep in mind that MRI has trouble running ruby and rake tasks in parallel due to the GIL.

You can really win some time by running certain commands in parallel. Doing that is easy, just put them in a parallel-block:

run "bundle install"

parallel do
  run "rspec"
  run "cucumber"
end

run "something else"

Commands that come after the parallel block, will wait until all the commands that run in parallel have finished.

There are only a few caveats to this. The scripts must be able to run simultaniously. If they both access the same global data, like a database or files on your hard disk, they will probably fail. Any output they produce will appear at the same time, possibly making it unreadable.

You can specify multiple parallel blocks.

By default, all commands will run, even if one failed. The exit status of the entire scripted run will hover reflect that one script has failed.

If one of your commands is so important that other commands cannot possibly succeed afterwards, mark it with important!:

run "bundle install" do
  important!
end

run "rspec"

If a command might fail, but you don't want the global exit status to change if it happens, mark the command with unimportant!

run "flickering tests" do
  unimportant!
end

If you have some clean up to do, that always must run, even if an important command failed, mark it with forced!:

run "start xvfb" do
  `/etc/init.d/xvfb start`
  unimportant! # it might be on already
end

run "bundle install" do
  important!
end

run "rspec"

run "stop xvfb" do
  `/etc/init.d/xvfb stop`
  forced!
end

There are two types of commands that can be run conditionally.

To run a command only when all previous commands were successful, mark it with only_when_success!:

run "mail me if everything went ok" do
  only_when_success!
end

To have a command run only if other commands have failed, mark it with only_when_failed!:

run "mail me if build failed" do
  only_when_failed!
end

Formatters determine what gets outputted. This can be to your screen, a file, or a websocket. You can specify the formatters via the command line, or via the configuration file.

Via the command line:

$ scripted --format my_formatter --out some_file.txt

Via the configuration file:

formatter :my_formatter, :out => "some_file.txt"

You can have multiple formatters. If you don't specify the out option, it will send the output to STDOUT.

The formatter that is used if you don't specify anything is default. This formatter will output the output of your scripts and display stacktraces. If you specify different formatters, the default formatter will not be used. So if you still want output to the terminal, you need to add this formatter.

$ scripted -f default -f some_other_formatter

The table formatter will display an ASCII table when it's done, giving an overview of all commands.

It looks something like this:

┌─────────────────┬─────────┬─────────┐
│ Command         │ Runtime │ Status  │
├─────────────────┼─────────┼─────────┤
│ rspec           │  0.661s │ success │
│ cucumber        │ 18.856s │ success │
│ cucumber -p wip │  0.558s │ success │
└─────────────────┴─────────┴─────────┘
  Total runtime: 19.527s

To use it:

$ scripted --format table

This will print a banner before each command, so you can easily see when a command is executed.

It looks something like this:

┌────────────────────────────────────────────────┐
│                 bundle update                  │
└────────────────────────────────────────────────┘

To use it:

$ scripted --format announcer

The stats formatter will print a csv file with the same contents as the table-formatter. This is handy if you want to keep track of how long your test suite takes over time, for example.

Example:

name,runtime,status
bundle update,5.583716,success
rspec,4.319095,success
cucumber,22.292316,failed
cucumber -p wip,0.649777,success

To use it:

$ scripted --format stats --out runtime.csv

Note: make sure you backup the file afterwars, because each time it runs, it will override the file. Also, if you're running on Ruby 1.8, you'll have to install FasterCSV.

And last, but not least, the websocket formatter. This awesome formatter will publish the output of your commands directly to a websocket.

This is done via Faye, a simple pub/sub messaging system. It is tricky to implement this, so be sure to check out the example code, which includes a fully functioning Ember.js application.

$ scripted -f websocket -o http://localhost:9292/faye

Make sure you have Faye running. The example does this for you.

You can also make your own formatter. As the name of the formatter, just specify the class name:

$ scripted -f MyAwesome::Formatter

Have a look at the existing formatters in lib/scripted/formatters to see how to make one.

You can specify different groups of commands by putting commands in a group block:

group :test do
  run "rspec"
  run "cucumber"
end

group :install do
  run "bundle install"
  rake "db:setup"
end

Then you can specify one or many groups to run on the command line:

$ scripted --group install --group test

Commands that are not defined in any group are put in the default group.

Besides calling Rake tasks from Scripted, you can also launch scripted via Rake.

The simplest example is:

require 'scripted/rake_task'
Scripted::RakeTask.new(:scripted)

Then you can run rake scripted

You can pass a block to specify your commands in-line if you like:

require 'scripted/rake_task'
Scripted::RakeTask.new(:install) do
  run "foo"
  run "bar"
end

You can also supply different groups to run:

require 'scripted/rake_task'
Scripted::RakeTask.new(:ci, :install, :test)

Running rake ci will run both the install and test group.

Calling scripted from within another Ruby process is easy:

require 'scripted'
Scripted.run do
  run "something"
end

I first named this library "test_suite", and most examples show running test suites. But Scripted isn't only for running tests. Here are some ideas:

• Installing stuff, like installing stuff you want
• Running a command perminantly and seeing the output via websockets. Like ping, your server, or a tool that monitors your worker queues.

The beauty if plain bash scripts is that they can be run without having anything installed. The problem with Scripted is that it is a gem and you might need to gem install scripted or bundle install before it will work.

I prefer to have the README of my projects say, something along the lines of:

## How To

* Install: `script/install`
* Upgrade: `script/upgrade`
* Deploy: `script/deploy`

Nothing more. No complicated 10 step plan, just type one command and you're good to go. You need a bash script for that.

So here is an example of how such a bash script might look like:

#!/usr/bin/env bash
set -e
gem which scripted >/dev/null 2>&1 || gem install scripted
scripted --group install

This gem is in alpha state. YMMV. I believe I got the basic functionality, but not everything is as cleanly implemented as it could be. For instance, there are undoubtedly edge cases I didn't think of and error handling can probably be more user friendly.

I'm putting this out there to get some feedback. Please don't hesitate to contact me if you have any questions or ideas for improvements. Mention me on Twitter, or open an issue on Github.

Some possible future features:

• Include groups from within other groups.
• Specify a working directory.
• Define exit status, like important! for multiple commands at once.

• Works on MRI and Rubinius.
• JRuby might have problems running shell commands.
• JRuby doesn't always allow you to compile C extensions, so you cannot install Faye. Use a different Ruby implementation or use the Node.js version.
• To get color in RSpec, use the --tty switch, or RSpec will not believe the shell supports color.
• Use the --color switch for Cucumber.
• Groups don't run in the specified order yet.

To set it up, just run ./install.

1. Fork it
2. Create your feature branch (git checkout -b my-new-feature)
3. Commit your changes (git commit -am 'Add some feature')
4. Push to the branch (git push origin my-new-feature)
5. Create new Pull Request