regressiontest

regressiontest, at this point, is a simple module for calling programs on the command line, capturing output and comparing output against an existing reference file. The idea is to capture changes in output at a very global level, perhaps with different input files and command line options.

Note that JRuby on Travis does not allow invoking the command line, but MRI and Rubinius work fine.

Note: this software is under active development, your mileage may vary!

    gem install regressiontest

    require 'regressiontest'

Simple usage

  RegressionTest::CliExec::exec("ls","-l").should be_true

by default a .ref and a .new file are created in the ./test/data/regression directory. A filter can be added to ignore lines of output (as a regex), e.g.

  RegressionTest::CliExec::exec("ls","-l",ignore: 'INFO bio-gff3: Memory used')

Other options are :timeout and :should_fail.

The API doc is online. For more code examples see the test files in the source tree. A good example can be found in the bio-table project which uses cucumber features combined with the regressiontest gem. The features look like

Scenario: Test the numerical filter by indexed column values
  Given I have input file(s) named "test/data/input/table1.csv"
  When I execute "./bin/bio-table --num-filter 'values[3] > 0.05'"
  Then I expect the named output to match "table1-0_05"

and are listed in cli.feature and the matching steps are simply

Given /^I have input file\(s\) named "(.*?)"$/ do |arg1|
  @filenames = arg1.split(/,/)
end

When /^I execute "(.*?)"$/ do |arg1|
  @cmd = arg1 + ' ' + @filenames.join(' ')
end

Then /^I expect the named output to match "(.*?)"$/ do |arg1|
  RegressionTest::CliExec::exec(@cmd,arg1).should be_true
end

and listed in cli.rb. The automatically generated regression output files are checked into git in the test/data/regression directory and checked with 'bundle exec rake'.

Information on the source tree, documentation, examples, issues and how to contribute, see

http://github.com/pjotrp/regressiontest

The BioRuby community is on IRC server: irc.freenode.org, channel: #bioruby.

If you use this software, please cite one of

• BioRuby: bioinformatics software for the Ruby programming language
• Biogem: an effective tool-based approach for scaling up open source software development in bioinformatics

This Biogem is published at #regressiontest

Copyright (c) 2012-2014 Pjotr Prins. See LICENSE.txt for further details.