What to Test

Stop running tests which are clearly not affected by the change you introduced in your commit!

This project started as a fork of Test This, Not That! A Google Summer of Code 2015 project under Ruby on Rails. Where TTNT is specific to the Minitest framework, WTT is easily extensible to support other frameworks.

Add the gems to your gemfile.

gem 'wtt-core'
gem 'wtt-rspec'

Add the tasks to your Rakefile, they accept the same arguements as the normal RSpec tasks.

require 'wtt/rspec'

WTT::Core.create_core_tasks  # Define the anchor_raise/drop tasks
namespace :wtt do
	desc 'Produce a map by running all RSpec tests for the gem'
	WTT::RSpec::MapTask.new(:map_specs, :command_options) do |t, args|
	  t.rspec_opts = '-t ~wip -t ~off --color '
	  t.rspec_opts << args[:command_options] if args[:command_options]
	end
	
	desc 'Run only specs likely to break.'
	WTT::RSpec::SpecTask.new(:specs, :command_options) do |t, args|
	  t.rspec_opts = '-t ~wip -t ~off --color '
	  t.rspec_opts << args[:command_options] if args[:command_options]
	end
end

Map your code with:

bundle exec rake wtt:map_specs

Run tests:

bundle exec rake wtt:specs

You can call WTT.active? or check if ENV['WTT_ACTIVE'] is non-null.

WTT uses git to determine which files you've changed so that it knows which tests to run. Without an anchor, it will use HEAD. When an anchor is dropped WTT will consider each since the anchor revsion.

Dropping an anchor at the start of feature work allows you an extra measure of confidence that you're running the correct tests.

Drop an anchor with:

bundle exec rake wtt:anchor_drop

And raise it with:

bundle exec rake wtt:anchor_raise

WTT exposes a configuration property that can be accessed like so:

require 'wtt'

WTT.configure do |config|
  # Do stuff with config
end

The configuration object exposes the following methods:

There are currently three matchers that can be used to select your tests. It's not currently known which is the best to use so the most inclusive (Touch) is used as the default.

Touch: Will match a test if the line that was changed lies between the first and last line the test hit in the file.

WTT.configure do |config|
  config.matcher = WTT.touch_matcher
end

Exact: Will match a test if the exact line that was changed was touched by the test.

WTT.configure do |config|
  config.matcher = WTT.exact_matcher
end

Fuzzy: Will match a test if the line that was changed is within a specified number of lines (Default 11).

WTT.configure do |config|
  num_lines = 5	
  config.matcher = WTT.fuzzy_matcher(num_lines)
end

WTT includes support for capturing coverage from external processes and mapping it to tests.

Configure WTT on the server side (in boot.rb for example):

require 'wtt'

WTT.configure do |config|
  config.traced_port = 4242 # Any unique port will do
end

WTT.start_service unless Rails.env.production?

Configure WTT on the client side with the address/port of your remote (in spec_helper.rb for example)

require 'wtt'

WTT.configure do |config|
  config.add_remote 'localhost:4242'
end

Multiple remotes can be added by calling add_remote again and passing a different host/port.

First the WTT "Map Task" runs each test and gathers code coverage for individual test scenarios using the Tracepoint API in Ruby 2.x. Data from any remotes is merged with the local coverage data to produce a map.

After the map has been generated the WTT can use it to determine which tests to run.

The WTT spec task will interrogate your repo to determine which source files have changed. Using the patchset, it will select tests which cover the files (and lines within the files) that have changed. If a test has been modified it will likewise be selected to be run.