Compatriot

Compatibility + riot! It's the friend that helps with browser compatibility! This Ruby gem makes cross-browser testing less painful. Its goal is to help identify pages that appear to have significant variations when rendered in different browsers. You can spend your time fixing the cross-browser problems rather than looking for them. Don't let your users find the inconsistencies and get to them first.

• Adds a assert_no_ui_changes method to your test assertions
• When called from your tests, it takes a screenshot and compares it to a control image to get percentage difference
• If the difference is greater then the threshold, it fails the tests
• Pairs well with BrowserStacks automation feature to run across many environments

• Have a screenshot of sample results in the README
• Have more and better tests
• Perform better on the image processing (by sampling/resizing, using oily_png, etc)
• Show a better image difference

Requirements

• Ruby v2.1.7
• Capybara

1. Require compatriot in your test helper,

2. and include Compatriot::Assertions in your test class

3. and configure Compatriot

   require 'compatriot'
   
   class IntegrationTest
   include Compatriot::Assertions
   
   Minitest::Reporters.use! Compatriot::MinitestReportDriver.new # Optional: if you want html output. See MinitestReporters for more info
   
     Compatriot.configure do |config|
       screenshot_directory = 
         './test/screenshots/' +
         caps['os'] + '_' +       #caps(Capabilities) is how we tell browserstack what environment to use
         caps['os_version'] + '_' +
         caps['browser'] + '_' +
         caps['browser_version']
       config.screenshot_directory = screenshot_directory.downcase
       config.ui_difference_threshold = 0.05
     end
     ...

4. Call assert_no_ui_changes from your tests

   it 'does not break the ui' do
     visit some_page_path
     assert_no_ui_changes
   end

The first time through it will create the control image in #{screenshot_directory}/control. You should then review the screenshots and check them into source control as your baseline. Every run after that, it will take a variable image screenshot in #{screenshot_directory}/variable and generate a difference image in #{screenshot_directory}/diffs.

assert_no_ui_changes takes an optional string parameter to allow you to make multiple assertions in a test and not run into name conflicts.

example:

it 'has multiple ui change assertions' do
  visit page_1
  assert_no_ui_changes 'page 1'
  visit page_2
  assert_no_ui_changes 'page 2'
end

To see the % difference for every assertion, and get the path the the difference, set config.show_diffs = true when you configure Compatriot.

The intentions for running this against multiple environments is that you would run your test suite once for every environment you wish to run it against. To accomplish this, you should set your screenshot_directory to include the current environment name, this way you will store your control images in separate directories.

Note: The configuration example above shows you how to set the directory using the specified capabilities.

Using at least ruby 2.1.7:

bundle install
bundle exec rake test

This is using minispec for testing.

• Issues

We'd really like to know if something is wrong, so please file an issue on the Issue List if you have a problem, suggestion, unsupported use case, etc.

There are many opportunities for improvement. Feel free to:

• Fork the repository
• Clone the repository locally, or edit via Github
• Create a new branch with a meaningful name of the issue or feature you're working on
• Commit often and when important
• Write tests specifically for the changes you've made (unless you're fixing a failing test. Also: just submitting new tests for untested code is a big help too!)
• Push your feature or bug fix branch to your Github fork
• Make a Pull Request from your fork to the main repo

Ryan Bates did an awesome Railscast on contributing to open source that walks through this process, but please let us know if you have any questions or problems.

Many thanks to the wonderful libraries that make this gem possible:

• capybara
• selenium-webdriver
• chunky_png (and especially this blog post about using chunky_png to create image diffs by Jeff Kreeftmeijer)
• travis for CI

• Carol Nichols (twitter, website)
• Andrew Cox (twitter, website)
• Kurtis Rainbolt-Greene (twitter, website)
• Steve Klabnik (twitter, website)
• Jeff Koenig (twitter
• You???

(The MIT License)

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the 'Software'), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.