Buildkite Collectors for Ruby

DEPRECATION NOTICE Versions prior to 2.1.x are unsupported and will not work after mid-2023. Please upgrade to the latest version.

Official Buildkite Test Engine collectors for Ruby test frameworks ✨

⚒ Supported test frameworks: RSpec, Minitest, and more coming soon.

📦 Supported CI systems: Buildkite, GitHub Actions, CircleCI, Codeship, and others via the BUILDKITE_ANALYTICS_* environment variables.

Create a test suite, and copy the API token that it gives you.

Add the buildkite-test_collector gem:

gem install buildkite-test_collector

Or add this to your Gemfile’s test group:

group :test do
  gem 'buildkite-test_collector'
end

Add the following code to your RSpec setup file:

# spec/spec_helper.rb
require 'buildkite/test_collector'
Buildkite::TestCollector.configure(hook: :rspec)

Run your tests locally:

BUILDKITE_ANALYTICS_TOKEN=xyz rspec

Add the following code to your Minitest setup file:

# test/test_helper.rb
require 'buildkite/test_collector'
Buildkite::TestCollector.configure(hook: :minitest)

Run your tests locally:

BUILDKITE_ANALYTICS_TOKEN=xyz rake

Add the BUILDKITE_ANALYTICS_TOKEN secret to your CI, push your changes to a branch, and open a pull request 🎉

git checkout -b add-buildkite-test-engine
git commit -am "Add Buildkite Test Engine"
git push origin add-buildkite-test-engine

If your test suites use VCR to stub network requests, you'll need to modify the config to allow actual network requests to Test Engine.

VCR.configure do |c|
  c.ignore_hosts "analytics-api.buildkite.com"
end

This gem allows adding custom annotations to the span data sent to Buildkite using the .annotate method. For example:

Buildkite::TestCollector.annotate("User logged in successfully")

This is particularly useful for tests that generate a lot of span data such as system/feature tests.

For builds that execute the same test multiple times - such as when running the same test suite against multiple versions of ruby/rails - it's possible to tag each test execution with a prefix/suffix. This prefix/suffix is displayed for each execution on the test show page to differentiate the build environment. The prefix/suffix is specified using these environment variables:

BUILDKITE_ANALYTICS_EXECUTION_NAME_PREFIX
BUILDKITE_ANALYTICS_EXECUTION_NAME_SUFFIX

See the GitHub 'enhancement' issues for planned features. Pull requests are always welcome, and we’ll give you feedback and guidance if you choose to contribute 💚

After cloning the repository, install the dependencies:

bundle

And run the tests:

bundle exec rspec

Useful resources for developing collectors include the Buildkite Test Engine docs.

See DESIGN.md for an overview of the design of this gem.

Bug reports and pull requests are welcome on GitHub at https://github.com/buildkite/test-collector-ruby

1. Bump the version in version.rb and run bundle to update the Gemfile.lock.
2. Update the CHANGELOG.md with your new version and a description of your changes.
3. Git tag your changes and push

git tag v.x.x.x
git push --tags

Once your PR is merged to main:

1. Run rake release from main.
2. Create a new release in github.

The gem is available as open source under the terms of the MIT License.