rspec-puppet-yaml
This gem enables Puppet module authors to write RSpec unit tests as YAML instead of Ruby (omitting the single, trivial line of code necessary to pass your YAML to RSpec). It also adds a new capability: a trivial means to create test variants (repeat 0-N tests while tweaking any set of inputs and expectations to detect unintended side-effects). If you're more comfortable with YAML than Ruby, or want to easily work with test variants, then you'll want this gem.
While this gem spares you from needing to learn Ruby just to test your Puppet module, you still need to do a little research into what RSpec tests are available to your YAML. The good news is that very knowledge is already documented for the rspec-puppet gem and you just need to know how to translate it into YAML. While not initially obvious, it really is very easy.
The source code examples in this document are expanded or direct translations of each of the matcher samples that are shown in the rspec-puppet README. Copyright for the original examples is owned by the maintainers of that gem and are duplicated here in good faith that these translations into YAML are not competitive and will benefit our common audience. This rpsec-puppet-yaml gem extends the reach of the rspec-puppet gem to include users who speak YAML better than Ruby; rpsec-puppet-yaml does not replace rspec-puppet but rather the written language that is necessary to interact with it (YAML rather than Ruby).
All following examples will be presented both in the original Ruby and in the
new YAML for comparison. In the most general terms, you can express an existing
RSpec entity in YAML simply by transcribing its name as a Hash key and its
attributes or contents as its children. Just know that I decided to use tests
instead of it
to identify the RSpec examples and I abstracted is_expected.to
and is_expected.not_to
rather than exposing them directly to YAML. This was
both an aesthetic decision and to reduce complexity.
Installation
Add this line to your application's Gemfile:
gem 'rspec-puppet-yaml'
And then execute:
$ bundle
Or install it yourself as:
$ gem install rspec-puppet-yaml
Usage
As per the existing documentation for rspec-puppet,
you will still create RSpec entry-point files at your_module/spec/**/*_spec.rb
(else rake
won't find your tests, YAML or otherwise). However, these files
now need only two lines of Ruby code (usually) and no RSpec code. You'll still
require your spec_helper
as always, and then just call the global-scope
entry-point function to parse your YAML into RSpec. The function shown here
expects to receive the fully-qualified path and name of your *_spec.rb
file,
not your YAML file. The function will search for your YAML files based on the
automatic value of the __FILE__
variable.
For example:
require 'spec_helper'
parse_yaml_from_spec(__FILE__)
Yes, that's it! You can copy-paste those two lines into every *_spec.rb
file
and then spend your time writing RSpec Puppet tests in YAML rather than Ruby.
To do so, create another file in the same directory with the same base name as
your *_spec.rb
file except change the extension to .yaml
or .yml
. In
fact, you don't even need the _spec
part of the name, so for an RSpec file
named my_module_spec.rb
, you can use any of these YAML file-names:
my_module_spec.yaml
my_module_spec.yml
my_module.yaml
my_module.yml
Fair warning: the parser will look for all of these file-names, so if you create more than one of them, they will all be processed, in turn.
Defining RSpec Puppet YAML Tests
For the most general-case example, this Ruby:
describe 'my_class', :type => 'class' {
let(:params) { my_attribute => 'value' }
it { is_expected.to some_matcher.with_attribute('value').and_attribute }
it { is_expected.not_to another_matcher}
}
becomes this YAML:
describe:
name: my_class
type: class
let:
params:
my_attribute: value
tests:
some_matcher:
with_attribute: value
and_attribute: nil # nil indicates a matcher method with no arguments
'!another_matcher': {} # ! negates the matcher and every matcher needs either a Hash or Array value
NOTE: The top-most entity of every rspec-puppet-yaml file must be a
describe
. Each describe
must have a name
attribute. The top-most
describe
must additionally have a type
attribute unless you arrange your
*_spec.rb
files according to the recommended rspec-puppet directory structure
(in which case the class can be automatically derived from its file-system
location).
Setting custom facts, parameters, titles, and any other let
setting
Ruby:
describe 'my_module' do
let(:title) { 'baz' }
let(:params) do
{ 'value' => 'foo',
'user' => :undef,
'require' => ref('Package', 'sudoku'),
'nodes' => {
ref('Node', 'dbnode') => ref('Myapp::Mycomponent', 'myapp')
}
}
end
let(:node) { 'testhost.example.com' }
let(:environment) { 'production' }
let(:facts) do
{ 'os' => {
'family' => 'RedHat',
'release' => {
'major' => '7',
'minor' => '1',
'full' => '7.1.1503'
}
}
}
end
let(:node_params) do
{ 'hostgroup' => 'webservers',
'rack' => 'KK04',
'status' => 'maintenance' }
end
let(:pre_condition) { 'include other_class' }
let(:post_condition) { 'include another_class' }
let(:module_path) { '/path/to/your/module/dir' }
let(:trusted_facts) do
{ 'pp_uuid' => 'ED803750-E3C7-44F5-BB08-41A04433FE2E',
'1.3.6.1.4.1.34380.1.2.1' => 'ssl-termination' }
end
end
YAML:
describe:
name: my_module
let:
title: baz
node: testhost.example.com
environment: production
params:
value: foo
user: !ruby/symbol undef # The symbol-form of `undef` must be used to specify an :undef value
require: '%{eval:ref("Package", "my-package")}': # %{eval:...} expands into a command and its arguments which is run via `eval`, capturing its return value. `'` or `"` demarcation of `%{}` is required only because YAML values are forbidden from starting with a `%`.
nodes:
'%{eval:ref("Node", "dbnode")}': '%{eval:ref("Myapp::Mycomponent", "myapp")}'
facts:
os:
family: RedHat
release:
major: 7
minor: 1
full: 7.1.1503
node_params:
hostgroup: webservers
rack: KK04
status: maintenance
pre_condition: include other_class
post_condition: include another_class
module_path: /path/to/your/module/dir
trusted_facts:
pp_uuid: ED803750-E3C7-44F5-BB08-41A04433FE2E
'1.3.6.1.4.1.34380.1.2.1': ssl-termination
The compile matcher
Ruby:
# Plain test to ensure the module compiles without error
describe "my_module" {
it { is_expected.to compile }
}
# Expanded test to ensure it compiles with all dependencies
describe "my_module" {
it { is_expected.to compile.with_all_deps }
}
# Ensure the module *fails* to compile, issuing an expected error message
describe "my_module" {
it { is_expected.to compile.and_raise_error(/error message match/) }
}
YAML:
# Plain test to ensure the module compiles without error
describe:
name: my_module
tests:
compile: {}
# Expanded test to ensure it compiles with all dependencies
# Multiple ways to achieve the same net effect!
describe:
name: my_module
tests:
compile: true # `with_all_deps` is the default test when the `compile` matcher is given any "truthy" value
describe:
name: my_module
tests:
compile: # Methods that don't accept arguments can be called from an Array
- with_all_deps
describe:
name: my_module
tests:
compile: # Methods that don't accept arguments can be called from a Hash with nil as their value
with_all_deps: nil
# Ensure the module *fails* to compile, issuing an expected error message.
# Method 1: Use a Regular Expression to match part of the error message. Note
# that in this case, you must identify your value as being a Ruby Regular
# Expression data type with the leading `!ruby/regexp` marker.
describe:
name: my_module
tests:
compile:
and_raise_error: !ruby/regexp /error message match/
# Method 2: Match the *entire* error message, not just part of it. This
# employs a normal String value that requires no additional marker.
describe:
name: my_module
tests:
compile:
and_raise_error: FULL text of the error message to match
The contain (or create) matcher
Ruby:
# Ensure a set of resources exist in the manifest, some with specific attributes
describe "my_module" {
it { is_expected.to contain_augeas('bleh') }
it { is_expected.to contain_class('foo') }
it { is_expected.to contain_foo__bar('baz') }
it { is_expected.to contain_package('mysql-server').with_ensure('present') }
it { is_expected.to contain_package('httpd').only_with_ensure('latest') }
it do
is_expected.to contain_service('keystone').with(
'ensure' => 'running',
'enable' => 'true',
'hasstatus' => 'true',
'hasrestart' => 'true'
)
end
it do
is_expected.to contain_user('luke').only_with(
'ensure' => 'present',
'uid' => '501'
)
end
it { is_expected.to contain_file('/foo/bar').without_mode }
it { is_expected.to contain_service('keystone_2').without(
['restart', 'status']
)}
it { is_expected.to contain_file('/path/file').with_content(/foobar/) }
it do
is_expected.to contain_file('/other/path/file')
.with_content(/foobar/)
.with_content(/bazbar/)
end
}
YAML:
# Ensure a set of resources exist in the manifest, some with specific attributes
describe:
name: my_module
tests:
contain_augeas:
bleh: {}
contain_class:
foo: {}
contain_foo__bar:
baz: {}
contain_package:
mysql-server: present # `with_ensure` is the default test when packages are given any scalar value
httpd:
only_with_ensure: latest
contain_service:
keystone:
with:
ensure: running
enable: true
hasstatus: true
hasrestart: true
keystone_2:
without:
- restart
- status
contain_user:
luke:
only_with:
ensure: present
uid: 501
contain_file:
/foo/bar:
- without_mode
/path/file:
with_content: !ruby/regexp /foobar/
/other/path/file:
with_content:
- !ruby/regexp /foobar/
- !ruby/regexp /bazbar/
With stipulated resource relationships
Ruby:
describe "my_module" {
# Ensure the file, foo, has specific relationships with other resources
it { is_expected.to contain_file('foo').that_requires('File[bar]') }
it { is_expected.to contain_file('foo').that_comes_before('File[baz]') }
it { is_expected.to contain_file('foo').that_notifies('File[bim]') }
it { is_expected.to contain_file('foo').that_subscribes_to('File[bom]') }
# Ensure the file, bar, has specific relationships with many other resources
it { is_expected.to contain_file('bar').that_requires(['File[fim]', 'File[fu]']) }
it { is_expected.to contain_file('bar').that_comes_before(['File[fam]','File[far]']) }
it { is_expected.to contain_file('bar').that_notifies(['File[fiz]', 'File[faz]']) }
it { is_expected.to contain_file('bar').that_subscribes_to(['File[fuz]', 'File[fez]']) }
# Other relationship example
it { is_expected.to contain_notify('bar').that_comes_before('Notify[foo]') }
it { is_expected.to contain_notify('foo').that_requires('Notify[bar]') }
}
YAML:
describe:
name: my_module
tests:
contain_file:
# Ensure the file, foo, has specific relationships with other resources
foo:
that_requires: File[bar]
that_comes_before: File[baz]
that_notifies: File[bim]
that_subscribes_to: File[bom]
# Ensure the file, bar, has specific relationships with many other resources
bar:
that_requires:
- File[fim]
- File[fu]
that_comes_before:
- File[fam]
- File[far]
that_notifies:
- File[fiz]
- File[faz]
that_subscribes_to:
- File[fuz]
- File[fez]
contain_notify:
bar:
that_comes_before: Notify[foo]
foo:
that_requires: Notify[bar]
The count matcher
Ruby:
# Ensure certain resource counts are true
describe "my_module" {
it { is_expected.to have_resource_count(2) }
it { is_expected.to have_class_count(2) }
it { is_expected.to have_exec_resource_count(1) }
it { is_expected.to have_logrotate__rule_resource_count(3) }
}
YAML:
# Ensure certain resource counts are true
describe:
name: my_module
tests:
have_resource_count: 2
have_class_count: 2
have_exec_resource_count: 1
have_logrotate__rule_resource_count: 3
Type alias matchers
Ruby:
describe 'MyModule::Shape' do
it { is_expected.to allow_value('square') }
it { is_expected.to allow_values('circle', 'triangle') }
it { is_expected.not_to allow_value('blue') }
end
YAML:
describe:
name: MyModule::Shape
tests:
be_valid_type:
allow_value: square
allow_values:
- circle
- triangle
'!be_valid_type': # The leading ! switches is_expected.to to is_expected.not_to
allow_value: blue
Function matchers
Ruby:
describe 'my_function' {
it { is_expected.to run.with_params('foo').and_return('bar') }
}
describe 'my_other_function' {
it { is_expected.to run.with_params('foo', 'bar', ['baz']) }
}
YAML:
describe:
'my_function':
tests:
run:
with_params: foo
and_return: bar
run:
'my_other_function':
tests:
run:
with_params:
- foo
- bar
- ['baz']
Negative tests and error matching
Ruby:
describe 'my_function' {
it { is_expected.not_to run.with_params('a').and_raise_error(Puppet::ParseError) }
}
YAML:
describe:
name: my_function
tests:
'!run': # Negate a test with a ! prefix, but `'` or `"` demarcate the matcher name becaus a leading ! in YAML denotes a data-type specification.
with_params: a
and_raise_error: !ruby/exception Puppet::ParseError
run:
Using before
and after
Ruby:
describe 'my_function' {
before(:each) { scope.expects(:lookupvar).with('some_variable').returns('some_value') }
it { is_expected.to run.with_params('...').and_return('...') }
}
YAML:
YAML indirectly supports executable code in RSpec's before
and after
blocks.
It is emulated by first writing a global scope function in your *_spec.rb
file and then calling that function from the YAML file, as shown in these two
snippets:
spec/function/my_function_spec.rb
require 'spec_helper'
def my_before
scope.expects(:lookupvar).with('some_variable').returns('some_value')
end
parse_yaml_from_spec(__FILE__)
spec/function/my_function_spec.yaml
describe:
name: my_function
before: my_before
tests:
run:
with_params: ...
and_return: ...
Note that before:
can specify an Array of global-scope functions to call,
though it may be more intuitive to just define a global-scope function which
calls all of the other functions you need to chain together. Or not. It
depends on your logic.
This technique also helps define custom functions for your tests.
Hiera integration
Apart from ensuring that your metadata.json
file is valid, you just don't need
to do anything at all to enable module-level Hiera data integration; it's
already built into rspec-puppet and it runs quite well without any additional
configuration. If however, you have a valid use-case for customizing Hiera in
order to test out-of-module data -- which should be a really hard sell since you
should be testing your Module's code, not any out-of-module Hiera data -- the
above documentation should be adequate to express such custom Hiera
configuration, whether via let
settings or custom functions run during
begin
. Should you find the existing support to be inadequate, feel free to
open a qualifying Pull Request that adds whatever additional support you need.
Unsupported RSpec Puppet Features
This extension does not support the following features found in rspec-puppet:
- There is no way to create an example
it
that uses theexpect()
function instead ofis_expected
. - With the highest available version of rspec-puppet at the time of this
writing, there doesn't seem to be any way to use
subject { exported_resources }
because rspec-puppet throws an error message when you try, even when you follow its advice as to where to place thesubject
. You can still addsubject
to your YAML; it's just that rspec-puppet'sexported_resources
doesn't seem to be accessible from anysubject
today. - In the Function matcher, lambdas are not known to be supported via YAML. So,
the rspec-puppet example showing
run.with_lambda
has no obvious equivalent in rspec-puppet-yaml. A clever application of the%{eval:...}
expander might help in some cases, but feel free to experiment and share back if you find a way to make this work.
Variants
This gem adds a new feature that isn't present in the gem it extends:
variants
. A variant in unit testing is a named repeat of its parent container
with certain inputs and expectations tweaked. For example, imagine you have 10
base test examples and you want to test the limits of one input while
simultaneously ensuring there are no unintended side-effects (so, you want to
re-run the other 9 tests for each iteration of changes to the input-under-test).
Without variants, you'd have to duplicate those other 9 test examples over and
over. Variants eliminate all that dupliation in your test definitions, handling
the repeating configuration for you.
Here's an example of a classic package.pp
that enables customization of the
single package's ensure
attribute. The parent context defines two matcher
tests, have_package_resource_count
and contain_package
. Each variant
inherits both, then tweaks one aspect or another of the parent examples.
Further, a separate context, package.pp negative tests
does not inherit any
tests from the package.pp
context; they are peers rather than dependents.
describe:
name: my-package
context:
'package.pp':
tests:
have_package_resource_count: 1
contain_package:
my-package: present
variants: # These will all inherit the parent's tests, tweaking as needed
'uninstall':
let:
params:
package_ensure: absent
tests:
contain_package:
my-package: absent
'pinned package version':
let:
params:
package_ensure: '1.0.0.el7'
tests:
contain_package:
my-package: '1.0.0.el7'
'package.pp negative tests':
variants:
'bad package_ensure':
let:
params:
package_ensure: 2.10
tests:
compile:
and_raise_error: !ruby/regexp /parameter 'package_ensure' expects a String value, got Float/
Note that variants
inherit everything from their parent describe
or
context
except the following attributes:
- variants
- before
- after
- subject
Style Choices
Many of the elements can be expressed in multiple ways to achieve the same
effect. For example, containers like describe
, context
, and variants
can
be listed as more-than-one using either of these forms:
# As implicity named Hash entities
describe:
name: my_entity
context:
'context 1 name':
attributes...
'context 2 name':
attributes...
# As explicitly named Hash entities
describe:
name: my_entity
context:
- name: context 1 name
attributes...
- name: context 2 name
attributes...
Keys can also be expressed as symbols, strings, or any combination of them:
# Keys as strings
describe:
name: my_entity
context:
- name: context name
attribute: value
# Keys as symbols
:describe:
:name: my_entity
:context:
- :name: context name
:attribute: value
Development
After checking out the repo, run bin/setup
to install dependencies. Then, run
bundle install && bundle exec rake spec
to run the tests. Run yard
to
generate HTML documentation files (these generated files are ignored by git, so
this is useful for local preview). You can also run bin/console
for an
interactive prompt that will allow you to experiment.
To install this gem onto your local machine, run bundle exec rake install
. To
release a new version, update the version number in version.rb
, and then run
bundle exec rake release
, which will create a git tag for the version, push
git commits and tags, and push the .gem
file to
rubygems.org.
API documentation is available at github.io.
Contributing
Bug reports and pull requests are welcome on GitHub at https://github.com/wwkimball/rspec-puppet-yaml. Contributors are expected to adhere to the Contributor Covenant.
License
The gem is available as open source under the terms of the MIT License.
Code of Conduct
Everyone interacting in the rspec-puppet-yaml project’s codebases, issue trackers, chat rooms and mailing lists is expected to follow the code of conduct.