knife-stencil is a knife plugin which enables you to consistently launch Chef-managed hosts across multiple clouds by capturing the options that you would typically pass to knife $CLOUD server create in JSON templates.

Brought to you by the folks at OpsUnit.

When provisioning infrastructures with Chef, traditionally one has invoked a long command containing multiple switches. In larger teams, or teams with tiers of engineers, this unfortunately leads to errors, as nodes are launched on the incorrect instance size or in incorrect security groups. It also means that there are some aspects of your infrastructure that you cannot recreate from source control.

As an example, if you wanted to launch a server in EC2, the knife invocation might look like this:

knife ec2 server create -r "role[base],role[webserver],role[search]" -I ami-b4c1a4d4 -G default,web,search -x root -N web24.prod.search.mycompany.com

By using knife-stencil to capture the above information in JSON templates, this could be simplified to:

knife stencil server create -N web24.prod.search.mycompany.com

knife-stencil expresses knife configuration options using a system of templates (stencils) that can inherit and override each other. Stencils are selected based on user-defined regexes that describe various potential node names, as befits your existing site naming convention.

The upshot of all of this is that the plugin enables:

• Launching hosts with the correct configuration options simply by following your local naming convention
• The capture in source control of node metadata such as availability zone, volume persistence and cloud provider (more on this below)
• Easy operational integration with your existing site, conventions and infrastructure

The plugin also works with most knife cloud plugins automatically. With the correct stencil configuration, it is simple to launch hosts in multiple clouds based on their name. Adding new clouds is simply a matter of configuring the appropriate credentials and installing the corresponding knife plugin gem.

Some examples of what can be achieved with knife-stencil are:

• Launch hosts in different clouds depending on their environment: launch development nodes on a local Openstack farm, DR servers in HP's cloud and production servers in Amazon EC2
• Ensure hosts land in the correct availability zone depending on some aspect of their role: ensure all slave database servers are in a different AZ to the masters
• Dynamically determine host instance type based on client: client bigcorp has a premium contract, launch their database servers as an m2.4xlarge, otherwise launch as m1.xlarge
• Split teams into "tooling" and "operational" streams: avoid resorting to runbooks to capture knowledge; reproduce it programatically and versioned over time
• Enforce legal jurisdiction restrictions: all hosts containing "backup" for client "bigcorp" should be in the EU region

Stencils take the general form:

{
  "matches": "^[00-99]+\.production\.bigclient\.mycompany\.com$",
  "inherits": [
    "environments/production.json",
    "clients/bigclient.json"
  ],
  "options": {
    ":availability_zone": "us-west-1a"
  }

Invoking the plugin to launch a new node as follows: knife stencil server create -N 1380211968.production.bigclient.mycompany.com will cause all stencil files under ~/.chef/stencils (configurable via :stencil_root in your knife.rb) to be evaluated.

Those that express a matches regular expression will be compared to the passed node name and ranked in order of strength of match. Once a "root" stencil has been selected, any stencils specified through inherits will also be evaluated. This continues sequentially and iteratively until a final configuration is arrived upon, at which point the correct cloud server creation class will be instantiated and run.

Where an inherited stencil expresses an option that already exists from a prior stencil, that value is overwritten. The inheritance tree of a stencil is evaluated in reverse order. That is to say, the further away the inherited stencil from the matching "root" stencil the more generic it is intended to be.

Not all stencils need express matches - nor do they need to inherit anything or express an option. You may choose to create stencils whose job it is to provide a convenient point to aggregate many other stencils, for example.

If all this sounds complicated you may find the examples directory helps to clear things up. You can also invoke knife stencil server explain live-pretendtown-app00 to have the plugin show you a non-destructive "query plan" of how it arrived at the configuration it would launch the node with. Try this with :stencil_root pointing at the 'deadfish' subdirectory of the examples directory.

You are free to organise your stencils, their names and the directory structure they are stored under as you wish. The examples directory provides just that.

If the final :options hash of a server that is being launched via knife-stencil contains the key :plugin, knife-stencil will attempt to load and instantiate classes from a similarly-named knife plugin. For example, a value of ec2 will have knife-stencil look for the knife-ec2 plugin and the class Chef::Knife::Ec2ServerCreate with which to launch the node. Likewise hp with knife-hp and Chef::Knife::HpServerCreate and so on for all plugins that follow this standard convention.

Given that the cloud plugin to use is expressed via the options hash, which is overridden as inherited stencils are parsed, it is nearly as trivial to provision in other clouds as it is to specify security groups, EBS retention policies or instance size.

Options are specified using the form they take in the cloud plugin's options hash. The easiest way to locate the option you are looking for is:

1. knife hp server create --help (note switch)
2. Execute gem environment to find the root of the GEM_PATH into which your gems are being installed
3. Locate the cloud gem and move into lib/chef/knife
4. Grep or otherwise locate the option stanza (there may be more than one) for the option you are concerned with
5. Take the symbol from the `option' and use that in the configuration:

For example, the following options stanza:

 option :rackspace_servicelevel_wait,
   :long => "--rackspace-servicelevel-wait",
   :description => "Wait until the Rackspace service level automation setup is complete before bootstrapping chef",
   :boolean => true,
   :default => false

Yields the following option:

"options": {
  "rackspace_servicelevel_wait": true,
}

1. Install the gem

   gem install knife-stencil

2. Configure knife.rb to point :stencil_root to your preferred location, or create ~/.chef/stencils

   mkdir -p ~/.chef/stencils

   OR

   add knife[:stencil_root] = "/alternative/location" to ~/.chef/knife.rb

3. Use the examples to start building your stencils and knife stencil server explain HOSTNAME to debug and explore. Try copying them into ~/.chef/stencils.

4. Manage ~/.chef/stencils using your source control workflow

At the time of writing only the knife-ec2 plugin from version 0.6.6 supports deletion via :chef_node_name. Most other plugins require an instance-id or similar parameter to be passed to delete a server. This is in counterpoint to how the stencil plugin would like to work. If the other plugins supported this kind of thing deletion would be a unified interface, like the creation of servers.

• The plugin does not currently work with chef/knife 11.8.0 or above, due to changes in the internal options parsing code at that point.

The gem and its dependencies are tested against the following ruby versions:

• 1.9.3
• 2.0.0
• 2.1.0