chef-provisioning-fog

This is the Fog driver for Chef Provisioning. It provides DigitalOcean, IBM Softlayer, Joyent, OpenStack, Rackspace, vCloud Air and XenServer functionality.

These are the primary documents to help learn about using Provisioning and creating Provisioning drivers:

• Chef Docs
• Frequently Asked Questions
• Configuration
• Writing Drivers
• Embedding
• Providers

A note about key pairs - The key name used in fog_key_pair must be the same as the filename of the local key to be used. If the key does not exist in one of private_key_paths (which you can set in knife.rb or in a client.rb) it will be created.

NOTE: Using chef-provisioning-fog for AWS is discouraged and new users should use chef-provisioning-aws. To use chef-provisioning-fog the fog-aws gem must be installed by hand (it is not list a dependency of this gem any more). ChefDK users should chef gem install fog-aws.

You'll need to update your knife.rb to work:

You need one of the following for the driver:

fog:AWS:<account_id>:<region>
fog:AWS:<profile_name>
fog:AWS:<profile_name>:<region>

For example:

driver 'fog:AWS:<account_id>:<region>'
driver_options :compute_options => {
                                     :aws_access_key_id => 'YOUR-ACCESS-KEY-ID',
                                     :aws_secret_access_key => 'YOUR-SECRET-ACCESS-KEY',
                                     :region => 'THE-REGION-YOU-WANT-TO-PROVISION-IN'
                                    }

For a full example see <examples/aws/simple.rb>.

Update your knife.rb to contain your DigitalOcean API token and the driver:

driver 'fog:DigitalOcean'
driver_options :compute_options => {
                                    :digitalocean_token => 'token'
                                   }

fog 1.38.0 (Newest release) has an issue with DigitalOcean. If you want to use DigitalOcean and chef-provisioning-fog you need to down grade your fog gem.

$ gem install fog --version 1.37.0
Successfully installed fog-1.37.0
Parsing documentation for fog-1.37.0
$ gem uninstall fog

Select gem to uninstall:
 1\. fog-1.37.0
 2\. fog-1.38.0
 3\. All versions
> 2
Successfully uninstalled fog-1.38.0
$

For a full example see <examples/digitalocean/simple.rb>.

You'll need to update your knife.rb to work with this also:

driver 'fog:SoftLayer'
driver_options :compute_options => {
                                     :provider => 'softlayer',
                                     :softlayer_username => 'username',
                                     :softlayer_api_key => 'api_key',
                                     :softlayer_default_domain => 'example.com',
                                   }

Once you or your administrator has created a SoftLayer account you can manage your API key at https://control.softlayer.com/account/users

:bootstrap_options => {:key_name => 'label'} is looked up by_label; make sure you have a public key created on control portal at https://control.softlayer.com/devices/sshkeys with a matching label.

NOTE: the SoftLayer driver injects a custom post provisioning script that ensures some packages needed by chef-provisioning-fog to install chef are present (e.g. sudo). The injected script will call your :postInstallScriptUri if you define one. The driver will wait until the injected script is done. The driver and script communicate using userMetadata so you cannot use metadata.

For a full example see <examples/softlayer/simple.rb>.

You'll need to update your knife.rb to work:

driver 'fog:Joyent'
driver_options :compute_options => {
                                     :joyent_username => 'YOUR-JOYENT-LOGIN',
                                     :joyent_password => 'YOUR-JOYENT-PASSWORD',
                                     :joyent_keyname => 'THE-SSH-KEY-YOUVE-UPLOADED',
                                     :joyent_version => '7.3.0', # if you are using the joyent public cloud
                                     :joyent_keyfile => 'YOUR-PRIVATE-SSH-KEY-LOCATION' # Such as '/Users/jasghar/.ssh/id_rsa'
                                    }

Tested this with the Joyent Public Cloud. For the package names, use the GUI to find the name(s) that you want to use. This is also required to figure out the Image UUID, there doesn't seem to be an effective way of doing this without the GUI.

For a more in-depth usage of this driver to use with either Private or Public Joyent cloud, checkout this blog post by mhicks from #smartos on freenode.

For a infrastructure container example see <examples/joyent/infrastructure.rb>.

You'll need to update your knife.rb to work:

driver 'fog:OpenStack'
driver_options :compute_options => {
                                     :openstack_auth_url => 'http://YOUROPENSTACK-CLOUD:5000/v2.0/tokens',
                                     :openstack_username => 'YOUR-USERNAME',
                                     :openstack_api_key  => 'YOUR-PASSWORD',
                                     :openstack_tenant   => 'YOUR-TENANT-ID-NAME'
                                    }

For a full example see <examples/openstack/simple.rb>.

For this example, you must configure knife.rb with your credentials and a region to operate on. This example is also available as a Github repo.

You must configure some credentials and region in a knife.rb file like so:

driver 'fog:Rackspace'
driver_options :compute_options => {
                                     :rackspace_username => 'MY-RACKSPACE-USERr',
                                     :rackspace_api_key  => 'API-KEY-FOR-USER',
                                     :rackspace_region => 'dfw' # could be 'org', 'iad', 'hkg', etc
                                    }

For a full example see <examples/rackspace/simple.rb>.

NOTE: The fog mega-gem has been removed as a direct dependency of chef-provisioning-fog. Since support for vcair is only in the fog gem and fog does not supply any "meta-gem" for vcair specifically, that means that the fog gem must be manually installed. ChefDK users should manually chef gem install fog.

Docs TODO.

You should configure XenServer driver with your credentials:

with_driver "fog:XenServer:<XEN-SERVER-IP/NAME>", compute_options: {
  xenserver_username: 'MY-XEN-USER',
  xenserver_password: 'MY-XEN-PASSWORD',
  xenserver_defaults: {
    'template' => 'ubuntu-14.04'
  }
}

For a full example see <examples/xenserver/simple.rb>.

You should configure the driver with your credentials or :

with_driver "fog:Scaleway:Your-Organisation-UUID:region", compute_options: {
  scalewat_token: 'your-api-token',
}

or just use the fog configuration (~/.fog):

with_driver 'fog:Scaleway'

For full examples, see <examples/scaleway>.

require 'chef/provisioning'

machine_batch do
  machines search(:node, '*:*').map { |n| n.name }
  action :destroy
end

When you are done with the examples, run this to clean up:

$ chef-client -z destroy_all.rb

Chef Provisioning has two major abstractions: the machine resource, and drivers.

You declare what your machines do (recipes, tags, etc.) with the machine resource, the fundamental unit of Chef Provisioning. You will typically declare machine resources in a separate, OS/provisioning-independent file that declares the topology of your app--your machines and the recipes that will run on them.

The machine resources from the cluster.rb example are pretty straightforward. Here's a copy/paste, it'll create a database machine then one web server.

machine 'mario' do
  recipe 'postgresql'
  recipe 'mydb'
  tag 'mydb_master'
end

num_webservers = 1

1.upto(num_webservers) do |i|
  machine "luigi#{i}" do
    recipe 'apache'
    recipe 'mywebapp'
  end
end

You will notice the dynamic nature of the number of web servers. It's all code, your imagination is the limit :)

Drivers handle the real work of getting those abstract definitions into real, physical form. They handle the following tasks, idempotently (you can run the resource again and again and it will only create the machine once--though it may notice things are wrong and fix them!):

• Acquiring machines from the cloud, creating containers or VMs, or grabbing bare metal
• Connecting to those machines via ssh, winrm, or other transports
• Bootstrapping chef onto the machines and converging the recipes you suggested

The driver API is separated out so that new drivers can be made with minimal effort (without having to rewrite ssh, tunneling, bootstrapping, and OS support). But to the user, they appear as a single thing, so that the machine acquisition can use its smarts to autodetect the other bits (transports, OS's, etc.).

Drivers save their data in the Chef node itself, so that they will be accessible to everyone who is using the Chef server to manage the nodes.

chef-provisioning-fog interacts with your cloud provider's endpoint. Most of the time, default timeouts for the following would be sufficient.

Modify the defaults if your Fog endpoint takes awhile to send/receive API requests. Normally, if you see Excon timeouts you should tune these parameters.

with_driver 'fog:foo',
  :compute_options => {
    :connection_options => {
      # set connection to persist (default is false)
      :persistent => true,
      # set longer connect_timeout (default is 60 seconds)
      :connect_timeout => 360,
      # set longer read_timeout (default is 60 seconds)
      :read_timeout => 360,
      # set longer write_timeout (default is 60 seconds)
      :write_timeout => 360,
    }
  }

Modify these timeouts if you need Chef to wait a bit of time to allow for the machine to be ready.

with_machine_options({
  :bootstrap_options => {
    # set longer to wait for the instance to boot to ssh (defaults to 180)
    :create_timeout => 360,
    # set longer to wait for the instance to start (defaults to 180)
    :start_timeout => 360,
  }
  # set longer to wait for ssh to be available if the instance is detected up (defaults to 20)
  :ssh_timeout => 360
})

Modify this if your chef client convergences take awhile.

with_machine_options({
  :convergence_options => {
    # set longer if you need more time to converge (default: 2 hours)
    :chef_client_timeout => 120*60
  }
})