vagrant-s3auth

Private, versioned Vagrant boxes hosted on Amazon S3.

From the command line:

$ vagrant plugin install vagrant-s3auth

• Vagrant, v1.5.1+

vagrant-s3auth will automatically sign requests for S3 URLs

s3://bucket.example.com/path/to/metadata

with your AWS access key.

This means you can host your team's sensitive, private boxes on S3, and use your developers' existing AWS credentials to securely grant access.

If you've already got your credentials stored in the standard environment variables:

# Vagrantfile

Vagrant.configure('2') do |config|
  config.vm.box     = 'simple-secrets'
  config.vm.box_url = 's3://example.com/secret.box'
end

AWS credentials are read from the standard environment variables AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY.

You may find it more convenient to use the centralized credential file to create a credential profile. Select the appropriate profile using the AWS_PROFILE environment variable. For example:

# ~/.aws/credentials

[vagrant-s3auth]
aws_access_key_id = AKIA...
aws_secret_access_key = ...

# Vagrantfile

ENV.delete_if { |name| name.start_with?('AWS_') }  # Filter out rogue env vars.
ENV['AWS_PROFILE'] = 'vagrant-s3auth'

Vagrant.configure("2") { |config| ... }

CAUTION: If AWS_ACCESS_KEY_ID exists in your environment, it will take precedence over AWS_PROFILE! Either take care to filter rogue environment variables as above, or set the access key explicitly:

access_key, secret_key = whizbang_inc_api.fetch_api_creds()
ENV['AWS_ACCESS_KEY_ID']     = access_key
ENV['AWS_SECRET_ACCESS_KEY'] = secret_key

The detected AWS access key and its source (environment variable or profile file) will be displayed when the box is downloaded. If you use multiple AWS credentials and see authentication errors, verify that the correct access key was detected.

IAM accounts will need at least the following policy:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "s3:GetObject",
      "Resource": "arn:aws:s3:::BUCKET/*"
    },
    {
      "Effect": "Allow",
      "Action": ["s3:GetBucketLocation", "s3:ListBucket"],
      "Resource": "arn:aws:s3:::BUCKET"
    }
  ]
}

IMPORTANT: You must split up bucket and object permissions into separate policy statements as written above! See Writing IAM Policies: How to grant access to an Amazon S3 Bucket.

Also note that s3:ListBucket permission is not strictly necessary. vagrant-s3auth will never make a ListBucket request, but without ListBucket permission, a misspelled box name results in a 403 Forbidden error instead of a 404 Not Found error. (Why?)

See AWS S3 Guide: User Policy Examples for more.

You can use any valid HTTP(S) URL for your object:

# path style
http://s3.amazonaws.com/bucket/resource
https://s3.amazonaws.com/bucket/resource

# host style
http://bucket.s3.amazonaws.com/resource
https://bucket.s3.amazonaws.com/resource

Or the S3 protocol shorthand

s3://bucket/resource

which expands to the path-style HTTPS URL.

If your bucket is not hosted in the US Standard region, you'll need to specify the correct region endpoint as part of the URL:

https://s3-us-west-2.amazonaws.com/bucket/resource
https://bucket.s3-us-west-2.amazonaws.com/resource

Or just use the S3 protocol shorthand, which will automatically determine the correct region at the cost of an extra API call:

s3://bucket/resource

For additional details on specifying S3 URLs, refer to the S3 Developer Guide: Virtual hosting of buckets.

Simply point your box_url at a supported S3 URL:

Vagrant.configure('2') do |config|
  config.vm.box     = 'simple-secrets'
  config.vm.box_url = 'https://s3.amazonaws.com/bucket.example.com/secret.box'
end

If you've got a box version on Vagrant Cloud, just point it at a supported S3 URL:

Then configure your Vagrantfile like normal:

Vagrant.configure('2') do |config|
  config.vm.box = 'benesch/test-box'
end

Metadata boxes were added to Vagrant in 1.5 and power Vagrant Cloud. You can host your own metadata and bypass Vagrant Cloud entirely.

Essentially, you point your box_url at a JSON metadata file that tells Vagrant where to find all possible versions:

# Vagrantfile

Vagrant.configure('2') do |config|
  config.vm.box     = 'examplecorp/secrets'
  config.vm.box_url = 's3://example.com/secrets'
end

"s3://example.com/secrets"

{
  "name": "examplecorp/secrets",
  "description": "This box contains company secrets.",
  "versions": [{
    "version": "0.1.0",
    "providers": [{
      "name": "virtualbox",
      "url": "https://s3.amazonaws.com/example.com/secrets.box",
      "checksum_type": "sha1",
      "checksum": "foo"
    }]
  }]
}

Within your metadata JSON, be sure to use supported S3 URLs.

Note that the metadata itself doesn't need to be hosted on S3. Any metadata that points to a supported S3 URL will result in an authenticated request.

IMPORTANT: Your metadata must be served with Content-Type: application/json or Vagrant will not recognize it as metadata! Most S3 uploader tools (and most webservers) will not automatically set the Content-Type header when the file extension is not .json. Consult your tool's documentation for instructions on manually setting the content type.

The beauty of Vagrant is the magic of "vagrant up and done." Making your users install a plugin is lame.

But wait! Just stick some shell in your Vagrantfile:

unless Vagrant.has_plugin?('vagrant-s3auth')
  # Attempt to install ourself. Bail out on failure so we don't get stuck in an
  # infinite loop.
  system('vagrant plugin install vagrant-s3auth') || exit!

  # Relaunch Vagrant so the plugin is detected. Exit with the same status code.
  exit system('vagrant', *ARGV)
end