puppet_metadata

The gem intends to provide an abstraction over Puppet's metadata.json file. Its API allow easy iteration over its illogical data structures.

To get outputs usable in Github Actions, there is the metadata2gha command available. This generates based on metadata.json, such as Beaker setfiles, Puppet major versions and a Puppet unit test matrix.

$ metadata2gha
puppet_major_versions=[{"name":"Puppet 8","value":8,"collection":"puppet8"},{"name":"Puppet 7","value":7,"collection":"puppet7"}]
puppet_unit_test_matrix=[{"puppet":8,"ruby":"3.2"},{"puppet":7,"ruby":"2.7"}]
puppet_beaker_test_matrix=[{"name":"Puppet 8 - Debian 12","env":{"BEAKER_PUPPET_COLLECTION":"puppet8","BEAKER_SETFILE":"debian12-64{hostname=debian12-64-puppet8}"}},{"name":"Puppet 7 - Debian 12","env":{"BEAKER_PUPPET_COLLECTION":"puppet7","BEAKER_SETFILE":"debian12-64{hostname=debian12-64-puppet7}"}}]

Puppet major versions formatted for readability:

[
  {
    "name": "Puppet 8",
    "value": 8,
    "collection": "puppet8"
  },
  {
    "name": "Puppet 7",
    "value": 7,
    "collection": "puppet7"
  }
]

Puppet unit test matrix formatted for readability:

[
  {
    "puppet": 8,
    "ruby": "3.2"
  },
  {
    "puppet": 7,
    "ruby": "2.7"
  }
]

Beaker test matrix formatted for readability

[
  {
    "name": "Puppet 8 - Debian 12",
    "env": {
      "BEAKER_PUPPET_COLLECTION": "puppet8",
      "BEAKER_SETFILE": "debian12-64{hostname=debian12-64-puppet8}"
    }
  },
  {
    "name": "Puppet 7 - Debian 12",
    "env": {
      "BEAKER_PUPPET_COLLECTION": "puppet7",
      "BEAKER_SETFILE": "debian12-64{hostname=debian12-64-puppet7}"
    }
  }
]

It is possible to specify the path to metadata.json and customize the setfiles. For example, to ensure the setfiles use FQDNs and apply the systemd PIDFile workaround under docker. This either means either using an older image (CentOS 7, Ubuntu 16.04) or skipping (CentOS 8).

$ metadata2gha --use-fqdn --pidfile-workaround true /path/to/metadata.json

This results in the following JSON data

[
  {
    "name": "Puppet 7 - CentOS 7",
    "env": {
      "BEAKER_PUPPET_COLLECTION": "puppet7",
      "BEAKER_SETFILE": "centos7-64{hostname=centos7-64-puppet7.example.com,image=centos:7.6.1810}"
    }
  },
  {
    "name": "Puppet 7 - Debian 12",
    "env": {
      "BEAKER_PUPPET_COLLECTION": "puppet7",
      "BEAKER_SETFILE": "debian12-64{hostname=debian12-64-puppet7.example.com}"
    }
  },
  {
    "name": "Puppet 7 - Ubuntu 22.04",
    "env": {
      "BEAKER_PUPPET_COLLECTION": "puppet7",
      "BEAKER_SETFILE": "ubuntu2204-64{hostname=ubuntu2204-64-puppet7.example.com}"
    }
  }
]

If you need custom hostname or multiple hosts in your integration tests this could be achived by using the --beaker-hosts option

Option argument is 'HOSTNAME:ROLES;HOSTNAME:..;..' where

• hosts are separated by ';'
• host number and roles are separated by ':'
• Roles follow beaker-hostgenerator syntax If you don't need any extra roles use '1;2;..'

$ metadata2gha --beaker-hosts 'foo:primary.ma;bar:secondary.a'

This results in the following JSON data

[
  {
    "name": "Puppet 7 - Debian 12",
    "env": {
      "BEAKER_PUPPET_COLLECTION": "puppet7",
      "BEAKER_SETFILE": "debian12-64primary.ma{hostname=foo-puppet7}-debian12-64secondary.a{hostname=bar-puppet7}"
    }
  }
]

If you need to Expand the matrix ie by product versions it could be achived by using the --beaker-facter option

Option argument is 'FACT:LABEL:VALUE,VALUE,..' where

• Fact, label and values are separated by ':'
• Values are separated by ','

$ metadata2gha --beaker-facter 'mongodb_repo_version:MongoDB:4.4,5.0,6.0,7.0'

This results in the following JSON data

[
  {
    "name": "Puppet 7 - Debian 12 - MongoDB 4.4",
    "env": {
        "BEAKER_PUPPET_COLLECTION": "puppet7",
        "BEAKER_SETFILE": "debian12-64{hostname=debian12-64-puppet7}",
        "BEAKER_FACTER_mongodb_repo_version": "4.4"
    }
  },
  {
    "name": "Puppet 7 - Debian 12 - MongoDB 5.0",
    "env": {
      "BEAKER_PUPPET_COLLECTION": "puppet7",
      "BEAKER_SETFILE": "debian12-64{hostname=debian12-64-puppet7}",
      "BEAKER_FACTER_mongodb_repo_version": "5.0"
    }
  },
  {
    "name": "Puppet 7 - Debian 12 - MongoDB 6.0",
    "env": {
      "BEAKER_PUPPET_COLLECTION": "puppet7",
      "BEAKER_SETFILE": "debian12-64{hostname=debian12-64-puppet7}",
      "BEAKER_FACTER_mongodb_repo_version": "6.0"
    }
  },
  {
    "name": "Puppet 7 - Debian 12 - MongoDB 7.0",
    "env": {
      "BEAKER_PUPPET_COLLECTION": "puppet7",
      "BEAKER_SETFILE": "debian12-64{hostname=debian12-64-puppet7}",
      "BEAKER_FACTER_mongodb_repo_version": "7.0"
    }
  }
]

The API can be initialised like this:

require 'puppet_metadata'
metadata = PuppetMetadata.read('/path/to/a/metadata.json')

The metadata object has several different methods that we can call

[9] pry(main)> metadata.operatingsystems
=> {"Archlinux"=>nil, "Gentoo"=>nil, "Fedora"=>["32", "33"], "CentOS"=>["7", "8"], "RedHat"=>["7", "8"], "Ubuntu"=>["18.04"], "Debian"=>["9", "10"], "VirtuozzoLinux"=>["7"]}
[10] pry(main)>

[10] pry(main)> metadata.puppet_major_versions
=> [6, 7]
[11] pry(main)>

[6] pry(main)> metadata.os_release_supported?('Archlinux', nil)
=> true
[7] pry(main)>

When running beaker on the CLI, you can specify a specific setfile. puppet_metadata provides bin/setfiles to list all setfiles:

$ bundle exec setfiles
Skipping EOL operating system Debian 10
Skipping EOL operating system Ubuntu 18.04
BEAKER_SETFILE="centos9-64{hostname=centos9-64-puppet8.example.com}"
BEAKER_SETFILE="centos9-64{hostname=centos9-64-puppet7.example.com}"
BEAKER_SETFILE="debian11-64{hostname=debian11-64-puppet8.example.com}"
BEAKER_SETFILE="debian11-64{hostname=debian11-64-puppet7.example.com}"
BEAKER_SETFILE="ubuntu2004-64{hostname=ubuntu2004-64-puppet8.example.com}"
BEAKER_SETFILE="ubuntu2004-64{hostname=ubuntu2004-64-puppet7.example.com}"

As an argument you can provide a path to a metadata.json. If none provided, it assumes that there's a metadata.json in the same directory where you run the command. To make copy and paste easier, each setfile string is prefixed so it can directly be used as environment variable.

This plugin was originally authored by Ewoud Kohl van Wijngaarden. The maintainer preferred that Vox Pupuli take ownership of the module for future improvement and maintenance. Existing pull requests and issues were transferred, please fork and continue to contribute here.

This gem is licensed under the Apache-2 license.

To make a new release, please do:

• Update the version in the puppet_metadata.gemspec file
• Install gems with bundle install --with release --path .vendor
• generate the changelog with bundle exec rake changelog
• Create a PR with it
• After it got merged, push a tag. A github workflow will do the actual release