Puppet Forge Server

Private Puppet Forge Server supporting local files and both v1 and v3 API proxies.

Puppet Forge Server provides approximated implementation of both v1 and v3 APIs, but behavioral deviations from the official implementation might occur.

Puppet 2, 3 and 4 as well as librarian-puppet are supported.

• Installation
  • Dependencies
• Getting Started
  • Proxy
    • Proxy the official Puppet Forge v3 API
    • Proxy the official Puppet Forge v1 API and local Pulp puppet repository
  • Locally stored modules
  • All-in
    • r10k
    • librarian-puppet
  • Daemon
• Web UI
• Architecture
  • API (view)
  • App (controller)
  • Models
  • Backends
• Limitations
• Reference

Install the gem

gem install puppet-forge-server
puppet-forge-server --help

or get the latest source

git clone https://github.com/unibet/puppet-forge-server
cd puppet-forge-server
bundle install
bundle exec bin/puppet-forge-server --help

To install the Puppet Forge Server with puppet, you can use this module: https://github.com/unibet/puppet-forge_server

The gem installtion requires the ruby development packages and GCC.

Red Hat (and derivitives) yum install gcc ruby-devel

Debian (and derivitives) apt-get install gcc ruby-dev

Just start the server providing the forge URL

bundle exec bin/puppet-forge-server -x https://forgeapi.puppetlabs.com

Run puppet module install to test it

puppet module install --module_repository=http://localhost:8080 puppetlabs-stdlib

Just start the server providing forge and local pulp URLs

bundle exec bin/puppet-forge-server -x http://forge.puppetlabs.com -x http://my.local.pulp/pulp_puppet/forge/repository/demo

Run puppet module install to test it. Please note that depending on your internet connection and puppet version it might take a while for the first call to get executed. v3 API requires checksuming the files, which means all involved module files will be cached.

puppet module install --module_repository=http://localhost:8080 puppetlabs-stdlib

Download given modules from the official forge and start the server pointing into directory with module files

mkdir modules
wget -P modules/ forge.puppetlabs.com/system/releases/p/puppetlabs/puppetlabs-apache-0.9.0.tar.gz
wget -P modules/ forge.puppetlabs.com/system/releases/p/puppetlabs/puppetlabs-concat-1.0.0.tar.gz
wget -P modules/ forge.puppetlabs.com/system/releases/p/puppetlabs/puppetlabs-stdlib-2.4.0.tar.gz
bundle exec bin/puppet-forge-server -m modules/

Run puppet module install to test it

puppet module install --module_repository=http://localhost:8080 puppetlabs-stdlib

Download given modules from the official forge and start the server pointing into directory with module files and proxy URLs

mkdir modules
wget -P modules/ forge.puppetlabs.com/system/releases/p/puppetlabs/puppetlabs-apache-0.9.0.tar.gz
wget -P modules/ forge.puppetlabs.com/system/releases/p/puppetlabs/puppetlabs-concat-1.0.0.tar.gz
wget -P modules/ forge.puppetlabs.com/system/releases/p/puppetlabs/puppetlabs-stdlib-2.4.0.tar.gz
bundle exec bin/puppet-forge-server -m modules/ -x https://forgeapi.puppetlabs.com -x http://my.local.pulp/pulp_puppet/forge/repository/demo

Create an example Puppetfile

cat > Puppetfile <<EOF
mod 'puppetlabs/apache'
EOF

Add forge/baseurl to r10k.yaml

cat >> r10k.yaml <<EOF
forge:
  baseurl: 'http://localhost:8080'
EOF

Create an example Puppetfile

cat > Puppetfile <<EOF
forge 'http://localhost:8080'

mod 'puppetlabs/apache'
EOF

Run librarian-puppet with --no-use-v1-api option to instruct it to use v3 API for better performance

librarian-puppet install --no-use-v1-api

Normally one would want to run server as a deamon:

# Assuming puppet-forge-server gem was installed
# Create deamon user
sudo adduser forge -d /opt/forge -s /bin/false
# Create log, cache and modules directories
sudo -u forge mkdir -p /opt/forge/log /opt/forge/modules /opt/forge/cache
# Start the server
sudo -u forge puppet-forge-server -D -m /opt/forge/modules -x https://forgeapi.puppetlabs.com --log-dir /opt/forge/log --cache-basedir /opt/forge/cache --pidfile /opt/forge/server.pid

You are done. Now go install some puppet modules.

Apache virtualhost config:

<VirtualHost *:80>
    ServerName localhost
    DocumentRoot /opt/forge/public
    <Directory /opt/forge/public>
        Allow from all
        Options -MultiViews
        Require all granted
    </Directory>
</VirtualHost>

Create a config.ru one folder down from the Apache DocumentRoot, eg: /opt/forge/config.ru:

require 'rubygems'
require 'puppet_forge_server'

# Set base cache directory for proxy backends 
cache_dir = '/opt/forge/cache' # default: File.join(Dir.tmpdir.to_s, 'puppet-forge-server', 'cache')

# Create backends
backends = [
  PuppetForgeServer::Backends::Directory.new('/opt/forge/modules'),
  # Add directory backend for serving cached modules in case proxy flips over
  PuppetForgeServer::Backends::Directory.new(cache_dir),
  PuppetForgeServer::Backends::ProxyV3.new('https://forgeapi.puppetlabs.com', cache_dir)
]

# Disable access logging, log errors to STDERR
PuppetForgeServer::Logger.set({:server => STDERR, :access => File.open(File::NULL, "w")})

# Run
run PuppetForgeServer::Server.new.build(backends, PuppetForgeServer::Utils::OptionParser.class_eval('@@DEFAULT_WEBUI_ROOT'))

You can now connect to http://localhost and see the web interface, start using and adding modules in the same way as you would running as a Daemon.

Puppet forge server comes with built-in web UI looking very similar to the official puppet forge web page and providing a simple module search feature. Each view haml file corresponds to the request endpoint; for example / or index is formed by the index.haml located in the views directory and obviously combined with layout.haml that is being refered to during any request.

It is possible to set an external web UI root directory containing at least views directory with required haml files. See https://github.com/unibet/puppet-forge-server/tree/master/lib/puppet_forge_server/app/views for built-in reference implementation.

Code is structured with MVC in mind to allow easier maintenance and readability

API classes (actually modules only) are used to extend Sinatra application classes. Every module corresponds to official API endpoint and used to present received model data in fasion required by the given API version.

Every App class is a Sinatra application class and is responsible for mapping API endpoints, querying backends for requested data and providing the results using API (view) modules.

Puppet module metadata json representation is used as a main business model.

Backend classes are providing the means of fetching required data and creating model instances.

1. Modulefile is not supported with the directory backend

Apache 2.0

• Puppet Anvil