Bunchr
Overview
Bunchr is a Rake/Ruby-based DSL for building and bundling complex software projects into various package formats, ie: RPM, DEB, etc (packaging performed by fpm.
Originally developed to create "omnibus" style packages that include an entire ruby stack along with one or more gems, but useful for general compilation and packaging as well.
It is typically intended to be used in conjunction with Vagrant but can be used without Vagrant as well.
Motivation
Bunchr was conceived as a tool to help the Sensu monitoring project find a better way to create packages that would be as easy as possible to deploy on a variety of platforms with minimal friction and with little or no knowledge about the idiosyncrasies of the Ruby universe (version incompatibilities, gem conflicts, etc). This was particularly desirable for Sensu because one of the components is an agent that will be installed on most or all servers in an infrastructure. Thus, the package should be easy to install and should not interfere with any other Ruby apps or scripts on the server.
About the time the Sensu project was discussing a new packaging approach, Adam Jacob from Opscode/Chef took notice and suggested a slightly different approach that achieved the same goals. The approach was called 'omnibus' and was already in use at Opscode to produce a simple and uniform installer for Chef.
Opscode was using a Clojure-based tool at the time called omnibus and were working on a Ruby-based version of omnibus as well.
However, the Ruby based Omnibus was not available at the time, so Bunchr was created and it re-implements many of the concepts of the Clojure-based Omnibus but on top of Ruby / Rake with a few differences.
Installation
gem install bunchr
Usage
The simplest workflow is:
- Create a
Rakefile
- Define 1 or more
Bunchr::Software
objects - Define 1
Bunchr::Packages
object. - run
rake
- receive new packages (.rpm or .deb, depending on platform) !
For details on the DSL and examples, continue reading.
DSL Overview
A typical Bunchr project is comprised of one or more Bunchr::Software
objects
and a single Bunchr::Packages
object. Define these all in a single Rakefile
or
separate them into files for readability and re-use.
Software
objects are responsible for downloading, building, and installing
a single software component, such as zlib
or ruby
. Software
objects
support platform-specific compilation options by making Ohai data available
for decision making.
Packages
objects are used to combine Software
objects into a single package.
It will automatically try to generate any packages supported by the current
platform, for example, RPMs will be built if the platform supports it, DEB
will be built if the platform supports it.
The goal is to be able to use a single code base to build and package a project on multiple platforms.
Currently supported package types:
- tar.gz
- .deb
- .rpm
Packages DSL
A project will typically only contain a single Bunchr::Packages
object which
is essentially a wrapper around fpm
.
The Ohai platform
data will be used to determined what type of packages can be
built on the current system. Typically you would run the same Bunchr code on a
Debian based box to build the .deb, and a Redhat based box to build the .rpm.
Vagrant can help with this.
Example Rakefile
:
require 'bunchr'
Bunchr.build_dir = '/tmp/build'
Bunchr.install_dir = '/opt/sensu'
Bunchr.load_recipes Dir['recipes/**/*.rake']
Bunchr::Packages.new do |t|
t.name = 'sensu'
t.version = ENV['SENSU_VERSION'] || '0.9.5'
t.iteration = ENV['BUILD_NUMBER'] || '1'
t.category = 'Monitoring'
t.license = 'MIT License'
t.vendor = 'Sonian Inc.'
t.url = 'https://github.com/sonian/sensu'
t.description = 'A monitoring framework that aims to be simple, malleable, and scalable. Publish/subscribe model.'
case t.ohai.platform_family
when 'debian'
t.scripts[:after_install] = 'pkg_scripts/deb/postinst'
t.scripts[:before_remove] = 'pkg_scripts/deb/prerm'
t.scripts[:after_remove] = 'pkg_scripts/deb/postrm'
when 'rhel', 'fedora'
t.scripts[:before_install] = 'pkg_scripts/rpm/pre'
t.scripts[:after_install] = 'pkg_scripts/rpm/post'
t.scripts[:before_remove] = 'pkg_scripts/rpm/preun'
end
t.include_software('ruby')
t.include_software('sensu')
t.include_software('sensu_dashboard')
t.include_software('sensu_configs')
t.include_software('sensu_bin_stubs')
t.files << Bunchr.install_dir # /opt/sensu
t.files << '/usr/share/sensu'
t.files << '/var/log/sensu'
t.files << '/etc/init.d/sensu-api'
t.files << '/etc/init.d/sensu-client'
t.files << '/etc/init.d/sensu-server'
t.files << '/etc/init.d/sensu-dashboard'
# need to enumerate config files for fpm
# these are installed from recipe/sensu_configs.rake
t.config_files << "/etc/sensu/handlers/default"
t.config_files << "/etc/sensu/conf.d/client.json"
t.config_files << "/etc/sensu/conf.d/checks.json"
t.config_files << "/etc/sensu/conf.d/handlers.json"
t.config_files << "/etc/sensu/config.json"
t.config_files << '/etc/logrotate.d/sensu'
end
# default task executed when `rake` is run with no args.
task :default => ['packages:sensu']
Let's break it all down:
-
Bunchr.build_dir
- Required. This variable is available to allBunchr::Software
recipes specifying a temporary directory used for downloading and compiling. -
Bunchr.install_dir
- Required. This variable is available to allBunchr::Software
recipes. It will typically be the base directory where all software is installed.
The following variables are used to drive fpm
when building packages:
-
t.name
,t.version
,t.iteration
- Required. Used byfpm
to construct the package names, ie:name-version-iteration-arch.rpm
-
t.arch
- Optional. Can be used to override the default detected architecture, eg:all
ornoarch
. -
t.category
,t.license
,t.vendor
,t.url
,t.description
- Optional. Package metadata. -
t.include_software(name)
- Optional. This is used to tell Bunchr whatSoftware
components should be built and installed before creating packages. -
t.scripts
- Optional. A hash with keys::after_install
,:before_install
,:after_remove
, and:before_remove
. The specified files will be included with the packages. -
t.files
- Required. An array of files and directories to include. -
t.config_files
- Optional. An array of files that will be marked as configuration files (if supported by the underlying package type). Config_files are automatically added to thet.files
array.NOTE: you must specify individual files, not directories.
All tasks created by a Bunchr::Packages
object are prefixed into the
packages:
namespace. To see the generated tasks:
$ rake -T packages
rake packages:sensu # Create bunchr packages for sensu 0.9.5-1
rake packages:sensu:build # Build all packages: sensu-0.9.5-1-x86_64
rake packages:sensu:build_deb # Build deb: sensu-0.9.5-1-x86_64
rake packages:sensu:build_rpm # Build RPM: sensu-0.9.5-1-x86_64
rake packages:sensu:build_tarball # Build tarball: sensu-0.9.5-1-x86_64.tar.gz
The main task is packages:#{name}
. Exec this task to create all relevant
packages.
Software DSL
- Example software recipes: https://github.com/joemiller/bunchr/tree/master/example_recipes
Example recipe for building and installing ruby
:
Bunchr::Software.new do |t|
t.name = 'ruby'
t.version = '1.9.3-p125'
t.depends_on('autoconf')
t.depends_on('zlib')
t.depends_on('openssl')
t.depends_on('libyaml')
install_prefix = "#{Bunchr.install_dir}/embedded"
os = t.ohai['os']
arch = t.ohai['kernel']['machine']
t.download_commands << "curl -O http://ftp.ruby-lang.org/pub/ruby/1.9/ruby-1.9.3-p125.tar.gz"
t.download_commands << "tar xfvz ruby-1.9.3-p125.tar.gz"
if os == 'darwin' && arch == 'x86_64'
t.build_environment['LDFLAGS'] = "-arch x86_64 -R#{install_prefix}/lib -L#{install_prefix}/lib -I#{install_prefix}/include"
t.build_environment['CFLAGS'] = "-arch x86_64 -m64 -L#{install_prefix}/lib -I#{install_prefix}/include"
elsif os == 'linux'
t.build_environment['LDFLAGS'] = "-Wl,-rpath #{install_prefix}/lib -L#{install_prefix}/lib -I#{install_prefix}/include"
t.build_environment['CFLAGS'] = "-L#{install_prefix}/lib -I#{install_prefix}/include"
elsif os == 'solaris2'
t.build_environment['LDFLAGS'] = "-R#{install_prefix}/lib -L#{install_prefix}/lib -I#{install_prefix}/include"
t.build_environment['CFLAGS'] = "-L#{install_prefix}/lib -I#{install_prefix}/include"
end
t.build_commands << "./configure --prefix=#{install_prefix} \
--with-opt-dir=#{install_prefix} \
--enable-shared \
--disable-install-doc"
t.build_commands << "make"
t.install_commands << "make install"
CLEAN << install_prefix
end
-
t.name
- Required. Name of the software component. -
t.version
- Optional. Version of the software component. -
t.depends_on(name)
- Optional. This is used to tell Bunchr that otherSoftware
components should be built before this one. -
t.ohai
- This provides access to theBunchr::Ohai
object which contains Ohai data about the system. By default, only theplatform
andos
plugins are loaded. Since you have direct access to the Ohai object, other plugins can be loaded by callingt.ohai.require_plugin
-
t.download_commands
- An array of shell commands used to download and uncompress the source. You could also do agit clone ..
here. These commands are executed in thedownload_dir
which is#{Bunchr.build_dir}/#{t.name}
. Thedownload_dir
cannot be changed. -
t.build_commands
- An array of shell commands used to compile the downloaded source. These commands are executed in the directory defined byt.work_dir
which will be automatically determined from:-
#{download_dir}/#{t.name}-#{t.version}
(common for most tarballs), or -
#{download_dir}/#{t.name}
(if not.version
set), or - explicitly set by calling
t.work_dir = '/some/absolute/path'
-
-
t.install_commands
- An array of shell commands used to download and uncompress the source. You could also do agit clone ..
here. These commands are executed int.work_dir
directory. -
CLEAN
- Optional. This is an array of files and directories that should be deleted whenrake clean
is executed.
download_commands
, build_commands
, and install_commands
are all optional,
but unless one of them contains some commands your Software
object won't be
doing anything useful.
If any of download_commands
, build_commands
, or install_commands
exit
with a non-zero status, the entire Bunchr process will stop
and print any STDOUT or STDERR from the failed command to the Logger.
Bunchr will keep track of what has succeeded so that you can restart a failed
build after fixing an error. This can save quite a bit of time during package
development, but you should consider doing a full rake clean
before building
any official packages.
All tasks created by a Bunchr::Software
object are prefixed into the
software:
namespace. To see the generated tasks:
$ rake -T software:ruby
rake software:ruby # Download, build, and install ruby 1.9.3-p125
rake software:ruby:build # Build ruby 1.9.3-p125
rake software:ruby:download # Download ruby 1.9.3-p125
rake software:ruby:install # Install ruby 1.9.3-p125
Software
recipes can be defined directly in the Rakefile or they can
be separated into individual files and loaded via Bunchr.load_recipes(files)
.
Integration with Vagrant
- TODO. maybe link to sensu-bunchr here.
Other Examples
- TODO. maybe link to sensu-bunchr here, or a complete example of fpm
Author
License
Author:: Joe Miller (<joeym@joeym.net>)
Copyright:: Copyright (c) 2012 Joe Miller
License:: Apache License, Version 2.0
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.