Cmds
Cmds
tries to make it easier to read, write and remember using shell commands in Ruby.
It treats generating shell the in a similar fashion to generating SQL or HTML.
Best read at
http://www.rubydoc.info/gems/cmds/
where the API doc links should work and you got a table and contents.
Status
Ya know, before you get too excited...
It's kinda starting to work. I'll be using it for stuff and seeing how it goes, but no promises until 1.0
of course.
License
MIT
Real-World Examples
Or, "what's it look like?"...
-
Instead of
`psql \ --username=#{ (db_config['username'] || ENV['USER']).shellescape } \ #{ db_config['database'].shellescape } \ < #{ filepath.shellescape }`
write
Cmds 'psql %{opts} %{db} < %{dump}', db: db_config['database'], dump: filepath, opts: { username: db_config['username'] || ENV['USER'] }
to run a command like
psql --username=nrser that_db < ./some/file/path
Cmds takes care of shell escaping for you.
-
Instead of
`PGPASSWORD=#{ config[:password].shellescape } \ pg_dump \ --username=#{ config[:username].shellescape } \ --host=#{ config[:host].shellescape } \ --port=#{ config[:port].shellescape } \ #{ config[:database].shellescape } \ > #{ filepath.shellescape }`
which can be really hard to pick out what's going on from a quick glance, write
Cmds.new( 'pg_dump %{opts} %{database}', kwds: { opts: { username: config[:username], host: config[:host], port: config[:port], }, database: config[:database], }, env: { PGPASSWORD: config[:password], }, ).stream! { |io| io.out = filename }
I find it much easier to see what's going on their quickly.
Again, with some additional comments and examples:
# We're going to instantiate a new {Cmds} object this time, because we're # not just providing values for the string template, we're specifying an # environment variable for the child process too. # cmd = Cmds.new( # The string template to use. 'pg_dump %{opts} %{database}', kwds: { # Hashes will automatically be expanded to CLI options. By default, # we use `--name=VALUE` format for long ones and `-n VALUE` for short, # but it's configurable. opts: { username: config[:username], host: config[:host], port: config[:port], }, # As mentioned above, everything is shell escaped automatically database: config[:database], }, # Pass environment as it's own Hash. There are options for how it is # provided to the child process as well. env: { # Symbol keys are fine, we'll take care of that for you PGPASSWORD: config[:password], }, ) # Take a look! cmd.prepare # => "PGPASSWORD=shhh\\! pg_dump --host=localhost --port=5432 --username=nrser blah" # Now stream it. the `!` means raise if the exit code is not 0 exit_code = cmd.stream! { |io| # We use the block to configure I/O. Here we send the standard output to # a file, which can be a String, Pathname or IO object io.out = filename }
Installation
Add this line to your application's Gemfile
:
gem 'cmds'
And then execute:
bundle install
Or install it globally with:
gem install cmds
Overview
Cmds is based around a central {Cmds} class that takes a template for the command and a few options and operates by either wrapping the results in a {Cmds::Result} instance or streaming the results to IO
objects or handler blocks.
Features
Templates
ERB
Templates are processed with "Embedded Ruby" (eRuby/ERB) using the Erubis gem.
For how it works check out
- {Cmds::ERBContext}
- {Cmds::ShellEruby}
- {Cmds#render}
Positional Values from args
-
Use the
args
array made available in the templates with entry indexes.Example when constructing:
Cmds.new( 'cp <%= args[0] %> <%= args[1] %>', args: [ 'source.txt', 'dest.txt', ], ).prepare # => "cp source.txt dest.txt"
This will raise an error if it's called after using the last positional argument, but will not complain if all positional arguments are not used.
-
Use the
arg
method made available in the templates to get the next positional arg.Example when using "sugar" methods that take
args
as the single-splat (*args
):Cmds.prepare 'cp <%= arg %> <%= arg %>', 'source.txt', 'dest.txt' # => "cp source.txt dest.txt"
Keyword Values from kwds
Just use the key as the method name.
When constructing:
Cmds.new(
'cp <%= src %> <%= dest %>',
kwds: {
src: 'source.txt',
dest: 'dest.txt',
},
).prepare
# => "cp source.txt dest.txt"
When using "sugar" methods that take kwds
as the double-splat (**kwds
):
Cmds.prepare 'cp <%= src %> <%= dest %>',
src: 'source.txt',
dest: 'dest.txt'
# => "cp source.txt dest.txt"
Key Names to Avoid
If possible, avoid naming your keys:
arg
args
initialize
get_binding
method_missing
If you must name them those things, don't expect to be able to access them as shown above; use <%= @kwds[key] %>
.
Keys That Might Not Be There
Normally, if you try to interpolate a key that doesn't exist you will get a KeyError
:
Cmds.prepare "blah <%= maybe %> <%= arg %>", "value"
# KeyError: couldn't find keys :maybe or "maybe" in keywords {}
I like a lot this better than just silently omitting the value, but sometimes you know that they key might not be set and want to receive nil
if it's not.
In this case, append ?
to the key name (which is a method call in this case) and you will get nil
if it's not set:
Cmds.prepare "blah <%= maybe? %> <%= arg %>", "value"
# => "blah value"
Cmds.prepare "blah <%= maybe? %> <%= arg %>", "value", maybe: "yes"
# => "blah yes value"
Shell Escaping
Cmds automatically shell-escapes values it interpolates into templates by passing them through the Ruby standard libray's Shellwords.escape.
Cmds.prepare "cp <%= src %> <%= dest %>",
src: "source.txt",
dest: "path with spaces.txt"
=> "cp source.txt path\\ with\\ spaces.txt"
It doesn't always do the prettiest job, but it's part of the standard library and seems to work pretty well... shell escaping is a messy and complicated topic (escaping for which shell?!), so going with the built-in solution seems reasonable for the moment, though I do hate all those backslashes... they're a pain to read.
Raw Interpolation
You can render a raw string with <%== %>
.
To see the difference with regard to the previous example (which would break the cp
command in question):
Cmds.prepare "cp <%= src %> <%== dest %>",
src: "source.txt",
dest: "path with spaces.txt"
=> "cp source.txt path with spaces.txt"
And a way it make a little more sense:
Cmds.prepare "<%== bin %> <%= *args %>",
'blah',
'boo!',
bin: '/usr/bin/env echo'
=> "/usr/bin/env echo blah boo\\!"
Splatting (*
) To Render Multiple Shell Tokens
Render multiple shell tokens (individual strings the shell picks up - basically, each one is an entry in ARGV
for the child process) in one expression tag by prefixing the value with *
:
Cmds.prepare '<%= *exe %> <%= cmd %> <%= opts %> <%= *args %>',
'x', 'y', # <= these are the `args`
exe: ['/usr/bin/env', 'blah'],
cmd: 'do-stuff',
opts: {
really: true,
'some-setting': 'dat-value',
}
# => "/usr/bin/env blah do-stuff --really --some-setting=dat-value x y"
ARGV
tokenization by the shell would look like:
[
'/usr/bin/env',
'blah',
'do-stuff',
'--really',
'--some-setting=dat-value',
'x',
'y',
]
-
Compare to without splats:
Cmds.prepare '<%= exe %> <%= cmd %> <%= opts %> <%= args %>', 'x', 'y', # <= these are the `args` exe: ['/usr/bin/env', 'blah'], cmd: 'do-stuff', opts: { really: true, 'some-setting': 'dat-value', } # => "/usr/bin/env,blah do-stuff --really --some-setting=dat-value x,y"
Which is probably not what you were going for... it would produce an
ARGV
something like:[ '/usr/bin/env,blah', 'do-stuff', '--really', '--some-setting=dat-value', 'x,y', ]
You can of course use "splatting" together with slicing or mapping or whatever.
Logic
All of ERB is available to you. I've tried to put in features and options that make it largely unnecessary, but if you've got a weird or complicated case, or if you just like the HTML/Rails-esque templating style, it's there for you:
cmd = Cmds.new <<-END
<% if use_bin_env %>
/usr/bin/env
<% end %>
docker build .
-t <%= tag %>
<% if file %>
--file <%= file %>
<% end %>
<% build_args.each do |key, value| %>
--build-arg <%= key %>=<%= value %>
<% end %>
<% if yarn_cache %>
--build-arg yarn_cache_file=<%= yarn_cache_file %>
<% end %>
END
cmd.prepare(
use_bin_env: true,
tag: 'nrser/blah:latest',
file: './prod.Dockerfile',
build_args: {
yarn_version: '1.3.2',
},
yarn_cache: true,
yarn_cache_file: './yarn-cache.tgz',
)
# => "/usr/bin/env docker build . -t nrser/blah:latest
# --file ./prod.Dockerfile --build-arg yarn_version=1.3.2
# --build-arg yarn_cache_file=./yarn-cache.tgz"
# (Line-breaks added for readability; output is one line)
printf
-Style Short-Hand (%s
, %{key}
, %<key>s
)
Cmds also supports a printf-style short-hand. Sort-of.
It's a clumsy hack from when I was first writing this library, and I've pretty moved to using the ERB-style, but there are still some examples that use it, and I guess it still works (to whatever extent it ever really did), so it's probably good to mention it.
It pretty much just replaces some special patterns with their ERB-equivalent via the {Cmds.replace_shortcuts} method before moving on to ERB processing:
Format | ERB Replacement |
---|---|
%s |
<%= arg %> |
%{key} |
<%= key %> |
%{key?} |
<%= key? %> |
%<key>s |
<%= key %> |
%<key?>s |
<%= key? %> |
And the escaping versions, where you can put anothe %
in front to get the literal intead of the subsitution:
Format | ERB Replacement |
---|---|
%%s |
%s |
%%{key} |
%{key} |
%%{key?} |
%{key?} |
%%<key>s |
%<key>s |
%%<key?>s |
%<key?>s |
That's it. No printf
formatting beyond besides s
(string).
Old docs I haven't cleaned up yet...
execution
you can provide three types of arguments when executing a command:
- positional arguments for substitution
- keyword arguments for substitution
- input to stdin
all Cmds
instance execution methods have the same form for accepting these:
-
positional arguments are provided in an optional array that must be the first argument:
Cmds "cp <%= arg %> <%= arg %>", [src_path, dest_path]
note that the arguments need to be enclosed in square braces. Cmds does NOT use *splat for positional arguments because it would make a
Hash
final parameter ambiguous. -
keyword arguments are provided as optional hash that must be the last argument:
Cmds "cp <%= src %> <%= dest %>", src: src_path, dest: dest_path
in this case, curly braces are not required since Ruby turns the trailing keywords into a
Hash
provided as the last argument (or second-to-last argument in the case of a block included in the method signature). -
input and output is handled with blocks:
`Cmds(“wc -l”){ “one\ntwo\nthree\n” }
Cmds.stream './test/tick.rb <%= times %>', times: times do |io| io.on_out do |line| # do something with the output line end
io.on_err do |line| # do something with the error line end end`
Reuse Commands
playbook = Cmds.new "ansible-playbook -i %{inventory} %{playbook}"
playbook.call inventory: "./hosts", playbook: "blah.yml"
currying
dev_playbook = playbook.curry inventory: "inventory/dev"
prod_playbook = playbook.curry inventory: "inventory/prod"
# run setup.yml on the development hosts
dev_playbook.call playbook: "setup.yml"
# run setup.yml on the production hosts
prod_playbook.call playbook: "setup.yml"
defaults
NEEDS TEST
can be accomplished with reuse and currying stuff
playbook = Cmds.new "ansible-playbook -i %{inventory} %{playbook}", inventory: "inventory/dev"
# run setup.yml on the development hosts
playbook.call playbook: "setup.yml"
# run setup.yml on the production hosts
prod_playbook.call playbook: "setup.yml", inventory: "inventory/prod"
input
c = Cmds.new("wc", input: "blah blah blah).call
future..?
exec
want to be able to use to exec commands
formatters
kinda like sprintf
formatters or string escape helpers in Rails, they would be exposed as functions in ERB and as format characters in the shorthand versions:
Cmds "blah <%= j obj %>", obj: {x: 1}
# => blah \{\"x\":1\}
Cmds "blah %j", [{x: 1}]
# => blah \{\"x\":1\}
Cmds "blah %<obj>j", obj: {x: 1}
# => blah \{\"x\":1\}
the s
formatter would just format as an escaped string (no different from <%= %>
).
other formatters could include
-
j
for JSON (as shown above) -
r
for raw (unescaped) -
l
or,
for comma-separated list (which some commands like as input) -
y
for YAML -
p
for path, joining withFile.join