Coque

Create, manage, and interop with shell pipelines from Ruby. Like Plumbum, for Ruby, with native (Ruby) code streaming integration.

Add to your gemfile:

gem 'coque'

Create Coque commands:

cmd = Coque["echo", "hi"]
# => <Coque::Sh ["echo", "hi"]>

And run them:

res = cmd.run
# => #<Coque::Result:0x007feb5930e408 @out=#<IO:fd 13>, @pid=58688>
res.to_a
# => ["hi"]

Or pipe them:

pipeline = cmd | Coque["wc", "-c"]
# => #<Coque::Pipeline:0x007feb598730b0 @commands=[<Coque::Sh ["echo", "hi"]>, <Coque::Sh ["wc", "-c"]>]>
pipeline.run.to_a
# => ["3"]

Coque can also create "Rb" commands, which integrate Ruby code with streaming, line-wise processing of other commands:

c1 = Coque["printf", '"a\nb\nc\n"']
c2 = Coque.rb { |line| puts line.upcase }
(c1 | c2).run.to_a
# => ["A", "B", "C"]

Rb commands can also take "pre" and "post" blocks

dict = Coque["cat", "/usr/share/dict/words"]
rb_wc = Coque.rb { @lines += 1 }.pre { @lines = 0 }.post { puts @lines }

(dict | rb_wc).run.to_a
# => ["235886"]

Commands can have Stdin, Stdout, and Stderr redirected

(Coque["echo", "hi"] > "/tmp/hi.txt").run.wait
File.read("/tmp/hi.txt")
# => "hi\n"

(Coque["head", "-n", "4"] < "/usr/share/dict/words").run.to_a
# => ["A", "a", "aa", "aal"]

(Coque["cat", "/doesntexist.txt"] >= "/tmp/error.txt").run.wait
File.read("/tmp/error.txt")
# => "cat: /doesntexist.txt: No such file or directory\n"

Coque commands can also be derived from a Coque::Context, which enables changing directory, setting environment variables, and unsetting child env:

c = Coque.context
c["pwd"].run.to_a
# => ["/Users/worace/code/coque"]

Coque.context.chdir("/tmp")["pwd"].run.to_a
# => ["/private/tmp"]

Coque.context.setenv("my_key": "pizza")["echo", "$my_key"].run.to_a
# => ["pizza"]

ENV["my_key"] = "pizza"
Coque["echo", "$my_key"].run.to_a
# => ["pizza"]

Coque.context.disinherit_env["echo", "$my_key"].to_a
# => [""]

Coque also includes a Coque.source helper for feeding Ruby enumerables into shell pipelines:

(Coque.source(1..500) | Coque["wc", "-l"]).run.to_a
# => ["500"]

Running a Coque command forks a new process, and by default these processes run asynchronously. Calling .run on a Coque command or pipeline returns a Coque::Result object which can be used to get the output (.to_a) or exit code (.exit_code) of the process:

result = Coque['echo', 'hi'].run
# => #<Coque::Result:0x000055da63437838 @out=#<IO:fd 15>, @pid=29236>
puts "its running in the background..."
its running in the background...
result.to_a
# => ["hi"]
result.exit_code
# => 0

However you can also just use .wait to block on a process while it runs:

result = Coque['echo', 'hi'].run.wait
# => #<Coque::Result:0x000055da633c98b0 @exit_code=0, @out=#<IO:fd 17>, @pid=29536>

Or, use .run! to block on the process and raise an exception if it exits with a non-zero response:

Coque["head", "/usr/share/dict/words"].run!
# => nil
Coque["head", "/usr/share/dict/pizza"].run!
# head: cannot open '/usr/share/dict/pizza' for reading: No such file or directory
# RuntimeError: Coque Command Failed: <Coque::Sh head /usr/share/dict/pizza>
from /home/horace/.gem/ruby/2.4.4/gems/coque-0.7.1/lib/coque/runnable.rb:13:in `run!'

There's also a to_a! variant on commands which combines the error handling of run! with the array-slurping of stdout:

Coque['head', '-n 1', '/usr/share/dict/words'].to_a!
=> ["A"]

Coque['head', '-n 1', '/usr/share/dict/asdf'].to_a!
head: cannot open '/usr/share/dict/asdf' for reading: No such file or directory
RuntimeError: Coque Command Failed: <Coque::Sh head -n 1 /usr/share/dict/asdf>
from /code/coque/lib/coque/runnable.rb:11:in `to_a!'

The main piping and redirection methods also include named alternatives:

• | is aliased to pipe
• > is aliased to out
• >= is aliased to err
• < is aliased to in

So these 2 invocations are equivalent:

(Coque["echo", "hi"] | Coque["wc", "-c"] > STDERR).run!
# is the same as...
Coque["echo", "hi"].pipe(Coque["wc", "-c"]).out(STDERR).run!

You can set a logger for Coque, which will be used to output messages when commands are executed:

Coque.logger = Logger.new(STDOUT)
(Coque["echo", "hi"] | Coque["wc", "-c"]).run!

Will log:

I, [2019-02-20T20:31:00.325777 #16749]  INFO -- : Executing Coque Command: <Pipeline <Coque::Sh echo hi> | <Coque::Sh wc -c> >
I, [2019-02-20T20:31:00.325971 #16749]  INFO -- : Executing Coque Command: <Coque::Sh echo hi>
I, [2019-02-20T20:31:00.327719 #16749]  INFO -- : Coque Command: <Coque::Sh echo hi> finished in 0.001683 seconds.
I, [2019-02-20T20:31:00.327771 #16749]  INFO -- : Executing Coque Command: <Coque::Sh wc -c>
I, [2019-02-20T20:31:00.329586 #16749]  INFO -- : Coque Command: <Coque::Sh wc -c> finished in 0.001739 seconds.
I, [2019-02-20T20:31:00.329725 #16749]  INFO -- : Coque Command: <Pipeline <Coque::Sh echo hi> | <Coque::Sh wc -c> > finished in 0.003796 seconds.

Should be little overhead compared with the equivalent pipeline from a standard shell.

From zsh:

head -c 100000000 /dev/urandom | pv | wc -c
95.4MiB 0:00:06 [14.1MiB/s] [      <=>      ]
 100000000

With coque:

p = Coque["head", "-c", "100000000", "/dev/urandom"] | Coque["pv"] | Coque["wc", "-c"]
p.run.wait
95.4MiB 0:00:06 [14.6MiB/s] [           <=> ]

• Setup local environment with standard bundle
• Run tests with rake
• See code coverage output in coverage/
• Start a pry console with bin/console
• Install current dev version with rake install
• Use rake release to release after bumping lib/coque/version.rb
• New issues welcome

The concept and API for this library was heavily inspired by Python's excellent Plumbum library.

I relied on many resources to understand Ruby's great facilities for Process creation and manipulation. Some highlights include:

• Avdi Grimm's A dozen (or so) ways to start sub-processes in Ruby (Part 1, Part 2, Part 3)
• Ryan Tomayko's I like Unicorn because it's Unix
• Jesse Storimer's Working With Unix Processes
• Brandon Wamboldt's blog series: How bash redirection works, How Linux pipes work, and Understanding how Linux creates processes

The gem is available as open source under the terms of the MIT License.

Everyone interacting in the Coque project’s codebases, issue trackers, chat rooms and mailing lists is expected to follow the code of conduct.

gem build coque.gemspec
gem push coque-<VERSION>.gem
git tag <VERSION>
git push origin <VERSION>