Floe

Floe is a runner for Amazon States Language workflows with support for Docker resources and running on Docker, Podman, or Kubernetes.

Install the gem and add to the application's Gemfile by executing:

$ bundle add floe

If bundler is not being used to manage dependencies, install the gem by executing:

$ gem install floe

Floe can be run as a command-line utility or as a ruby class.

bundle exec ruby exe/floe --workflow examples/workflow.asl --inputs='{"foo": 1}'

By default Floe will use docker to run docker:// type resources, but podman and kubernetes are also supported runners. A different runner can be specified with the --docker-runner option:

bundle exec ruby exe/floe --workflow examples/workflow.asl --inputs='{"foo": 1}' --docker-runner podman
bundle exec ruby exe/floe --workflow examples/workflow.asl --inputs='{"foo": 1}' --docker-runner kubernetes --docker-runner-options namespace=default server=https://k8s.example.com:6443 token=my-token

If your workflow has Credentials you can provide a payload that will help resolve those credentials references at runtime.

For example if your workflow had the following Credentials field with a JSON Path property:

"Credentials": {
  "RoleArn.$": "$.roleArn"
}

You can provide that at runtime via the --credentials parameter:

bundle exec ruby exe/floe --workflow my-workflow.asl --credentials='{"roleArn": "arn:aws:iam::111122223333:role/LambdaRole"}'

Or if you are running the floe command programmatically you can securely provide the credentials via a stdin pipe via --credentials=-:

echo '{"roleArn": "arn:aws:iam::111122223333:role/LambdaRole"}' | bundle exec ruby exe/floe --workflow my-workflow.asl --credentials -

Or you can pass a file path with the --credentials-file parameter:

bundle exec ruby exe/floe --workflow my-workflow.asl --credentials-file /tmp/20231218-80537-kj494t

If you need to set a credential at runtime you can do that by using the "ResultPath": "$.Credentials" directive, for example to user a username/password to login and get a Bearer token:

bundle exec ruby exe/floe --workflow my-workflow.asl --credentials='{"username": "user", "password": "pass"}'

{
  "StartAt": "Login",
  "States": {
    "Login": {
      "Type": "Task",
      "Resource": "docker://login:latest",
      "Credentials": {
        "username.$": "$.username",
        "password.$": "$.password"
      },
      "ResultPath": "$.Credentials",
      "Next": "DoSomething"
    },
    "DoSomething": {
      "Type": "Task",
      "Resource": "docker://do-something:latest",
      "Credentials": {
        "token.$": "$.bearer_token"
      },
      "End": true
    }
  }
}

require 'floe'

workflow = Floe::Workflow.load("workflow.asl")
workflow.run!

You can also specify a specific docker runner and runner options:

require 'floe'

Floe::Workflow::Runner.docker_runner = Floe::Workflow::Runner::Podman.new
# Or
Floe::Workflow::Runner.docker_runner = Floe::Workflow::Runner::Kubernetes.new("namespace" => "default", "server" => "https://k8s.example.com:6443", "token" => "my-token")

workflow = Floe::Workflow.load("workflow.asl")
workflow.run!

It is also possible to step through a workflow without blocking, and any state which would block will return Errno::EAGAIN.

require 'floe'

workflow = Floe::Workflow.load("workflow.asl")

# Step through the workflow while it would not block
workflow.run_nonblock

# Go off and do some other task

# Continue stepping until the workflow is finished
workflow.run_nonblock

You can also use the Floe::Workflow.wait class method to wait on multiple workflows and return all that are ready to be stepped through.

require 'floe'

workflow1 = Floe::Workflow.load("workflow1.asl")
workflow2 = Floe::Workflow.load("workflow2.asl")

running_workflows = [workflow1, workflow2]
until running_workflows.empty?
  # Wait for any of the running workflows to be ready (up to the timeout)
  ready_workflows = Floe::Workflow.wait(running_workflows)
  # Step through the ready workflows until they would block
  ready_workflows.each do |workflow|
    workflow.run_nonblock
  end
  # Remove any finished workflows from the list of running_workflows
  running_workflows.reject!(&:end?)
end

Options supported by the Docker docker runner are:

• network - What docker to connect the container to, defaults to "bridge". If you need access to host resources for development you can pass network=host.
• pull-policy - Pull image policy. The default is missing. Allowed values: always, missing, never

Options supported by the podman docker runner are:

• identity=string - path to SSH identity file, (CONTAINER_SSHKEY)
• log-level=string - Log messages above specified level (trace, debug, info, warn, warning, error, fatal, panic)
• network=string - What docker to connect the container to, defaults to "bridge". If you need access to host resources for development you can pass network=host.
• noout=boolean - do not output to stdout
• pull-policy=string - Pull image policy. The default is missing. Allowed values: always, missing, never, newer
• root=string - Path to the root directory in which data, including images, is stored
• runroot=string - Path to the 'run directory' where all state information is stored
• runtime=string - Path to the OCI-compatible binary used to run containers
• runtime-flag=stringArray - add global flags for the container runtime
• storage-driver=string - Select which storage driver is used to manage storage of images and containers
• storage-opt=stringArray - Used to pass an option to the storage driver
• syslog=boolean - Output logging information to syslog as well as the console
• tmpdir=string - Path to the tmp directory for libpod state content
• transient-store=boolean - Enable transient container storage
• volumepath=string - Path to the volume directory in which volume data is stored

Options supported by the kubernetes docker runner are:

• kubeconfig - Path to a kubeconfig file, defaults to KUBECONFIG environment variable or ~/.kube/config
• kubeconfig_context - Context to use in the kubeconfig file, defaults to "default"
• namespace - Namespace to use when creating kubernetes resources, defaults to "default"
• pull-policy - Pull image policy. The default is Always. Allowed values: IfNotPresent, Always, Never
• server - A kubernetes API Server URL, overrides anything in your kubeconfig file. If set KUBERNETES_SERVICE_HOST and KUBERNETES_SERVICE_PORT will be used
• token - A bearer_token to use to authenticate to the kubernetes API, overrides anything in your kubeconfig file. If present, /run/secrets/kubernetes.io/serviceaccount/token will be used
• ca_file - Path to a certificate-authority file for the kubernetes API, only valid if server and token are passed. If present /run/secrets/kubernetes.io/serviceaccount/ca.crt will be used
• verify_ssl - Controls if the kubernetes API certificate-authority should be verified, defaults to "true", only vaild if server and token are passed

The following are not yet supported:

• Map State Fields:
  • ItemReader
  • ItemSelector/ItemBatcher
  • ResultWriter

After checking out the repo, run bin/setup to install dependencies. You can also run bin/console for an interactive prompt that will allow you to experiment.

To install this gem onto your local machine, run bundle exec rake install. To release a new version, update the version number in version.rb, and then run bundle exec rake release, which will create a git tag for the version, push git commits and the created tag, and push the .gem file to rubygems.org.

Bug reports and pull requests are welcome on GitHub at https://github.com/ManageIQ/floe.

The gem is available as open source under the terms of the Apache License 2.0.