Taverna Player¶ ↑

Authors

Robert Haines

Contact

support@mygrid.org.uk

URL

www.taverna.org.uk

Licence

BSD (See LICENCE or www.opensource.org/licenses/bsd-license.php)

Copyright

© 2013-2015 The University of Manchester, UK

<img src=“https://badge.fury.io/rb/taverna-player.svg” alt=“Gem Version” /> <img src=“https://codeclimate.com/github/myGrid/taverna-player/badges/gpa.svg” /> <img src=“https://travis-ci.org/myGrid/taverna-player.svg?branch=master” alt=“Build Status” /> <img src=“https://coveralls.io/repos/myGrid/taverna-player/badge.svg?branch=master” alt=“Coverage Status” />

Taverna Player is a Ruby on Rails plugin to run Taverna workflows on a Taverna Server.

Taverna Player handles all aspects of running a workflow including:

• Gathering inputs and upload to Taverna Server

• Monitoring the run

• Presenting workflow interactions to the user

• Gathering and download of workflow outputs

It, purposefully, does not manage users and its default workflow model is intentionally oversimplified.

Taverna Player requires access to a Taverna Server version 2.5 or later.

This library uses the new rubyzip API. If your code, or any of its dependencies, need the old API then please read the Important note in the rubyzip ReadMe for details of how to get everything working together.

These instructions assume that you are familiar with Rails and its concepts. Also, knowledge of Rails Engines and how they integrate with other Rails applications will be useful.

Taverna Player works with Rails 3.2. Add it to your Gemfile with:

gem "taverna-player"

And run the bundle install command to install it.

Next you need to run the install generator:

rails generate taverna_player:install

This installs two initializer files (into your application’s config/initializers directory) which describe all of Taverna Player’s configuration options; there are some things that MUST be configured before it will work - see below for more information. The install generator also creates a locale file at config/locales/taverna_player.en.yml and prints out a list of other steps for setting up Taverna Player; these are repeated and detailed below.

Once you have Taverna Player installed and configured in your Rails application it will largely take care of itself. The nature of Rails Engines is that they become part of the containing application relatively seamlessly.

The only extra step required for full operation of Taverna Player is to start the delayed_job script so that workflows are actually run. Even if you already use delayed_job in your application it is worth checking this section because Taverna Player uses its own named queue to run jobs.

If you simply want to have a single worker running while testing then you can use the rake task supplied by delayed_job:

rake jobs:work

In production you will want to run lots of workers for maximum throughput. A good starting number of concurrent workers will probably be the number of workflows that your Taverna Server can run at the same time. Any lower than this and you are not running Taverna Server at its configured capacity; any higher and you just end up with workers waiting for Taverna Server to have space for them anyway. You might pick up some efficiencies starting new workflow runs while the results of those just finished are being collected but it is probably not worth having more than one or two extra workers for this.

So if your Taverna Server can run 20 concurrent workflows at once then you would start your workers like this:

RAILS_ENV=production bundle exec ./script/delayed_job -n 20 --queue=player start

By default Taverna Player puts its jobs into the “player” queue which is why this is specified above. If you change this configuration (see below) you will need to specify a different queue in the command above.

See the documentation for delayed_job for more options and information.

Mount the Taverna Player engine in your config/routes.rb. For example:

mount TavernaPlayer::Engine, :at => "/"

Taverna Player provides four resources with paths:

• /runs/*

• /workflows/*

• /service_credentials/*

• /job_queue

Runs are what you will interact with most as this is the model that represents a workflow execution.

Service credentials and the job queue are administrator resources. If you have users you may like to restrict access to these resources (see the section on overriding controllers, below). These two resources can be moved to a different namespace if you like. In the Taverna Player initializer you can set (for example):

config.admin_scope = "admin"

And the resource paths would become:

• /runs/*

• /workflows/*

• /admin/service_credentials/*

• /admin/job_queue

The supplied workflow model also nests the run resources within it:

• /workflows

• /workflows/{workflow-id}/runs/*

If you override the default workflow model (see below for details) you can also nest the Taverna Player runs resources within your workflows resources if you wish, like this (within your application’s config/routes.rb):

resources :workflows do
  resources :runs, :controller => "TavernaPlayer::Runs", :except => :edit
end

The runs resources in Taverna Player do not provide an edit view by default so, unless you add it yourself by overriding the controller you should add the :except clause to the routes.

Perform Taverna Player’s migrations:

rake taverna_player:install:migrations
rake db:migrate

Make sure you have defined root_url to something in your config/routes.rb. For example:

root :to => "home#index"

Add Taverna Player’s assets to your application’s manifests.

In app/assets/javascripts/application.js:

//= require taverna_player/application

In app/assets/stylesheets/application.css

*= require taverna_player/application

And everything should be found by the asset pipeline automatically.

Make sure you have flash messages in your main layout (usually app/views/layouts/application.html.erb). For example:

<p id="notice"><%= notice %></p>
<p id="alert"><%= alert %></p>

Taverna Player uses delayed_job to run workflows on a Taverna Server. If your application is not already using delayed_job then you can install the delayed_job script in your script directory with:

rails generate taverna_player:job

Taverna Player comes with some very simple, unstyled views and layouts. If you wish to override these with your own customized views you can copy them into your application with:

rails generate taverna_player:views

The views are copied to the app/views/taverna_player directory so that they take precedence over the default ones. You can delete any that you do not need to customize but there are no penalties for leaving them there. There is more information on overriding views below.

If you need to override the Taverna Player controllers, to implement user authorization for example, you can copy some customizable stubs with:

rails generate taverna_player:controllers

The stubs are copied to the app/controllers/taverna_player directory so that they take precedence over the default ones. You can delete any that you do not need to customize but there are no penalties for leaving them there. There is more information on overriding controllers below.

If you need to override any Taverna Player models, to add columns to a table for example, you can copy customizable stubs with:

rails generate taverna_player:models

The stubs are copied to the app/models/taverna_player directory so that they take precedence over the defaults. There is more information on overriding models below.

If you want to use pre- and post-run callbacks you can setup some basic stubs with:

rails generate taverna_player:callbacks

They will be saved to lib/taverna_player_callbacks.rb. Don’t forget to then require and register them in the Taverna Player initializer. There is more information on callbacks below.

You can add to, or change, the workflow port render methods to better suit your particular application. To copy the defaults that Taverna Player ships with into your application for customization run:

rails generate taverna_player:renderers

They will be saved to lib/taverna_player_renderers.rb. Don’t forget to then require and register them in the Taverna Player initializer. There is more information on renderers below.

Two initializers are installed by the install generator:

• config/initializers/taverna_player.rb: This contains configuration of Taverna Player.

• config/initializers/taverna_server.rb.example: This is used to configure Taverna Player’s connection to a Taverna Server. It is initially set up as an example file as you will need to distribute this with your application but you must not check in the contents of the configured version to your repository.

Both of these files require minimal configuration for simple set ups and are fully commented with everything that needs to be set - more details below.

Firstly, Taverna Player needs to know what the model is that represents workflows within your application and it needs to know how to get a workflow file out of that model so it can run it. If your workflow model is called “Workflow” and the method to get the workflow filename from that model is called “file_name” then the following will configure this correctly:

config.workflow_model_proxy("Workflow") do |proxy|
  proxy.file_method_name = :file_name
end

Taverna Player has a very simple internal workflow model (TavernaPlayer::Workflow) that you can extend (see below) or replace with the above code.

Secondly, Taverna Player needs to know where your Taverna Server is and how to authenticate to it:

config.server_address = "http://localhost:8080/taverna"
config.server_username = "taverna"
config.server_password = "taverna"

Make sure you do not commit this information into your repository!

This should usually be enough to get Taverna Player up and running within your application but there are lots of other configuration options detailed below.

There are two settings to control how often Taverna Player communicates with Taverna Server:

config.server_poll_interval = 5
config.server_retry_interval = 10

The first specifies, in seconds, how often each run is polled. Polling is used to check a run’s status (has it finished yet?) and check for interactions. If you have long running workflows then it is probably worth setting this value higher; If you have lots of interactions then keeping it low is good to improve the responsiveness for users. Keep in mind that as polling is for each run then setting this value very low will produce a lot of polling requests when you have lots of running workflows.

The second specifies, in seconds, how long to wait before retrying an operation that Taverna Server has rejected due to its current load. This can happen in two places:

• Creating the run. Each Taverna Server has a limit (usually quite high) to how many workflow runs it can support at a time - this is the entire set of runs resident on the server in any state (initialized, running or finished). If it is at this limit then it will refuse requests to create any more.

• Starting the run. Each Taverna Server also has a limit to how many workflow runs it can have actually running at once - runs that are yet to start, or that have finished do not count towards this total. If it is at this limit then it will refuse requests to start any more.

Neither of these situations are fatal errors however, so Taverna Player will back off for the specified time and then try again. It is worth tailoring this number to the sort of workflows you will be running. For long-running, batch-style workflows then it can be set quite high but if you have workflows that make use of interactions (so you have users watching the workflows running) then it should probably be set lower.

There are a number of options for configuring the connection to the Taverna Server. The first specifies how many times an initial connection to Taverna Server will be retried if there are any low level network errors.

config.server_connection_error_retries = 5

These can be events such as timeouts, broken connections, refused connections or anything else out of our control. If such an error is detected Taverna Player will back off for the amount of time specified by config.server_retry_interval before retrying. If the maximum number of retries is reached and there is still an error then the run will fail; the network error message will be recorded as the reason for the run’s failure.

The rest of the connection options are actually provided by the underlying t2-server client library and surfaced here for extra control. They are documented in more detail elsewhere but broadly fall into two categories: Security and timeouts.

The following are all concerned with connecting to a secure Taverna Server and will be set to your Ruby/OS defaults if you leave them alone. If you use a self-signed certificate for your server, or you require client certificate authentication you will need to edit them appropriately.

config.server_connection[:verify_peer] = true
config.server_connection[:ca_file] = "/etc/certs/my-cert.crt"
config.server_connection[:ca_path] = "/etc/certs/"
config.server_connection[:client_certificate] = "client-cert.crt"
config.server_connection[:client_password] = "P@5$w0Rd!"
config.server_connection[:ssl_version] = :TLSv1

The following are timeout options and should be set with care. The underlying operating system defaults (usually 300 seconds each) will be used if they are not set explicitly.

config.server_connection[:open_timeout] = 300
config.server_connection[:read_timeout] = 300

If your application has users then there is some basic support for that out-of-the-box in Taverna Player. You can tell Taverna Player what your user model is called with (example “User” here):

config.user_model_proxy = "User"

And you can tell it how to discover who your current user is by specifying a callback. If you are using Devise (recommended) for your authentication then you would supply the provided “current_user” method like this:

config.current_user_callback = :current_user

With this set up then when a run is created Taverna Player will set the owner of that run to be the current logged in user if there is one.

Note that this does not automatically mean that Taverna Player is checking that users are authenticated! If you require this or if you have more complex requirements then you will need to override the Run model and the Runs controller. See below for more details.

Taverna Player needs to store various files for each run’s inputs and outputs. The paperclip gem is used to provide these facilities and the only configuration required is to specify where you want it to store its files:

config.file_store = ":rails_root/public/system"

Use “:rails_root” for the root of your application, or specify a full path for anywhere else.

Taverna Player uses the “player” queue for its delayed jobs. You can change this with:

config.job_queue_name = "my_queue"

Remember to start workers listening to the correct queue if you change it.

These are described in their own sections, below.

If you use the generator, described above, to copy the views into your application they will be used in preference to the default ones. This means that any changes you make will be immediately reflected in the output of your application. See the Rails documentation for more information on overriding views.

The supplied views provide a good example of how to maintain a current view of the state of a run and handle any workflow interactions that may occur during a run. It is worth understanding how they work before pulling them apart for your own needs.

You can override the following core components:

• Run (model)

• RunPort (model)

• RunPort::Input (model)

• RunPort::Output (model)

• Workflow (model)

• RunsController

• WorkflowController

• ServiceCredentialsController

• JobQueueController

Use the generators, detailed above, to create stubs for you to expand. These components have been designed for overriding with the decorator pattern using ActiveSupport::Concern. This allows for things to be overridden but still have the same name (as is required for such things within Rails). Please also see the information about overriding models and controllers in the Rails documentation.

The vitally important thing is to preserve the include statement within your overriding code. For example, the RunsController stub looks like this:

module TavernaPlayer
  class RunsController < TavernaPlayer::ApplicationController
    # Do not remove the next line.
    include TavernaPlayer::Concerns::Controllers::RunsController

    # Extend the RunsController here.
  end
end

You can add code both before and after the include statement but anything before it might be overridden by the original code itself if there are name clashes.

If you wanted to add a before filter to authenticate your users you would add that line before the include statement, for example.

Important note! If you override the RunPort model then you must override the RunPort::Input and RunPort::Output models too (even if you just leave the generated stubs unedited). This is because they are subclasses of RunPort and the inheritances must be re-established with the overridden model.

Taverna Player provides four points around a workflow run for you to specify callbacks to be run:

• Pre-run: This is called directly before Taverna Server is contacted.

• Post-run: This is called after all operations on Taverna Server have been completed when the run finishes normally.

• Run-cancelled: This is called after all operations on Taverna Server have been completed when the run has been cancelled by the user.

• Run-failed: This is called after all operations on Taverna Server have been completed when the run has failed.

Each of these callbacks is provided with the current run object as its parameter and can be set in the initializer by providing the name of the method (string or symbol) to run or a Proc object:

config.pre_run_callback = :player_pre_run_callback
config.post_run_callback = "player_post_run_callback"
config.run_cancelled_callback = Proc.new { |run| puts "Cancelled: #{run.name}" }
config.run_failed_callback = :player_run_failed_callback

Important! If your callback fails then the run itself will “fail”. This may or may not matter for the run-failed callback but if your pre-run callback fails then the run will never get to Taverna Server! How can it? Your pre-run callback may have been setting up vital resources for your run; Taverna Player can not second-guess this so “fails” the run.

For this reason it is not recommended to put a lot of complex functionality into the callbacks. An ideal use would be to gather statistics from the run (average time, how many times a user runs it, etc) once it has finished.

An example callback that just prints out the run’s name and workflow id would be:

def player_run_callback(run)
  w = TavernaPlayer::Workflow.find(run.workflow_id)
  puts "Callback called for run '#{run.name}' of workflow '#{w.id}'"
end

A set of example callbacks can be installed with the generator detailed above. Don’t forget to make sure any callback code is required somewhere and the callbacks themselves registered in the initializer.

Workflows can accept inputs and produce results in many different formats and Taverna Player tries to accomodate this as best it can. It provides basic facilities for rendering as many types as it can and these are extensible wherever possible.

Calling the port renderer is as simple as just passing it the port to be rendered in your view.

<% run.outputs.each do |output| %>
  <%= TavernaPlayer.port_renderer.render(output) %>
<% end %>

Taverna Player has a system of specific type renderers to handle different types of value. A number of defaults are supplied but these can be replaced and added to if required.

To install a set of example renderers you can use the generator detailed above.

To register a renderer for use add it into the renderers block in the initializer:

config.port_renderers do |renderers|
  ...
end

So to just register a single default renderer method (called “my_default_renderer”) you would do this:

config.port_renderers do |renderers|
  renderers.default(:my_default_renderer)
end

And it would be used to render every type of value. A more sensible example would be to have a renderer for PNG-type images and a renderer for text values as well:

config.port_renderers do |renderers|
  renderers.default(:my_default_renderer)
  renderers.add("text/plain", :text_renderer, true)
  renderers.add("image/png", :show_image)
end

This does three things:

• Registers a renderer for PNG images. This could be as simple as wrapping it in an <img ../> tag.

• Registers a renderer for values of type “text/plain” and sets this as the default renderer for all other types beginning with “text”. That is what the final parameter set to true does.

• Registers a default renderer for all other types. This should probably give an explanation as to why the value cannot be shown in the browser with a link to download the value to the user’s computer.

Note the use of MIME types for specifying all types.

Obviously values such as images and text are so common that Taverna Player provides these renderers for you and has them set up and registered by default. You would only need to override them if you wanted extra information to be displayed as well, such as sizes next to images, etc.

Note that it is not a good idea to register a single image renderer for all “image” types as many cannot be shown in most browsers, e.g. TIFF images. The default set of registered renderers is shown in the default initializer but, as an example, here is how it handles the images that most browsers can show:

renderers.add("image/jpeg", :show_image)
renderers.add("image/png", :show_image)
renderers.add("image/gif", :show_image)
renderers.add("image/bmp", :show_image)

Note that the same renderer callback is used for each one.

Taverna workflow inputs and output can be lists and rendering them requires a renderer too. The renderer you wish to use for displaying lists is set with the list method:

renderers.list(:render_list)

The default list renderer recurses into the list and calls the correct renderer for each leaf item when it gets to them.

To be a renderer callback a method must accept two parameters (in this order):

1. The port to be rendered.

2. A list of indices into the port. For a singleton port this will be an empty list. For a port of depth 2 this would be a list with two items, e.g. [0, 0]

All renderer callbacks are called by Taverna Player in a context that includes the ActionView::Helpers so your callbacks have access to them too, including helpers from third-party gems that register their helpers correctly.

These are some of the supplied renderers that are registered by default in Taverna Player.

Taverna Player provides a plain text renderer that formats text with a monospaced font, converts URI-like things to clickable links and respects carriage returns and newlines. It looks something like this:

def format_text(port, index = [])
  # Use CodeRay to format text so that newlines are respected.
  content = CodeRay.scan(port.value(index), :text).div(:css => :class)

  # Use auto_link to turn URI-like text into links.
  auto_link(content, :html => { :target => '_blank' }, :sanitize => false)
end

The coderay gem is used to format the text, preserving newlines and the rails_autolink gem is used to convert URI-like text into clickable links.

This renderer is registered as the default for all “text” media types.

This renderer catches “text/XML” outputs:

def format_xml(port, index = [])
  # Make sure XML is indented consistently.
  out = String.new
  REXML::Document.new(port.value(index)).write(out, 1)
  CodeRay.scan(out, :xml).div(:css => :class, :line_numbers => :table)
end

It uses REXML to format the XML and coderay to syntax-highlight it.

Note that for XML to be detected as XML it must, as per the standard, include the XML declaration, e.g.:

<?xml version="1.0" ?>

As described above, images are just dropped into an <img ../> tag:

def show_image(port, index = [])
  # Can't use image_tag() here because the image doesn't really exist (it's
  # in a zip file, really) and this confuses the Rails asset pipeline.
  tag("img", :src => port.path(index))
end

Note the comment about the Rails asset pipeline in there if you are writing your own image renderer and are using the asset pipeline.

Unless you can be absolutely sure that the workflows that will be run within your installation of Taverna Player will only have lists of a certain depth the lists renderer will need to be able to cope with anything that is thrown at it. The supplied renderer uses recursion to cope with what could, at least in theory, be infinitely deep lists:

def list_port(port, index = [], types = nil)
  types = port.metadata[:type] if types.nil?

  content = "<ol>"
  i = 0
  types.each do |type|
    if type.is_a?(Array)
      content += "<li><br />" +
      list_port(port, index + [i], type) + "</li>"
    else
      content += "<li>(#{type})<p>" +
        TavernaPlayer.port_renderer.render(port, index + [i]) +
        "</p></li>"
    end
    i += 1
  end

  content += "</ol>"
end

This method has an extra parameter that is used to drive the recursion. The types parameter contains the list structure of the whole port so can be used to loop over, or recurse into, each level as required.

Lists are simply rendered as a numbered list along with their type information. Other registered renderers are called as necessary to render individual values.

The default renderer for other, or unknown types, is:

def cannot_inline(port, index = [])
  "Sorry but we cannot show this type of content in the browser. Please " +
    link_to("download it", port.path(index)) + " to view it on your " +
    "local machine."
end

Please see the notes in the limitations section below!

At the moment the Service Credentials facilities are basic and provided purely to allow access to services which are ostensibly public but still require a login of some kind.

In practice very few services that will be used via public tools such as Taverna Player are in use so this should not affect most users at this time. It is the intention to fill this gap in functionality as soon as possible, however.

If you do have services in your workflows that require such a login then you can add them through the Service Credentials model. The types of credential that Taverna Server can accept are detailed in the Client Library Documentation but Taverna Player supports a subset:

REST services are commonly secured via HTTP Basic or HTTP Digest authentication and Taverna treats these two schemes in the same way. Simply register the username and password with the host name of the server on which the service is running:

https://example.com:8443/

The above example shows a https server running on port 8443. If the service is on port 80 for http or port 443 for https then you don’t need to specify the port.

If there are services on the same host that require different credentials then you will need to specify the realm for which each set of credentials applies. This is done by adding the name of the realm to the end of the host name with a # separating them:

https://example.com:8443/#realm

SOAP services are commonly secured via WS-Security. Simply register the WSDL address of the service with your username and password:

https://example.com:8443/services/MyService?wsdl

You can authenticate to R Servers in almost exactly the same way as for REST services - only the protocol scheme is different. So instead of http or https it is rserve:

rserve://example.com:6311

There is more information about how you can more closely integrate Taverna Player into your host application in the API Documentation.

Workflow runs handled by Taverna Player can be embedded in other Web sites in much the same way that media like YouTube videos can be. Documentation on how to achieve this {is available separately.

As well as providing HTML views to the host application, Taverna Player also provides a RESTful Web Service interface. Documentation for this interface is available separately.

The most serious limitation is with the Service Credentials. Because Taverna Server needs to be given the credentials in plaintext (which is why you should always use HTTPS) we have to store them as such. For the time being it is recommended to only use them for services that are ostensibly public, but with a login requirement. An example of this might be a R server on the local machine, which is only available locally, but nevertheless requires a login.

Service Credentials are also intended, at the moment, to be a global resource. They are not per-user. Every workflow run will be given these credentials.

Service Credentials only provide username/password type credentials at the moment.

In no particular order:

• User specific (and therefore private) credentials wallet for secure services.

• Comprehensive Taverna Server administration panel. This would allow admin users to view and manage runs directly on the Taverna Server along with other such admin tasks.

• oEmbed support (see oembed.com/ for more details).

Please email support@mygrid.org.uk for any questions relating to Taverna Player.

Bug reports or feature requests may be submitted to the public issue tracker at GitHub.