Running Taverna 1 Workflows on NGS resources
Recently Research Computing Services demonstrated how one might set up a user friendly mechanism to enact Taverna workflows on NGS compute resources.
NGS resources are accessible and secured via internationally agreed grid protocols. In some cases, however, user communities require quick and simple access mechanisms which are at odds with the complexities of these middle grid layers. The pressure on service providers to keep up the secure interfaces is often conflicts with the requirements of the user communities to make access easy and we have to hide away unnecessary complexities in order to facilitate them. This exercise was one such demonstration.
Taverna is a Workflow Management System whose primary mode of use is via a Graphical User Interface (GUI) to an underlying workflow enaction engine. This allows users to express and manage their scientific process as a workflow. "Effectively, Taverna allows a scientist with limited computing background and limited technical resources and support to construct highly complex analyses over public and private data and computational resources, all from a standard PC, UNIX box or Apple computer"1. The GUI provides a user-friendly environment in which this work can take place but has limits in time, compute and data resources imposed by the computer on which the Taverna WMS is running.
To address these limits it is necessary to be able to enact long-running or CPU/data intensive tasks remotely and detached from the Taverna GUI. Fortunately as of version 1.7 of the Taverna software a command-line tool was provided allowing workflows to be enacted without the need for the GUI. This opens up the possibility to run workflows in a HPC/HTC batch scheduled environment, e.g. the NGS compute environment.
This demonstrator comprises of the following components:
- Taverna 1.7.2 installation on grid resources (currently limited to vidar.ngs.manchester.ac.uk)
- A means of securely starting the Taverna program on the grid resource
Use of Globus to submit jobs
- Availability of a grid credential
- A way to hide the Globus and grid credentials from the user
- A simple portal was created
Use of SARoNGS to obtain necessary credentials using the UK's Shibboleth Federation
- A way to obtain and handle the workflows
MyExperiment is a community repository for the storage and social sharing of such workflows. This demonstrator injects a "Run on the NGS" button for all Taverna 1 workflows it finds.
This is an unofficial modification to MyExperiment which is effected for the Firefox browser using GreaseMonkey2 and a script to modify MyExperiment pages when they load3.
Setting up to run
To run the demo:
- Install Firefox (if you don't already use Firefox).
Install the GreaseMonkey Add-on.
Install the taverna_myexperiment_ngs.user.js GreaseMonkey user script.
Visit MyExperiment.
Follow ons
This Taverna 1 demo was primarily targeted at the Taverna and MyExperiment projects to show how one might integrate with and utilise NGS resources. Currently this demo is:
available to run provided that the individual driving it (when logging into SARoNGS) asserts their "staff" role and university group.
- set up to run interactively, i.e. the batch aspects while relatively simple to achieve aren't implemented
not part of MyExperiment and so limited to the Firefox + GreaseMonkey combination.