Arvados CWL Runner

The Arvados CWL Runner is a Python tool that allows you to register and submit workflows to Arvados. You can oversee a running workflow on your local system, or let that run inside an Arvados container. This tool requires the Python SDK installed in order to access Arvados services.

Installation

If you are logged in to a managed Arvados VM, the arvados-cwl-runner utility should already be installed.

To use the CWL Runner elsewhere, you can install it from a distribution package or PyPI.

Option 1: Install from distribution packages

First, add the appropriate package repository for your distribution.

Install python3-arvados-cwl-runner

Red Hat, AlmaLinux, and Rocky Linux

# dnf install python3-arvados-cwl-runner

Debian and Ubuntu

# apt install python3-arvados-cwl-runner

Option 2: Install with pip

Run pip install arvados-cwl-runner in an appropriate installation environment, such as a virtualenv.

Note:

The CWL Runner uses pycurl which depends on the libcurl C library. To build the module you may have to first install additional packages. On Debian-based distributions you can install them by running:

# apt install git build-essential python3-dev libcurl4-openssl-dev libssl-dev

Check Docker access

In order to pull and upload Docker images, arvados-cwl-runner requires access to Docker. You do not need Docker if the Docker images you intend to use are already available in Arvados.

You can determine if you have access to Docker by running docker version:

~$ docker version
Client:
 Version:      1.9.1
 API version:  1.21
 Go version:   go1.4.2
 Git commit:   a34a1d5
 Built:        Fri Nov 20 12:59:02 UTC 2015
 OS/Arch:      linux/amd64

Server:
 Version:      1.9.1
 API version:  1.21
 Go version:   go1.4.2
 Git commit:   a34a1d5
 Built:        Fri Nov 20 12:59:02 UTC 2015
 OS/Arch:      linux/amd64

If this returns an error, contact the sysadmin of your cluster for assistance.

Usage

Please refer to the Starting a Workflow at the Command Line tutorial for more information.

Previous: PyDoc Reference Next: Subscribing to database events