Arvados CWL Runner

The Arvados FUSE driver is a Python utility that allows you to see the Keep service as a normal filesystem, so that data can be accessed using standard tools. This driver requires the Python SDK installed in order to access Arvados services.

Installation

If you are logged in to a managed Arvados VM, the arv-mount utility should already be installed.

To use the FUSE driver elsewhere, you can install from a distribution package, or PyPI.

Option 1: Install from distribution packages

First, add the appropriate package repository for your distribution

Install python3-arvados-cwl-runner

Red Hat and Centos

# yum install python3-arvados-cwl-runner

Debian and Ubuntu

# apt-get install python3-arvados-cwl-runner

Option 2: Install with pip

Run pip install arvados-cwl-runner in an appropriate installation environment, such as a virtualenv.

Note:

The SDK uses pycurl which depends on the libcurl C library. To build the module you may have to first install additional packages. On Debian 10 this is:

$ apt-get install git build-essential python3-dev libcurl4-openssl-dev libssl1.0-dev python3-llfuse

Check Docker access

In order to pull and upload Docker images, arvados-cwl-runner requires access to Docker. You do not need Docker if the Docker images you intend to use are already available in Arvados.

You can determine if you have access to Docker by running docker version:

~$ docker version
Client:
 Version:      1.9.1
 API version:  1.21
 Go version:   go1.4.2
 Git commit:   a34a1d5
 Built:        Fri Nov 20 12:59:02 UTC 2015
 OS/Arch:      linux/amd64

Server:
 Version:      1.9.1
 API version:  1.21
 Go version:   go1.4.2
 Git commit:   a34a1d5
 Built:        Fri Nov 20 12:59:02 UTC 2015
 OS/Arch:      linux/amd64

If this returns an error, contact the sysadmin of your cluster for assistance.

Usage

Please refer to the Accessing Keep from GNU/Linux tutorial for more information.

Previous: Arvados FUSE driver Next: Code cookbook