The Python SDK provides access from Python to the Arvados API and Keep, along with a number of command line tools for using and administering Arvados and Keep.
If you are logged in to an Arvados VM, the Python SDK should be installed.
To use the Python SDK elsewhere, you can install from PyPI or a distribution package.
As of Arvados 2.2, the Python SDK requires Python 3.6+. The last version to support Python 2.7 is Arvados 2.0.4.
This installation method is recommended to make the CLI tools available system-wide. It can coexist with the installation method described in option 2, below.
First, configure the Arvados package repositories
# yum install python3-arvados-python-client
# apt-get install python3-arvados-python-client
This installation method is recommended to use the SDK in your own Python programs. If installed into a virtualenv
, it can coexist with the system-wide installation method from a distribution package.
Run pip install arvados-python-client
in an appropriate installation environment, such as a virtualenv
.
Note:
The SDK uses pycurl
which depends on the libcurl
C library. To build the module you may have to first install additional packages. On Debian 10 this is:
$ apt-get install git build-essential python3-dev libcurl4-openssl-dev libssl-dev
If your version of pip
is 1.4 or newer, the pip install
command might give an error: “Could not find a version that satisfies the requirement arvados-python-client”. If this happens, try pip install --pre arvados-python-client
.
If the SDK is installed and your ARVADOS_API_HOST
and ARVADOS_API_TOKEN
environment variables are set up correctly (see api-tokens for details), import arvados
should produce no errors.
If you installed with pip (option 1, above):
~$python
Python 3.7.3 (default, Jul 25 2020, 13:03:44) [GCC 8.3.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>>import arvados
>>>arvados.api('v1')
<apiclient.discovery.Resource object at 0x233bb50>
If you installed from a distribution package (option 2): the package includes a virtualenv, which means the correct Python environment needs to be loaded before the Arvados SDK can be imported. This can be done by activating the virtualenv first:
If you are on Ubuntu 18.04, please note that the Arvados packages that use Python depend on the python-3.8 package. This means they are installed under /usr/share/python3.8
, not /usr/share/python3
. You will need to update the commands below accordingly.
~$source /usr/share/python3/dist/python3-arvados-python-client/bin/activate
(python-arvados-python-client) ~$python
Python 3.7.3 (default, Jul 25 2020, 13:03:44) [GCC 8.3.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>>import arvados
>>>arvados.api('v1')
<apiclient.discovery.Resource object at 0x233bb50>
Or alternatively, by using the Python executable from the virtualenv directly:
~$/usr/share/python3/dist/python3-arvados-python-client/bin/python
Python 3.7.3 (default, Jul 25 2020, 13:03:44) [GCC 8.3.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>>import arvados
>>>arvados.api('v1')
<apiclient.discovery.Resource object at 0x233bb50>
Check out the examples and cookbook
The general form of an API call is:
arvados.api(api_version).plural_resource_type().api_method(parameter=value, ...).execute()
Many API methods accept a parameter whose name is the same as the resource type. For example, links.create
accepts a parameter called link
. This parameter should be given as body
.
arvados.api('v1').links().create(
uuid=test_link['uuid'],
body={'properties':{'foo':'bar'}}).execute()
One way to make API calls slightly less verbose is:
arv = arvados.api('v1')
j = arv.jobs().list().execute()
The SDK retrieves the list of API methods from the server at run time. Therefore, the set of available methods is determined by the server version rather than the SDK version.
The content of this documentation is licensed under the
Creative
Commons Attribution-Share Alike 3.0 United States licence.
Code samples in this documentation are licensed under the
Apache License, Version 2.0.