Code cookbook

Introduction
Working with the current user
1. Fetch the current user
2. List objects shared with the current user
Working with projects
1. Create a project
2. List the contents of a project
Working with permissions
Working with properties
Working with collections
Working with containers and workflow runs
Working with the container request queue

Introduction

This page provides example code to perform various high-level tasks using Arvados’ Python SDK. This page assumes you’ve already read the API client documentation and understand the basics of using the Python SDK client. You don’t have to have the details of every API method memorized, but you should at least be comfortable with the pattern of calling a resource type, API method, and execute(), as well as the dictionaries these methods return.

The code examples assume you’ve built the arv_client object by doing something like:

import arvados
arv_client = arvados.api('v1', ...)

These examples work no matter how you call arvados.api(), or if you use another constructor from arvados.api module. Just understand that arv_client represents your client object, no matter how you built it.

Whenever you see the Ellipsis object ... in these examples, that means you may need or want to fill something in. That might be list items, function arguments, or your own code. Comments will provide additional guidance.

Whenever you see the example UUID zzzzz-zzzzz-12345abcde67890, you should provide your own UUID from input.

Working with the current user

Fetch the current user

The API provides a dedicated users method named current. It returns the user object that is authenticated by your current API token. Use this method to get the current user’s UUID to use in other API calls, or include user details like name in your output.

current_user = arv_client.users().current().execute()

List objects shared with the current user

The API provides a dedicated groups method named shared to do this. Call it like you would any other list method. This example illustrates some popular arguments. Check the API reference for full details of all possible arguments.

for item in arvados.util.keyset_list_all(
    # Pass the method keyset_list_all will call to retrieve items.
    # Do not call it yourself.
    arv_client.groups().shared,
    # Pass filters to limit what objects are returned.
    # This example returns only subprojects.
    filters=[
        ['uuid', 'is_a', 'arvados#group'],
        ['group_class', '=', 'project'],
    ],
    # Pass order_key and ascending to control how the contents are sorted.
    # This example lists projects in ascending creation time (the default).
    order_key='created_at',
    ascending=True,
):
    ...  # Work on item as desired

Working with projects

Create a project

A project is represented in the Arvados API as a group with its group_class field set to "project".

new_project = arv_client.groups().create(
    body={
        'group': {
            'group_class': 'project',
            'name': 'Python SDK Test Project',
            # owner_uuid can be the UUID for an Arvados user or group.
            # Specify the UUID of an existing project to make a subproject.
            # If not specified, the current user is the default owner.
            'owner_uuid': 'zzzzz-j7d0g-12345abcde67890',
        },
    },
    ensure_unique_name=True,
).execute()

List the contents of a project

The API provides a dedicated groups method named contents to do this. Call it like you would any other list method. This example illustrates some popular arguments. Check the API reference for full details of all possible arguments.

current_user = arv_client.users().current().execute()
for item in arvados.util.keyset_list_all(
    # Pass the method keyset_list_all will call to retrieve items.
    # Do not call it yourself.
    arv_client.groups().contents,
    # The UUID of the project whose contents we're listing.
    # Pass a user UUID to list their home project.
    # This example lists the current user's home project.
    uuid=current_user['uuid'],
    # Pass filters to limit what objects are returned.
    # This example returns only subprojects.
    filters=[
        ['uuid', 'is_a', 'arvados#group'],
        ['group_class', '=', 'project'],
    ],
    # Pass recursive=True to include results from subprojects in the listing.
    recursive=False,
    # Pass include_trash=True to include objects in the listing whose
    # trashed_at time is passed.
    include_trash=False,
):
    ...  # Work on item as desired

Working with permissions

In brief, a permission is represented in Arvados as a link object with the following values:

link_class is "permission".
name is one of "can_read", "can_write", "can_manage", or "can_login".
tail_uuid identifies the user or role group that receives the permission.
head_uuid identifies the Arvados object this permission grants access to.

For details, refer to the Permissions model documentation. Managing permissions is just a matter of ensuring the desired links exist using the standard create, update, and delete methods.

Grant permission to an object

Create a link with values as documented above.

permission = arv_client.links().create(
    body={
        'link': {
            'link_class': 'permission',
            # Adjust name for the level of permission you want to grant
            'name': 'can_read',
            # tail_uuid must identify a user or role group
            'tail_uuid': 'zzzzz-tpzed-12345abcde67890',
            # head_uuid can identify any Arvados object
            'head_uuid': 'zzzzz-4zz18-12345abcde67890',
        },
    },
).execute()

Modify permission on an object

To modify an existing permission—for example, to change its access level—find the existing link object for the permission, then update it with the new values you want. This example shows changing all read-write permissions on a specific collection to read-only. Adjust the filters appropriately to find the permission(s) you want to modify.

import arvados.util
for permission in arvados.util.keyset_list_all(
    # Pass the method keyset_list_all will call to retrieve items.
    # Do not call it yourself.
    arv_client.links().list,
    filters=[
        # You should use this filter for all permission searches,
        # to exclude other kinds of links.
        ['link_class', '=', 'permission'],
        # Add other filters as desired.
        ['name', '=', 'can_write'],
        ['head_uuid', '=', 'zzzzz-4zz18-12345abcde67890'],
        ...,
    ],
):
    arv_client.links().update(
        uuid=permission['uuid'],
        body={
            'link': {
                'name': 'can_read',
            },
       },
    ).execute()

Revoke permission from an object

To revoke an existing permission, find the existing link object for the permission, then delete it. This example shows revoking one user’s permission to log into any virtual machines. Adjust the filters appropriately to find the permission(s) you want to revoke.

import arvados.util
for permission in arvados.util.keyset_list_all(
    # Pass the method keyset_list_all will call to retrieve items.
    # Do not call it yourself.
    arv_client.links().list,
    filters=[
        # You should use this filter for all permission searches,
        # to exclude other kinds of links.
        ['link_class', '=', 'permission'],
        # Add other filters as desired.
        ['name', '=', 'can_login'],
        ['tail_uuid', '=', 'zzzzz-tpzed-12345abcde67890'],
        ...,
    ],
):
    arv_client.links().delete(
        uuid=permission['uuid'],
    ).execute()

Working with properties

Container requests, collections, groups, and links can have metadata properties set through their properties field. For details, refer to the Metadata properties API reference.

An Arvados cluster can be configured to use a metadata vocabulary. If this is set up, the vocabulary defines standard identifiers and specific properties and their values. These identifiers can also have more human-friendly aliases. The cluster can also be configured to use the vocabulary strictly, so clients may only set properties on objects that are defined in the vocabulary. For more information about configuring a metadata vocabulary, refer to the Metadata vocabulary administration documentation.

Update the properties of an object

To set an object’s properties to a new value, just call the resource’s update method with a new properties field in the body. If you want to make changes to the current set of properties, get the object, build a new dictionary based on its properties field, then call the resource’s update method with your new dictionary as the properties. Below is an example for a container request.

container_request = arv_client.container_requests().get(
    uuid='zzzzz-xvhdp-12345abcde67890',
).execute()
new_properties = dict(container_request['properties'])
...  # Make your desired changes to new_proprties
container_request = arv_client.container_requests().update(
    uuid=container_request['uuid'],
    body={
        'container_request': {
            'properties': new_properties,
        },
    },
).execute()

Translate between vocabulary identifiers and labels

Client software might need to present properties to the user in a human-readable form or take input from the user without requiring them to remember identifiers. The Vocabulary.convert_to_labels and Vocabulary.convert_to_identifiers methods help with these tasks, respectively.

import arvados.vocabulary
vocabulary = arvados.vocabulary.load_vocabulary(arv_client)

# The argument should be a mapping of vocabulary keys and values using any
# defined aliases, like this:
#   {'Creature': 'Human', 'Priority': 'Normal'}
# The return value will be an analogous mapping where all the aliases have
# been translated to identifiers, like this:
#   {'IDTAGANIMALS': 'IDVALANIMALS2', 'IDTAGIMPORTANCES': 'IDTAGIMPORTANCES1'}
properties_by_identifier = vocabulary.convert_to_identifiers({...})

# You can use this to set metadata properties on objects that support them.
project = arv_client.groups().update(
    uuid='zzzzz-j7d0g-12345abcde67890',
    body={
        'group': {
            'properties': properties_by_identifier,
        },
    },
).execute()

# You can report properties to the user by their preferred name.
print(f"{project['name']} ({project['group_class']} {project['uuid']}) updated with properties:")
for key, value in vocabulary.convert_to_labels(project['properties']).items():
    print(f"↳ {key}: {value}")

Query the vocabulary definition

The arvados.vocabulary module provides facilities to interact with the active metadata vocabulary in the system. The Vocabulary class provides a mapping-like view of a cluster’s configured vocabulary.

import arvados.vocabulary
vocabulary = arvados.vocabulary.load_vocabulary(arv_client)

# You can use the vocabulary object to access specific keys and values by
# case-insensitive mapping, like this:
#   vocabulary_value = vocabulary[key_alias][value_alias]
# You can also access the `key_aliases` and `value_aliases` mapping
# attributes directly to view the entire vocabulary. The example below
# writes a plaintext table of the vocabulary.
for vocabulary_key in set(vocabulary.key_aliases.values()):
    print(
        vocabulary_key.identifier,
        vocabulary_key.preferred_label,
        ', '.join(vocabulary_key.aliases[1:]),
        sep='\t',
    )
    for vocabulary_value in set(vocabulary_key.value_aliases.values()):
        print(
            f'↳ {vocabulary_value.identifier}',
            vocabulary_value.preferred_label,
            ', '.join(vocabulary_value.aliases[1:]),
            sep='\t',
        )

Working with collections

The arvados.collection.Collection class provides a high-level interface to read, create, and update collections. It orchestrates multiple requests to API and Keep so you don’t have to worry about the low-level details of keeping everything in sync. It uses threads to make multiple requests to Keep in parallel.

This page only shows you how to perform common tasks using the Collection class. To see all the supported constructor arguments and methods, refer to the Collection class documentation.

Load and update an existing collection

Construct the Collection class with the UUID of a collection you want to read. You can pass additional constructor arguments as needed.

import arvados.collection
collection = arvados.collection.Collection('zzzzz-4zz18-12345abcde67890', ...)

If you make changes to the collection and want to update the existing collection, call the Collection.save method:

collection.save()

If you would rather save your changes as a new collection object, call the Collection.save_new method. This example illustrates some popular arguments. Check the API reference for full details of all possible arguments.

collection.save_new(
    name='Collection updated by Python SDK',
    # owner_uuid can be the UUID for an Arvados user or group.
    # Specify the UUID of a project to add this collection to it.
    owner_uuid='zzzzz-j7d0g-12345abcde67890',
)

Create and save a new collection

Construct the Collection class without an existing collection UUID or manifest text. You can pass additional constructor arguments as needed.

import arvados.collection
new_collection = arvados.collection.Collection(...)

Usually you’ll upload or copy files to the new collection. Once you’re done with that and ready to save your changes, call the Collection.save_new method. This example illustrates some popular arguments. Check the API reference for full details of all possible arguments.

new_collection.save_new(
    name='Collection created by Python SDK',
    # owner_uuid can be the UUID for an Arvados user or group.
    # Specify the UUID of a project to add this collection to it.
    owner_uuid='zzzzz-j7d0g-12345abcde67890',
)

Read a file from a collection

Once you have a Collection object, the Collection.open method lets you open files from a collection the same way you would open files from disk using Python’s built-in open function. It returns a file-like object that you can use in many of the same ways you would use any other file object. This example prints all non-empty lines from ExampleFile in your collection:

import arvados.collection
collection = arvados.collection.Collection(...)
with collection.open('ExampleFile') as my_file:
    # Read from my_file as desired.
    # This example prints all non-empty lines from the file to stdout.
    for line in my_file:
        if not line.isspace():
            print(line, end='')

Download a file from a collection

Once you have a Collection object, the Collection.open method lets you open files from a collection the same way you would open files from disk using Python’s built-in open function. You pass a second mode argument like 'rb' to open the file in binary mode. It returns a file-like object that you can use in many of the same ways you would use any other file object. You can pass it as a source to Python’s standard shutil.copyfileobj function to download it. This code downloads ExampleFile from your collection and saves it to the current working directory as ExampleDownload:

import arvados.collection
import shutil
collection = arvados.collection.Collection(...)
with (
  collection.open('ExampleFile', 'rb') as src_file,
  open('ExampleDownload', 'wb') as dst_file,
):
    shutil.copyfileobj(src_file, dst_file)

Write a file to a collection

Once you have a Collection object, the Collection.open method lets you open files from a collection the same way you would open files from disk using Python’s built-in open function. Pass a second mode argument like 'w', 'a', or 'wb' to write a file in the collection. It returns a file-like object that you can use in many of the same ways you would use any other file object. This example writes Hello, Arvados! to a file named ExampleHello in your collection:

import arvados.collection
collection = arvados.collection.Collection(...)
with collection.open('ExampleFile', 'w') as my_file:
    # Write to my_file as desired.
    # This example writes "Hello, Arvados!" to the file.
    print("Hello, Arvados!", file=my_file)
collection.save_new(...)  # or collection.save() to update an existing collection

Upload a file to a collection

Once you have a Collection object, the Collection.open method lets you open files from a collection the same way you would open files from disk using Python’s built-in open function. Pass a second mode argument like 'w', 'a', or 'wb' to write a file in the collection. It returns a file-like object that you can use in many of the same ways you would use any other file object. You can pass it as a destination to Python’s standard shutil.copyfileobj function to upload data from a source file. This example reads ExampleFile from the current working directory and uploads it into your collection as ExampleUpload:

import arvados.collection
import shutil
collection = arvados.collection.Collection(...)
with (
  open('ExampleFile', 'rb') as src_file,
  collection.open('ExampleUpload', 'wb') as dst_file,
):
    shutil.copyfileobj(src_file, dst_file)
collection.save_new(...)  # or collection.save() to update an existing collection

Delete a file from a collection

Once you have a Collection object, call the Collection.remove method with a file path to remove that file or directory from the collection.

import arvados.collection
collection = arvados.collection.Collection(...)
collection.remove('ExamplePath')
collection.save_new(...)  # or collection.save() to update an existing collection

Delete a directory from a collection recursively

Once you have a Collection object, call the Collection.remove method with a directory path and recursive=True to delete everything under that directory from the collection.

import arvados.collection
collection = arvados.collection.Collection(...)
collection.remove('ExampleDirectoryPath', recursive=True)
collection.save_new(...)  # or collection.save() to update an existing collection

Walk over all files in a collection

Once you have a Collection object, you can iterate over it to retrieve the names of all files and streams in it. Streams are like subdirectories: you can open them using the Collection.find method, and work with the files in them just like you would in the original collection. This example shows how to combine these techniques to iterate all files in a collection, including its streams.

import arvados.collection
import collections
import pathlib
root_collection = arvados.collection.Collection(...)
# Start work from the base stream.
stream_queue = collections.deque([pathlib.PurePosixPath('.')])
while stream_queue:
    stream_path = stream_queue.popleft()
    collection = root_collection.find(str(stream_path))
    for item_name in collection:
        try:
            my_file = collection.open(item_name)
        except IsADirectoryError:
            # item_name refers to a stream. Queue it to walk later.
            stream_queue.append(stream_path / item_name)
            continue
        with my_file:
            ...  # Work with my_file as desired

Copy a file between collections

Once you have one or more Collection objects, call the Collection.copy method on the destination collection to copy files to it. This method doesn’t re-upload data, so it’s very efficient.

import arvados.collection
src_collection = arvados.collection.Collection(...)
dst_collection = arvados.collection.Collection(...)
dst_collection.copy(
    # The path of the source file or directory to copy
    'ExamplePath',
    # The path where the source file or directory will be copied.
    # Pass an empty string like this to copy it to the same path.
    '',
    # The collection where the source file or directory comes from.
    # If not specified, the default is the current collection (so you'll
    # make multiple copies of the same data in the same collection).
    source_collection=src_collection,
    # Pass overwrite=True to force the method to overwrite any data
    # that already exists at the given path in the current collection.
    overwrite=False,
)
dst_collection.save_new(...)  # or dst_collection.save() to update an existing collection

Combine two or more collections

You can concatenate manifest texts from multiple collections to create a single collection that contains all the data from the source collections. Note that if multiple source collections have data at the same path, the merged collection will have a single file at that path with concatenated data from the source collections.

import arvados.collection

# Retrieve all of the source collection manifest texts
src_collection_uuid_list = [
    'zzzzz-4zz18-111111111111111',
    'zzzzz-4zz18-222222222222222',
    ...,
]
manifest_texts = [
    arvados.collection.Collection(uuid).manifest_text()
    for uuid in src_collection_uuid_list
]

# Initialize a new collection object from the concatenated manifest text
new_collection = arvados.collection.Collection(''.join(manifest_texts), ...)

# Record the new collection in Arvados
new_collection.save_new(
    name='Collection merged by Python SDK',
    owner_uuid='zzzzz-j7d0g-12345abcde67890',
)

You can create a sharing link for a collection by creating a new API token that is only allowed to read that collection; then constructing a link to your Keep web server that includes the collection UUID and the new token.

import urllib.parse

# The UUID of the collection you want to share
collection_uuid = 'zzzzz-4zz18-12345abcde67890'

sharing_token_scopes = [
    'GET /arvados/v1/keep_services/accessible',
    f'GET /arvados/v1/collections/{collection_uuid}',
    f'GET /arvados/v1/collections/{collection_uuid}/',
]
sharing_token = arv_client.api_client_authorizations().create(
    body={
        'api_client_authorization': {
            'scopes': sharing_token_scopes,
        },
    },
).execute()
plain_token = sharing_token['api_token']
token_parts = plain_token.split('/')
if token_parts[0] == 'v2':
    plain_token = token_parts[2]

sharing_url_parts = (
    # The scheme your Keep web server uses. Change this to 'http' if necessary.
    'https',
    # The hostname, and optionally port, your Keep web server uses
    'collections.zzzzz.example.com',
    # You shouldn't need to change any other items
    f'/c={collection_uuid}/t={plain_token}/_/',
    None,
    None,
)
sharing_url = urllib.parse.urlunsplit(sharing_url_parts)
print(sharing_url)

Working with containers

If you haven’t already, start by reading the Computing with Crunch guide. It provides a high-level overview of how users submit work to Arvados as container requests; how Arvados dispatches that work to containers; and how Arvados records the association and results back on the original container request record.

If you have experience running CWL workflows on Workbench 2, it runs through this same API. When you start that workflow run, Workbench 2 creates a small container request to run a “CWL runner” tool with the specific inputs you gave it. Once Crunch dispatches a container for it, the CWL runner creates additional container requests to run each step of the workflow, and oversees the process until the workflow runs to completion. The UUID of this container is recorded in the container_uuid field of the container request you submitted.

The UUID of the CWL runner container is recorded in the requesting_container_uuid field of each container request it creates. You can list container requests with a filter on this field to inspect each step of the workflow individually, as shown below.

The next few examples show how to perform a task with a container request generally, and then provide a more specific example of working with a CWL runner container.

Get input of a container

A container request’s most varied inputs are recorded in the mounts field, which can include data from Keep, specific collections, Git checkouts, and static files. You might also be interested in the environment, command, container_image, and secret_mounts fields. Refer to the container requests API documentation for details.

container_request = arv_client.container_requests().get(
    uuid='zzzzz-xvhdp-12345abcde67890',
).execute()
# From here, you can process any of the container request's input fields.
# Below is an example of listing all the mounts.
import pprint
for mount_name, mount_source in container_request['mounts'].items():
    mount_summary = []
    # These are the fields that define different types of mounts.
    # Try to collect them all. Just skip any that aren't set.
    for key in ['kind', 'uuid', 'portable_data_hash', 'commit', 'path']:
        try:
            mount_summary.append(mount_source[key])
        except KeyError:
            pass
    print(f"{mount_name}: {' '.join(mount_summary)}")
    if mount_source.get('kind') == 'json':
        pprint.pprint(mount_source.get('content'))

Get input of a CWL workflow run

When you run a CWL workflow, the CWL inputs are stored in the container request’s mounts field as a JSON mount named /var/lib/cwl/cwl.input.json.

container_request = arv_client.container_requests().get(
    uuid='zzzzz-xvhdp-12345abcde67890',
).execute()
cwl_input = container_request['mounts']['/var/lib/cwl/cwl.input.json']['content']
...  # Work with the cwl_input dictionary

Get output of a container

A container’s output files are saved in a collection. The UUID of that collection is recorded in the output_uuid of the container request, which you can load as you like.

import arvados.collection
container_request = arv_client.container_requests().get(
    uuid='zzzzz-xvhdp-12345abcde67890',
).execute()
container_output = arvados.collection.Collection(
    container_request.get('output_uuid'),
)
...  # Work with the container_output collection object

Get output of a CWL workflow run

When you run a CWL workflow, the container request’s output collection includes a file named cwl.output.json that provides additional information about other files in the output.

import arvados.collection
import json
cwl_container_request = arv_client.container_requests().get(
    uuid='zzzzz-xvhdp-12345abcde67890',
).execute()
cwl_output_collection = arvados.collection.Collection(
    cwl_container_request['output_uuid'],
)
with cwl_output_collection.open('cwl.output.json') as cwl_output_file:
    cwl_output = json.load(cwl_output_file)
...  # Work with the cwl_output dictionary

Get logs of a container or CWL workflow run

A container’s log files are saved in a collection. The UUID of that collection is recorded in the log_uuid of the container request, which you can load as you like.

import arvados.collection
container_request = arv_client.container_requests().get(
    uuid='zzzzz-xvhdp-12345abcde67890',
).execute()
log_collection = arvados.collection.Collection(
    container_request['log_uuid'],
)
# From here, you can process the container's log collection any way you like.
# Below is an example that writes the container's stderr to this process' stderr.
import shutil
import sys
with log_collection.open('stderr.txt') as containter_stderr:
    shutil.copyfileobj(container_stderr, sys.stderr)

Get status of a container or CWL workflow run

Workbench shows users a single status badge for container requests. This status is synthesized from different fields on the container request and associated container. This code shows how to do analogous reporting using the Python SDK.

container_request = arv_client.container_requests().get(
    uuid='zzzzz-xvhdp-12345abcde67890',
).execute()
if container_request['container_uuid'] is None:
    status = container_request['state']
else:
    container = arv_client.containers().get(
        uuid=container_request['container_uuid'],
    ).execute()
    container_state = container['state']
    if container_state == 'Queued' or container_state == 'Locked':
        status = "On hold" if container['priority'] == 0 else "Queued"
    elif container_state == 'Running':
        if container['runtime_status'].get('error'):
            status = "Failing"
        elif container['runtime_status'].get('warning'):
            status = "Warning"
        else:
            status = container_state
    elif container_state == 'Cancelled':
        status = container_state
    elif container_state == 'Complete':
        status = "Completed" if container['exit_code'] == 0 else "Failed"
...  # Report status as desired

List child requests of a container or CWL workflow run

import arvados.util
for child_container_requests in arvados.util.keyset_list_all(
    # Pass the method keyset_list_all will call to retrieve items.
    # Do not call it yourself.
    arv_client.container_requests().list,
    filters=[
        # Note this is a container UUID, *not* a container request UUID
        ['requesting_container_uuid', '=', 'zzzzz-dz642-12345abcde67890'],
        # You may add other filters for your listing.
        # For example, you could filter by 'name' to find specific kinds
        # of steps of a CWL workflow.
        ...,
    ],
):
    ...  # Work with each child container request

List child requests of a container request

When a running container creates a container request to do additional work, the UUID of the source container is recorded in the requesting_container_uuid field of the new container request. If all you have is the UUID of a container request, you can get that request, then list container requests with a filter where requesting_container_uuid matches the container_uuid of your request to find all its children.

import arvados.util
parent_container_request = arv_client.container_requests().get(
    uuid='zzzzz-xvhdp-12345abcde67890',
).execute()
parent_container_uuid = parent_container_request['container_uuid']
if parent_container_uuid is None:
    # No container has run for this request yet, so there cannot be child requests.
    child_container_requests = ()
else:
    child_container_requests = arvados.util.keyset_list_all(
        # Pass the method keyset_list_all will call to retrieve items.
        # Do not call it yourself.
        arv_client.container_requests().list,
        filters=[
            ['requesting_container_uuid', '=', parent_container_uuid],
            # You may add other filters for your listing.
            # For example, you could filter by 'name' to find specific kinds
            # of steps of a CWL workflow.
            ...,
        ],
    )
for child_container_request in child_container_requests:
    ...  # Work with each child container request

With each child container request, you could repeat any of the recipes listed earlier in this section: examine their status, inputs, outputs, logs, and so on.

Working with the container request queue

List completed container requests

Completed container requests have their state field set to "Final". You can list container requests with this filter to find completed requests.

import arvados.util
import datetime
time_filter = datetime.datetime.utcnow()
time_filter -= datetime.timedelta(days=7)

for container_request in arvados.util.keyset_list_all(
    # Pass the method keyset_list_all will call to retrieve items.
    # Do not call it yourself.
    arv_client.container_requests().list,
    filters=[
        # This is the filter you need to find completed container requests.
        ['state', '=', 'Final'],
        # There could be many completed container requests, so you should
        # provide additional filters. This example limits the listing to
        # container requests from the past week.
        ['created_at', '>=', f'{time_filter.isoformat()}Z'],
        ...,
    ],
):
    # Work with each container_request as desired.
    # This example provides a basic status table with the container request
    # UUID, time the request was created, and time the container finished
    # (both in UTC).
    print(
        container_request['uuid'],
        container_request['created_at'],
        container_request['modified_at'],
    )

Cancel a container request

To cancel a container request, update it to set its priority field to 0. See the containers API reference for details.

cancelled_container_request = arv_client.container_requests().update(
    uuid='zzzzz-xvhdp-12345abcde67890',
    body={
        'container_request': {
            'priority': 0,
        },
    },
).execute()

Cancel multiple pending container requests

If you want to cancel multiple pending container requests, you can list container requests with the state field set to "Committed", a priority greater than zero, and any other filters you like. Then update each container request to set its priority field to 0. See the containers API reference for details.

import arvados.util
for container_request in arvados.util.keyset_list_all(
    # Pass the method keyset_list_all will call to retrieve items.
    # Do not call it yourself.
    arv_client.container_requests().list,
    filters=[
        # These are the filters you need to find cancellable container requests.
        ['state', '=', 'Committed'],
        ['priority', '>', 0],
        # You can add other filters as desired.
        # For example, you might filter on `requesting_container_uuid` to
        # cancel only steps of one specific workflow.
        ...,
    ],
):
    cancelled_container_request = arv_client.container_requests().update(
        uuid=container_request['uuid'],
        body={
            'container_request': {
                'priority': 0,
            },
        },
    ).execute()

Previous: Arvados API Client Next: PyDoc Reference