Arvados™ Docs
WARNING - you are viewing the documentation for an old version of Arvados. For the latest version, click here.
  1. Concepts
    1. API Reference
    2. API Authorization
    3. REST API syntax
    4. Common resource methods
    5. Common resource fields
  2. Permission and authentication
    1. api_client_authorizations
    2. api_clients
    3. authorized_keys
    4. groups
    5. users
    6. user_agreements
  3. System resources
    1. keep_services
    2. links
    3. logs
    4. nodes
    5. virtual_machines
    6. keep_disks (deprecated)
  4. Data management
    1. WebDAV
    2. S3 API
    3. Keep-web URL patterns
    4. Projects and filter groups
    5. Metadata properties
    6. collections
    7. repositories
  5. Container engine
    1. container_requests
    2. containers
    3. workflows
  6. Management (admin/system)
    1. cloud dispatcher
  7. Jobs engine (legacy)
    1. Crunch scripts
    2. jobs
    3. job_tasks
    4. pipeline_instances
    5. pipeline_templates
  8. Metadata for bioinformatics (deprecated)
    1. humans
    2. specimens
    3. traits

Common resource methods

The following methods are available for most resources. Some resources may limit who can perform certain operations. Consult documentation for individual resource types for details.

The methods are relative to the base URI, e.g., /arvados/v1/resource_type. For arguments specifying a Location of path, the value of the argument is incorporated into the path portion of the URI. For example, a uuid of aaaaa-bbbbb-ccccccccccccccc in a path position yields a URI of /arvados/v1/resource_type/aaaaa-bbbbb-ccccccccccccccc.

Arguments specifying a Location of “query” are incorporated into the query portion of the URI or request body. For example, /arvados/v1/resource_type?count=none.

Certain method calls on certain object types support federation , that is, the ability to operate on objects owned by different clusters. API pages for specific object types list which federated operations are supported for that type (if any) in the “Methods” section. Methods which implicitly include a cluster id (such as GET on a specific uuid, using the uuid prefix) will be directed to the appropriate cluster. Methods that don’t implicitly include the cluster id (such as create) use the cluster_id query parameter to specify which cluster to direct the request.

create

The create method creates a new object of the specified type. Note that:

  • Only the listed attributes (and standard metadata) are set
  • Unset attributes will get default values
  • The attributes of a given resource type are fixed (you cannot introduce new toplevel attributes)

This method corresponds to the HTTP request POST /arvados/v1/resource_type. A successful create call returns a copy of the new object.

To create an object on a remote cluster (federated create), provide the cluster_id of the target cluster.

Arguments:

Argument Type Description Location
{resource_type} object Name is the singular form of the resource type, e.g., for the “collections” resource, this argument is “collection” body
{cluster_id} string Optional, the cluster on which to create the object if not the current cluster. query
select array Attributes of the new object to return in the response (by default, all available attributes are returned).
Example: ["uuid","name","modified_at"]		 query

delete

The delete method deletes an object of the specified type. It corresponds to the HTTP request DELETE /arvados/v1/resource_type/uuid. A successful delete call returns a copy of the deleted object.

The cluster id portion of the uuid is used to determine which cluster owns the object, a federated delete request will be routed to that cluster.

Arguments:

Argument Type Description Location
uuid string The UUID of the object in question. path
select array Attributes of the deleted object to return in the response (by default, all available attributes are returned).
Example: ["uuid","name","modified_at"]		 query

get

The get method gets a single object with the specified uuid. It corresponds to the HTTP request GET /arvados/v1/resource_type/uuid.

The cluster id portion of the uuid is used to determine which cluster owns the object, a federated get request will be routed to that cluster.

Arguments:

Argument Type Description Location
uuid string The UUID of the object in question. path
select array Attributes of the object to return in the response (by default, all available attributes are returned).
Example: ["uuid","name","modified_at"]		 query

list

The list method requests an list of resources of that type. It corresponds to the HTTP request GET /arvados/v1/resource_type. All resources support the list method unless otherwise noted.

Arguments:

Argument Type Description Location
limit integer Maximum number of resources to return. If not provided, server will provide a default limit. Server may also impose a maximum number of records that can be returned in a single request. query
offset integer Skip the first ‘offset’ number of resources that would be returned under the given filter conditions. query
filters array Conditions for selecting resources to return. query
order array Attributes to use as sort keys to determine the order resources are returned, each optionally followed by asc or desc to indicate ascending or descending order. (If not specified, it will be ascending).
Example: ["head_uuid asc","modified_at desc"]
Default: ["modified_at desc", "uuid asc"]		 query
select array Attributes of each object to return in the response (by default, all available attributes are returned, except collections, which do not return manifest_text unless explicitly selected).
Example: ["uuid","name","modified_at"]		 query
distinct boolean When returning multiple records whose selected attributes (see select) are equal, return them as a single response entry.
Default is false.		 query
count string "exact" (default): Include an items_available response field giving the number of distinct matching items that can be retrieved (irrespective of limit and offset arguments).
"none": Omit the items_available response field. This option will produce a faster response.		 query

Available list method filters

The value of the filters parameter is an array of conditions. The list method returns only the resources that satisfy all of the given conditions. In other words, the conjunction AND is implicit.

Each condition is expressed as an array with three elements: [attribute, operator, operand].

Index Element Type Description Examples
0 attribute string Name of the attribute to compare (or “any” to return resources with any matching attribute) script_version, head_uuid, any
1 operator string Comparison operator >, >=, like, not in
2 operand string, array, or null Value to compare with the resource attribute "d00220fb%", "1234", ["foo","bar"], nil

The following operators are available.

Operator Operand type Description Example
=, !=, <> string, number, timestamp, JSON-encoded array, JSON-encoded object, or null Equality comparison ["tail_uuid","=","xyzzy-j7d0g-fffffffffffffff"]
["tail_uuid","!=",null]
["storage_classes_desired","=","[\"default\"]"]
<, <=, >=, > string, number, or timestamp Ordering comparison ["script_version",">","123"]
like, ilike string SQL pattern match. Single character match is _ and wildcard is %. The ilike operator is case-insensitive ["script_version","like","d00220fb%"]
in, not in array of strings Set membership ["script_version","in",["main","d00220fb38d4b85ca8fc28a8151702a2b9d1dec5"]]
is_a string Arvados object type ["head_uuid","is_a","arvados#collection"]
exists string Presence of subproperty ["properties","exists","my_subproperty"]
contains string, array of strings Presence of one or more keys or array elements ["storage_classes_desired", "contains", ["foo", "bar"]] (matches both ["foo", "bar"] and ["foo", "bar", "baz"])
(note [..., "contains", "foo"] is also accepted, and is equivalent to [..., "contains", ["foo"]])

Filtering using substring search

Resources can also be filtered by searching for a substring in attributes of type string, array of strings, text, and hash, which are indexed in the database specifically for search. To use substring search, the filter must:

  • Specify any as the attribute
  • Use either the like or ilike operator
  • Have an operand of type string that is wrapped in the SQL pattern match wildcard character %

For example, the ["any", "like", "%foo%"] filter will return all resources that contain foo in the content of at least one attribute of the previously defined types. This is the recommended way to do keyword and file name search across the entire database. Note that only exact substring matches are returned and results are unranked and returned in the order specified by the list order argument.

Filtering on subproperties

Some record types have an additional properties attribute that allows recording and filtering on additional key-value pairs. To filter on a subproperty, the value in the attribute position has the form properties.user_property. You may also use JSON-LD / RDF style URIs for property keys by enclosing them in <...> for example properties.<http://example.com/user_property>. Alternately you may also provide a JSON-LD “@context” field, however at this time JSON-LD contexts are not interpreted by Arvados.

Operator Operand type Description Example
=, != string, number or boolean Equality comparison ["properties.my_subproperty", "=", "fizzy whizy sparkle pop"]
<, <=, >=, > string or number Ordering comparison ["properties.my_subproperty", "<", 3]
like, ilike string SQL pattern match, single character match is _ and wildcard is %, ilike is case-insensitive ["properties.my_subproperty", "like", "d00220fb%"]
in, not in array of strings Set membership ["properties.my_subproperty", "in", ["fizz", "buzz"]]
exists boolean Test if a subproperty is present or not (determined by operand). ["properties.my_subproperty", "exists", true]
contains string, number Filter where subproperty has a value either by exact match or value is element of subproperty list. ["properties.foo", "contains", "bar"] will find both {"foo": "bar"} and {"foo": ["bar", "baz"]}.

Note that exclusion filters != and not in will return records for which the property is not defined at all. To restrict filtering to records on which the subproperty is defined, combine with an exists filter.

Filtering using boolean expressions

In addition to the three-element array form described above, a string containing a boolean expression is also accepted. The following restrictions apply:

  • The expression must contain exactly one operator.
  • The operator must be =, <, <=, >, or >=.
  • There must be exactly one pair of parentheses, surrounding the entire expression.
  • Each operand must be the name of a numeric attribute like replication_desired (literal values like 3 and non-numeric attributes like uuid are not accepted).
  • The expression must not contain whitespace other than an ASCII space (newline and tab characters are not accepted).

Examples:

  • (replication_desired > replication_confirmed)
  • (replication_desired = replication_confirmed)

Both types of filter (boolean expressions and [attribute, operator, operand] filters) can be combined in the same API call. Example:

  • {"filters": ["(replication_desired > replication_confirmed)", ["replication_desired", "<", 2]]}

Federated listing

Federated listing forwards a request to multiple clusters and combines the results. Currently only a very restricted form of the “list” method is supported.

To query multiple clusters, the list request must:

  • Have filters only matching [["uuid", "in", [...]] or ["uuid", "=", "..."]
  • Specify count=none
  • Not specify limit, offset or order
  • Not request more items than the maximum response size

This form may be used to request a specific list of objects by uuid which are owned by multiple clusters.

Results of list method

A successful call to list will return the following object.

Attribute Type Description
kind string type of objects returned
offset integer query offset in effect
limit integer query limit in effect
items array actual query payload, an array of resource objects
items_available integer total items available matching query

update

The update method updates fields on the object with the specified uuid. It corresponds to the HTTP request PUT /arvados/v1/resource_type/uuid. Note that only the listed attributes (and standard metadata) are updated, unset attributes will retain their previous values, and the attributes of a given resource type are fixed (you cannot introduce new toplevel attributes). Also note that updates replace the value of the attribute, so if an attribute has an object value, the entire object is replaced. A successful update call returns the updated copy of the object.

The cluster id portion of the uuid is used to determine which cluster owns the object, a federated update request will be routed to that cluster.

Argument Type Description Location
uuid string The UUID of the resource in question. path
{resource_type} object query
select array Attributes of the updated object to return in the response (by default, all available attributes are returned).
Example: ["uuid","name","modified_at"]		 query
Previous: REST API syntax Next: Common resource fields

The content of this documentation is licensed under the Creative Commons Attribution-Share Alike 3.0 United States licence.
Code samples in this documentation are licensed under the Apache License, Version 2.0.