Module arvados.util

Functions

def ca_certs_path(fallback='/var/lib/gitolite3/venv-pdoc3/lib/python3.9/site-packages/certifi/cacert.pem')

Return the path of the best available CA certs source.

This function searches for various distribution sources of CA certificates, and returns the first it finds. If it doesn't find any, it returns the value of fallback (httplib2's CA certs by default).

def clear_tmpdir(path=None)

Ensure the given directory (or TASK_TMPDIR if none given) exists and is empty.

def collection_extract(collection, path, files=[], decompress=True)

Retrieve a collection from Keep and extract it to a local directory. Return the absolute path where the collection was extracted.

collection – collection locator path – where to extract: absolute, or relative to job tmp

def get_config_once(svc)

def get_vocabulary_once(svc)

def git_checkout(url, version, path)

def is_hex(s, *length_args)

is_hex(s[, length[, max_length]]) -> boolean

Return True if s is a string of hexadecimal digits. If one length argument is given, the string must contain exactly that number of digits. If two length arguments are given, the string must contain a number of digits between those two lengths, inclusive. Return False otherwise.

def keyset_list_all(fn, order_key='created_at', num_retries=0, ascending=True, **kwargs)

def list_all(fn, num_retries=0, **kwargs)

def listdir_recursive(dirname, base=None, max_depth=None)

listdir_recursive(dirname, base, max_depth)

Return a list of file and directory names found under dirname.

If base is not None, prepend "{base}/" to each returned name.

If max_depth is None, descend into directories and return only the names of files found in the directory tree.

If max_depth is a non-negative integer, stop descending into directories at the given depth, and at that point return directory names instead.

If max_depth==0 (and base is None) this is equivalent to sorted(os.listdir(dirname)).

def mkdir_dash_p(path)

def new_request_id()

def run_command(execargs, **kwargs)

def stream_extract(stream, path, files=[], decompress=True)

Retrieve a stream from Keep and extract it to a local directory. Return the absolute path where the stream was extracted.

stream – StreamReader object path – where to extract: absolute, or relative to job tmp

def tar_extractor(path, decompress_flag)

def tarball_extract(tarball, path)

Retrieve a tarball from Keep and extract it to a local directory. Return the absolute path where the tarball was extracted. If the top level of the tarball contained just one file or directory, return the absolute path of that single item.

tarball – collection locator path – where to extract the tarball: absolute, or relative to job tmp

def trim_name(collectionname)

trim_name takes a record name (collection name, project name, etc) and trims it to fit the 255 character name limit, with additional space for the timestamp added by ensure_unique_name, by removing excess characters from the middle and inserting an ellipse

def zipball_extract(zipball, path)

Retrieve a zip archive from Keep and extract it to a local directory. Return the absolute path where the archive was extracted. If the top level of the archive contained just one file or directory, return the absolute path of that single item.

zipball – collection locator path – where to extract the archive: absolute, or relative to job tmp