Package arvados :: Package commands :: Module arv_copy

[frames] | no frames]

Module arv_copy

source code

Classes

[hide private]

ProgressWriter

Functions

[hide private]

main()

source code

set_src_owner_uuid(resource, uuid, args)

source code

api_for_instance(instance_name)

source code

check_git_availability()

source code

copy_pipeline_instance(pi_uuid, src, dst, args)

source code

filter_iter(arg)
Iterate a filter string-or-list.

source code

migrate_repository_filter(repo_filter, src_repository, dst_repository)
Update a single repository filter in-place for the destination.

source code

migrate_script_version_filter(version_filter)
Update a single script_version filter in-place for the destination.

source code

attr_filtered(filter_, *attr_names)
Return True if filter_ applies to any of attr_names, else False.

source code

exception_handler(handler, *exc_types)
If any exc_types are raised in the block, call handler on the exception.

source code

migrate_components_filters(template_components, dst_git_repo)
Update template component filters in-place for the destination.

source code

copy_pipeline_template(pt_uuid, src, dst, args)

source code

copy_workflow(wf_uuid, src, dst, args)

source code

workflow_collections(obj, locations, docker_images)

source code

copy_collections(obj, src, dst, args)

source code

migrate_jobspec(jobspec, src, dst, dst_repo, args)
Copy a job's script to the destination repository, and update its record.

source code

copy_git_repos(p, src, dst, dst_repo, args)

source code

total_collection_size(manifest_text)
Return the total number of bytes in this collection (excluding duplicate blocks).

source code

create_collection_from(c, src, dst, args)
Create a new collection record on dst, and copy Docker metadata if available.

source code

copy_collection(obj_uuid, src, dst, args)

source code

select_git_url(api, repo_name, retries, allow_insecure_http, allow_insecure_http_opt)

source code

copy_git_repo(src_git_repo, src, dst, dst_git_repo, script_version, args)

source code

copy_docker_images(pipeline, src, dst, args)
Copy any docker images named in the pipeline components' runtime_constraints field from src to dst.

source code

copy_docker_image(docker_image, docker_image_tag, src, dst, args)
Copy the docker image identified by docker_image and docker_image_tag from src to dst.

source code

git_rev_parse(rev, repo)

source code

uuid_type(api, object_uuid)

source code

abort(msg, code=1)

source code

machine_progress(obj_uuid, bytes_written, bytes_expected)

source code

human_progress(obj_uuid, bytes_written, bytes_expected)

source code

Variables

[hide private]

COMMIT_HASH_RE = re.compile(r'^[0-9a-f]{1,40}$')

logger = logging.getLogger('arvados.arv-copy')

local_repo_dir = {}

collections_copied = {}

scripts_copied = set()

src_owner_uuid = None
hash(x)

Function Details

[hide private]

filter_iter(arg)

source code

Iterate a filter string-or-list.

Pass in a filter field that can either be a string or list. This will iterate elements as if the field had been written as a list.

migrate_repository_filter(repo_filter, src_repository, dst_repository)

source code

Update a single repository filter in-place for the destination.

If the filter checks that the repository is src_repository, it is updated to check that the repository is dst_repository. If it does anything else, this function raises ValueError.

migrate_script_version_filter(version_filter)

source code

Update a single script_version filter in-place for the destination.

Currently this function checks that all the filter operands are Git commit hashes. If they're not, it raises ValueError to indicate that the filter is not portable. It could be extended to make other transformations in the future.

exception_handler(handler, *exc_types)

source code

If any exc_types are raised in the block, call handler on the exception.

Decorators:

@contextlib.contextmanager

migrate_components_filters(template_components, dst_git_repo)

source code

Update template component filters in-place for the destination.

template_components is a dictionary of components in a pipeline template. This method walks over each component's filters, and updates them to have identical semantics on the destination cluster. It returns a list of error strings that describe what filters could not be updated safely.

dst_git_repo is the name of the destination Git repository, which can be None if that is not known.

migrate_jobspec(jobspec, src, dst, dst_repo, args)

source code

Copy a job's script to the destination repository, and update its record.

Given a jobspec dictionary, this function finds the referenced script from src and copies it to dst and dst_repo. It also updates jobspec in place to refer to names on the destination.

copy_docker_image(docker_image, docker_image_tag, src, dst, args)

source code

Copy the docker image identified by docker_image and docker_image_tag from src to dst. Create appropriate docker_image_repo+tag and docker_image_hash links at dst.