Package arvados :: Package commands :: Module put :: Class ArvPutUploadJob
[hide private]
[frames] | no frames]

Class ArvPutUploadJob

source code

builtins.object --+

Instance Methods [hide private]
__init__(self, paths, resume=True, use_cache=True, reporter=None, name=None, owner_uuid=None, api_client=None, ensure_unique_name=False, num_retries=None, put_threads=None, replication_desired=None, filename=None, update_time=60.0, update_collection=None, storage_classes=None, logger=logging.getLogger('arvados.arv_put'), dry_run=False, follow_links=True, exclude_paths=[], exclude_names=None) source code
Scan the requested paths to count file sizes, excluding files & dirs if requested and building the upload file list.
source code
start(self, save_collection)
Start supporting thread & file uploading
source code
save_collection(self) source code
destroy_cache(self) source code
_collection_size(self, collection)
Recursively get the total size of the collection
source code
Periodically called support task.
source code
_update(self, final=False)
Update cached manifest text and report progress.
source code
report_progress(self) source code
_write_stdin(self, filename) source code
_check_file(self, source, filename)
Check if this file needs to be uploaded
source code
_upload_files(self) source code
_write(self, source_fd, output) source code
_my_collection(self) source code
_setup_state(self, update_collection)
Create a new cache file or load a previously existing one.
source code
collection_file_paths(self, col, path_prefix='.')
Return a list of file paths by recursively go through the entire collection `col`
source code
_lock_file(self, fileobj) source code
Atomically save current state into cache.
source code
collection_name(self) source code
manifest_locator(self) source code
portable_data_hash(self) source code
manifest_text(self, stream_name=".", strip=False, normalize=False) source code
_datablocks_on_item(self, item)
Return a list of datablock locators, recursively navigating through subcollections
source code
data_locators(self) source code
Class Variables [hide private]
  CACHE_DIR = '.cache/arvados/arv-put'
  EMPTY_STATE = {'manifest': None, 'files': {}}
Method Details [hide private]


source code 

Periodically called support task. File uploading is asynchronous so we poll status from the collection.