Home       Trees       Indices       Help   
Package arvados :: Module collection :: Class Collection
[hide private]
[frames] | no frames]

Class Collection

source code

builtins.object --+        
                  |        
     CollectionBase --+    
                      |    
     RichCollectionBase --+
                          |
                         Collection

Represents the root of an Arvados Collection.

This class is threadsafe.  The root collection object, all subcollections
and files are protected by a single lock (i.e. each access locks the entire
collection).

Brief summary of
useful methods:

:To read an existing file:
  `c.open("myfile", "r")`

:To write a new file:
  `c.open("myfile", "w")`

:To determine if a file exists:
  `c.find("myfile") is not None`

:To copy a file:
  `c.copy("source", "dest")`

:To delete a file:
  `c.remove("myfile")`

:To save to an existing collection record:
  `c.save()`

:To save a new collection record:
`c.save_new()`

:To merge remote changes into this object:
  `c.update()`

Must be associated with an API server Collection record (during
initialization, or using `save_new`) to use `save` or `update`
Instance Methods [hide private]
 
__init__(self, manifest_locator_or_text=None, api_client=None, keep_client=None, num_retries=None, parent=None, apiconfig=None, block_manager=None, replication_desired=None, put_threads=None)
Collection constructor.		 source code
 
root_collection(self) source code
 
get_properties(self) source code
 
get_trash_at(self) source code
 
stream_name(self) source code
 
writable(self) source code
 
known_past_version(self, modified_at_and_portable_data_hash) source code
 
update(self, other=None, num_retries=None)
Merge the latest collection on the API server with the current collection.		 source code
 
_my_api(self) source code
 
_my_keep(self) source code
 
_my_block_manager(self) source code
 
_remember_api_response(self, response) source code
 
_populate_from_api_server(self) source code
 
_populate(self) source code
 
_has_collection_uuid(self) source code
 
__enter__(self) source code
 
__exit__(self, exc_type, exc_value, traceback)
Support scoped auto-commit in a with: block.		 source code
 
stop_threads(self) source code
 
manifest_locator(self)
Get the manifest locator, if any.		 source code
 
clone(self, new_parent=None, new_name=None, readonly=False, new_config=None) source code
 
api_response(self)
Returns information about this Collection fetched from the API server.		 source code
 
find_or_create(self, path, create_type)
See `RichCollectionBase.find_or_create`		 source code
 
find(self, path)
See `RichCollectionBase.find`		 source code
 
remove(self, path, recursive=False)
See `RichCollectionBase.remove`		 source code
 
save(self, properties=None, storage_classes=None, trash_at=None, merge=True, num_retries=None)
Save collection to an existing collection record.		 source code
 
save_new(self, name=None, create_collection_record=True, owner_uuid=None, properties=None, storage_classes=None, trash_at=None, ensure_unique_name=False, num_retries=None)
Save collection to a new collection record.		 source code
 
_unescape_manifest_path(self, path) source code
 
_import_manifest(self, manifest_text)
Import a manifest into a `Collection`.		 source code
 
notify(self, event, collection, name, item) source code

Inherited from RichCollectionBase: __contains__, __delitem__, __eq__, __getitem__, __iter__, __len__, __ne__, add, apply, committed, copy, diff, exists, flush, items, keys, manifest_text, mkdirs, modified, open, portable_data_hash, portable_manifest_text, rename, set_committed, subscribe, unsubscribe, values

Inherited from RichCollectionBase (private): _clonefrom, _get_manifest_text, _get_src_target

Inherited from CollectionBase: stripped_manifest

Class Variables [hide private]
  _token_re = re.compile(r'(\S+)(\s+|$)')
  _block_re = re.compile(r'[0-9a-f]{32}\+(\d+)(\+\S+)*')
  _segment_re = re.compile(r'(\d+):(\d+):(\S+)')
Method Details [hide private]

__init__(self, manifest_locator_or_text=None, api_client=None, keep_client=None, num_retries=None, parent=None, apiconfig=None, block_manager=None, replication_desired=None, put_threads=None)
(Constructor)

source code  
Collection constructor.

:manifest_locator_or_text:
  An Arvados collection UUID, portable data hash, raw manifest
  text, or (if creating an empty collection) None.

:parent:
  the parent Collection, may be None.

:apiconfig:
  A dict containing keys for ARVADOS_API_HOST and ARVADOS_API_TOKEN.
  Prefer this over supplying your own api_client and keep_client (except in testing).
  Will use default config settings if not specified.

:api_client:
  The API client object to use for requests.  If not specified, create one using `apiconfig`.

:keep_client:
  the Keep client to use for requests.  If not specified, create one using `apiconfig`.

:num_retries:
  the number of retries for API and Keep requests.

:block_manager:
  the block manager to use.  If not specified, create one.

:replication_desired:
  How many copies should Arvados maintain. If None, API server default
  configuration applies. If not None, this value will also be used
  for determining the number of block copies being written.
Overrides: RichCollectionBase.__init__

root_collection(self)

source code 
Overrides: RichCollectionBase.root_collection

stream_name(self)

source code 
Overrides: RichCollectionBase.stream_name

writable(self)

source code 
Overrides: RichCollectionBase.writable

known_past_version(self, modified_at_and_portable_data_hash)

source code 
Decorators:
  • @synchronized

update(self, other=None, num_retries=None)

source code 

Merge the latest collection on the API server with the current collection.

Decorators:
  • @synchronized
  • @retry_method

_my_api(self)

source code 
Decorators:
  • @synchronized
Overrides: RichCollectionBase._my_api

_my_keep(self)

source code 
Decorators:
  • @synchronized
Overrides: CollectionBase._my_keep

_my_block_manager(self)

source code 
Decorators:
  • @synchronized
Overrides: RichCollectionBase._my_block_manager

__enter__(self)

source code 
Overrides: CollectionBase.__enter__

__exit__(self, exc_type, exc_value, traceback)

source code 

Support scoped auto-commit in a with: block.

Overrides: CollectionBase.__exit__

manifest_locator(self)

source code 

Get the manifest locator, if any.

The manifest locator will be set when the collection is loaded from an API server record or the portable data hash of a manifest.

The manifest locator will be None if the collection is newly created or was created directly from manifest text. The method `save_new()` will assign a manifest locator.

Decorators:
  • @synchronized

clone(self, new_parent=None, new_name=None, readonly=False, new_config=None)

source code 
Decorators:
  • @synchronized
Overrides: RichCollectionBase.clone

api_response(self)

source code 

Returns information about this Collection fetched from the API server.

If the Collection exists in Keep but not the API server, currently returns None. Future versions may provide a synthetic response.

Decorators:
  • @synchronized

find_or_create(self, path, create_type)

source code 

See `RichCollectionBase.find_or_create`

Overrides: RichCollectionBase.find_or_create

find(self, path)

source code 

See `RichCollectionBase.find`

Overrides: RichCollectionBase.find

remove(self, path, recursive=False)

source code 

See `RichCollectionBase.remove`

Overrides: RichCollectionBase.remove

save(self, properties=None, storage_classes=None, trash_at=None, merge=True, num_retries=None)

source code  
Save collection to an existing collection record.

Commit pending buffer blocks to Keep, merge with remote record (if
merge=True, the default), and update the collection record. Returns
the current manifest text.

Will raise AssertionError if not associated with a collection record on
the API server.  If you want to save a manifest to Keep only, see
`save_new()`.

:properties:
  Additional properties of collection. This value will replace any existing
  properties of collection.

:storage_classes:
  Specify desirable storage classes to be used when writing data to Keep.

:trash_at:
  A collection is *expiring* when it has a *trash_at* time in the future.
  An expiring collection can be accessed as normal,
  but is scheduled to be trashed automatically at the *trash_at* time.

:merge:
  Update and merge remote changes before saving.  Otherwise, any
  remote changes will be ignored and overwritten.

:num_retries:
  Retry count on API calls (if None,  use the collection default)
Decorators:
  • @must_be_writable
  • @synchronized
  • @retry_method

save_new(self, name=None, create_collection_record=True, owner_uuid=None, properties=None, storage_classes=None, trash_at=None, ensure_unique_name=False, num_retries=None)

source code  
Save collection to a new collection record.

Commit pending buffer blocks to Keep and, when create_collection_record
is True (default), create a new collection record.  After creating a
new collection record, this Collection object will be associated with
the new record used by `save()`. Returns the current manifest text.

:name:
  The collection name.

:create_collection_record:
   If True, create a collection record on the API server.
   If False, only commit blocks to Keep and return the manifest text.

:owner_uuid:
  the user, or project uuid that will own this collection.
  If None, defaults to the current user.

:properties:
  Additional properties of collection. This value will replace any existing
  properties of collection.

:storage_classes:
  Specify desirable storage classes to be used when writing data to Keep.

:trash_at:
  A collection is *expiring* when it has a *trash_at* time in the future.
  An expiring collection can be accessed as normal,
  but is scheduled to be trashed automatically at the *trash_at* time.

:ensure_unique_name:
  If True, ask the API server to rename the collection
  if it conflicts with a collection with the same name and owner.  If
  False, a name conflict will result in an error.

:num_retries:
  Retry count on API calls (if None,  use the collection default)
Decorators:
  • @must_be_writable
  • @synchronized
  • @retry_method

_import_manifest(self, manifest_text)

source code  
Import a manifest into a `Collection`.

:manifest_text:
  The manifest text to import from.
Decorators:
  • @synchronized

notify(self, event, collection, name, item)

source code 
Decorators:
  • @synchronized
Overrides: RichCollectionBase.notify

   Home       Trees       Indices       Help   
Generated by Epydoc 3.0.1 on Thu Dec 28 19:40:39 2023 http://epydoc.sourceforge.net