Home       Trees       Indices       Help   
Package arvados :: Module collection :: Class RichCollectionBase
[hide private]
[frames] | no frames]

Class RichCollectionBase

source code

builtins.object --+    
                  |    
     CollectionBase --+
                      |
                     RichCollectionBase

Base class for Collections and Subcollections.

Implements the majority of functionality relating to accessing items in the Collection.

Instance Methods [hide private]
 
__init__(self, parent=None) source code
 
_my_api(self) source code
 
_my_keep(self) source code
 
_my_block_manager(self) source code
 
writable(self) source code
 
root_collection(self) source code
 
stream_name(self) source code
 
has_remote_blocks(self)
Recursively check for a +R segment locator signature.		 source code
 
set_has_remote_blocks(self, val) source code
 
find_or_create(self, path, create_type)
Recursively search the specified file path.		 source code
 
find(self, path)
Recursively search the specified file path.		 source code
 
mkdirs(self, path)
Recursive subcollection create.		 source code
 
open(self, path, mode="r", encoding=None)
Open a file-like object for access.		 source code
 
modified(self)
Determine if the collection has been modified since last commited.		 source code
 
committed(self)
Determine if the collection has been committed to the API server.		 source code
 
set_committed(self, value=True)
Recursively set committed flag.		 source code
 
__iter__(self)
Iterate over names of files and collections contained in this collection.		 source code
 
__getitem__(self, k)
Get a file or collection that is directly contained by this collection.		 source code
 
__contains__(self, k)
Test if there is a file or collection a directly contained by this collection.		 source code
 
__len__(self)
Get the number of items directly contained in this collection.		 source code
 
__delitem__(self, p)
Delete an item by name which is directly contained by this collection.		 source code
 
keys(self)
Get a list of names of files and collections directly contained in this collection.		 source code
 
values(self)
Get a list of files and collection objects directly contained in this collection.		 source code
 
items(self)
Get a list of (name, object) tuples directly contained in this collection.		 source code
 
exists(self, path)
Test if there is a file or collection at `path`.		 source code
 
remove(self, path, recursive=False)
Remove the file or subcollection (directory) at `path`.		 source code
 
_clonefrom(self, source) source code
 
clone(self) source code
 
add(self, source_obj, target_name, overwrite=False, reparent=False)
Copy or move a file or subcollection to this collection.		 source code
 
_get_src_target(self, source, target_path, source_collection, create_dest) source code
 
copy(self, source, target_path, source_collection=None, overwrite=False)
Copy a file or subcollection to a new path in this collection.		 source code
 
rename(self, source, target_path, source_collection=None, overwrite=False)
Move a file or subcollection from `source_collection` to a new path in this collection.		 source code
 
portable_manifest_text(self, stream_name=".")
Get the manifest text for this collection, sub collections and files.		 source code
 
manifest_text(self, stream_name=".", strip=False, normalize=False, only_committed=False)
Get the manifest text for this collection, sub collections and files.		 source code
 
_get_manifest_text(self, stream_name, strip, normalize, only_committed=False)
Get the manifest text for this collection, sub collections and files.		 source code
 
_copy_remote_blocks(self, remote_blocks={})
Scan through the entire collection and ask Keep to copy remote blocks.		 source code
 
diff(self, end_collection, prefix=".", holding_collection=None)
Generate list of add/modify/delete actions.		 source code
 
apply(self, changes)
Apply changes from `diff`.		 source code
 
portable_data_hash(self)
Get the portable data hash for this collection's manifest.		 source code
 
subscribe(self, callback) source code
 
unsubscribe(self) source code
 
notify(self, event, collection, name, item) source code
 
__eq__(self, other) source code
 
__ne__(self, other) source code
 
flush(self)
Flush bufferblocks to Keep.		 source code

Inherited from CollectionBase: __enter__, __exit__, stripped_manifest

Method Details [hide private]

_my_keep(self)

source code 
Overrides: CollectionBase._my_keep

has_remote_blocks(self)

source code 

Recursively check for a +R segment locator signature.

Decorators:
  • @synchronized

set_has_remote_blocks(self, val)

source code 
Decorators:
  • @synchronized

find_or_create(self, path, create_type)

source code  
Recursively search the specified file path.

May return either a `Collection` or `ArvadosFile`.  If not found, will
create a new item at the specified path based on `create_type`.  Will
create intermediate subcollections needed to contain the final item in
the path.

:create_type:
  One of `arvados.collection.FILE` or
  `arvados.collection.COLLECTION`.  If the path is not found, and value
  of create_type is FILE then create and return a new ArvadosFile for
  the last path component.  If COLLECTION, then create and return a new
  Collection for the last path component.
Decorators:
  • @must_be_writable
  • @synchronized

find(self, path)

source code 

Recursively search the specified file path.

May return either a Collection or ArvadosFile. Return None if not found. If path is invalid (ex: starts with '/'), an IOError exception will be raised.

Decorators:
  • @synchronized

mkdirs(self, path)

source code 

Recursive subcollection create.

Like `os.makedirs()`. Will create intermediate subcollections needed to contain the leaf subcollection path.

Decorators:
  • @synchronized

open(self, path, mode="r", encoding=None)

source code  
Open a file-like object for access.

:path:
  path to a file in the collection
:mode:
  a string consisting of "r", "w", or "a", optionally followed
  by "b" or "t", optionally followed by "+".
  :"b":
    binary mode: write() accepts bytes, read() returns bytes.
  :"t":
    text mode (default): write() accepts strings, read() returns strings.
  :"r":
    opens for reading
  :"r+":
    opens for reading and writing.  Reads/writes share a file pointer.
  :"w", "w+":
    truncates to 0 and opens for reading and writing.  Reads/writes share a file pointer.
  :"a", "a+":
    opens for reading and writing.  All writes are appended to
    the end of the file.  Writing does not affect the file pointer for
    reading.

committed(self)

source code 

Determine if the collection has been committed to the API server.

Decorators:
  • @synchronized

set_committed(self, value=True)

source code 

Recursively set committed flag.

If value is True, set committed to be True for this and all children.

If value is False, set committed to be False for this and all parents.

Decorators:
  • @synchronized

__iter__(self)

source code 

Iterate over names of files and collections contained in this collection.

Decorators:
  • @synchronized

__getitem__(self, k)
(Indexing operator)

source code 

Get a file or collection that is directly contained by this collection.

If you want to search a path, use `find()` instead.

Decorators:
  • @synchronized

__contains__(self, k)
(In operator)

source code 

Test if there is a file or collection a directly contained by this collection.

Decorators:
  • @synchronized

__len__(self)
(Length operator)

source code 

Get the number of items directly contained in this collection.

Decorators:
  • @synchronized

__delitem__(self, p)
(Index deletion operator)

source code 

Delete an item by name which is directly contained by this collection.

Decorators:
  • @must_be_writable
  • @synchronized

keys(self)

source code 

Get a list of names of files and collections directly contained in this collection.

Decorators:
  • @synchronized

values(self)

source code 

Get a list of files and collection objects directly contained in this collection.

Decorators:
  • @synchronized

items(self)

source code 

Get a list of (name, object) tuples directly contained in this collection.

Decorators:
  • @synchronized

remove(self, path, recursive=False)

source code  
Remove the file or subcollection (directory) at `path`.

:recursive:
  Specify whether to remove non-empty subcollections (True), or raise an error (False).
Decorators:
  • @must_be_writable
  • @synchronized

add(self, source_obj, target_name, overwrite=False, reparent=False)

source code  
Copy or move a file or subcollection to this collection.

:source_obj:
  An ArvadosFile, or Subcollection object

:target_name:
  Destination item name.  If the target name already exists and is a
  file, this will raise an error unless you specify `overwrite=True`.

:overwrite:
  Whether to overwrite target file if it already exists.

:reparent:
  If True, source_obj will be moved from its parent collection to this collection.
  If False, source_obj will be copied and the parent collection will be
  unmodified.
Decorators:
  • @must_be_writable
  • @synchronized

copy(self, source, target_path, source_collection=None, overwrite=False)

source code  
Copy a file or subcollection to a new path in this collection.

:source:
  A string with a path to source file or subcollection, or an actual ArvadosFile or Subcollection object.

:target_path:
  Destination file or path.  If the target path already exists and is a
  subcollection, the item will be placed inside the subcollection.  If
  the target path already exists and is a file, this will raise an error
  unless you specify `overwrite=True`.

:source_collection:
  Collection to copy `source_path` from (default `self`)

:overwrite:
  Whether to overwrite target file if it already exists.
Decorators:
  • @must_be_writable
  • @synchronized

rename(self, source, target_path, source_collection=None, overwrite=False)

source code  
Move a file or subcollection from `source_collection` to a new path in this collection.

:source:
  A string with a path to source file or subcollection.

:target_path:
  Destination file or path.  If the target path already exists and is a
  subcollection, the item will be placed inside the subcollection.  If
  the target path already exists and is a file, this will raise an error
  unless you specify `overwrite=True`.

:source_collection:
  Collection to copy `source_path` from (default `self`)

:overwrite:
  Whether to overwrite target file if it already exists.
Decorators:
  • @must_be_writable
  • @synchronized

portable_manifest_text(self, stream_name=".")

source code  
Get the manifest text for this collection, sub collections and files.

This method does not flush outstanding blocks to Keep.  It will return
a normalized manifest with access tokens stripped.

:stream_name:
  Name to use for this stream (directory)

manifest_text(self, stream_name=".", strip=False, normalize=False, only_committed=False)

source code  
Get the manifest text for this collection, sub collections and files.

This method will flush outstanding blocks to Keep.  By default, it will
not normalize an unmodified manifest or strip access tokens.

:stream_name:
  Name to use for this stream (directory)

:strip:
  If True, remove signing tokens from block locators if present.
  If False (default), block locators are left unchanged.

:normalize:
  If True, always export the manifest text in normalized form
  even if the Collection is not modified.  If False (default) and the collection
  is not modified, return the original manifest text even if it is not
  in normalized form.

:only_committed:
  If True, don't commit pending blocks.
Decorators:
  • @synchronized

_get_manifest_text(self, stream_name, strip, normalize, only_committed=False)

source code  
Get the manifest text for this collection, sub collections and files.

:stream_name:
  Name to use for this stream (directory)

:strip:
  If True, remove signing tokens from block locators if present.
  If False (default), block locators are left unchanged.

:normalize:
  If True, always export the manifest text in normalized form
  even if the Collection is not modified.  If False (default) and the collection
  is not modified, return the original manifest text even if it is not
  in normalized form.

:only_committed:
  If True, only include blocks that were already committed to Keep.
Decorators:
  • @synchronized

_copy_remote_blocks(self, remote_blocks={})

source code  
Scan through the entire collection and ask Keep to copy remote blocks.

When accessing a remote collection, blocks will have a remote signature
(+R instead of +A). Collect these signatures and request Keep to copy the
blocks to the local cluster, returning local (+A) signatures.

:remote_blocks:
  Shared cache of remote to local block mappings. This is used to avoid
  doing extra work when blocks are shared by more than one file in
  different subdirectories.
Decorators:
  • @synchronized

diff(self, end_collection, prefix=".", holding_collection=None)

source code 

Generate list of add/modify/delete actions.

When given to `apply`, will change `self` to match `end_collection`

Decorators:
  • @synchronized

apply(self, changes)

source code 

Apply changes from `diff`.

If a change conflicts with a local change, it will be saved to an alternate path indicating the conflict.

Decorators:
  • @must_be_writable
  • @synchronized

subscribe(self, callback)

source code 
Decorators:
  • @synchronized

unsubscribe(self)

source code 
Decorators:
  • @synchronized

notify(self, event, collection, name, item)

source code 
Decorators:
  • @synchronized

__eq__(self, other)
(Equality operator)

source code 
Decorators:
  • @synchronized

flush(self)

source code 

Flush bufferblocks to Keep.

Decorators:
  • @synchronized

   Home       Trees       Indices       Help   
Generated by Epydoc 3.0.1 on Thu Dec 21 19:09:45 2023 http://epydoc.sourceforge.net