Package arvados :: Module arvfile :: Class ArvadosFile
[hide private]
[frames] | no frames]

Class ArvadosFile

source code

builtins.object --+
                  |
                 ArvadosFile

Represent a file in a Collection.

ArvadosFile manages the underlying representation of a file in Keep as a sequence of segments spanning a set of blocks, and implements random read/write access.

This object may be accessed from multiple threads.

Instance Methods [hide private]
 
__init__(self, parent, name, stream=[], segments=[])
ArvadosFile constructor.
source code
 
writable(self) source code
 
permission_expired(self, as_of_dt=None)
Returns True if any of the segment's locators is expired
source code
 
has_remote_blocks(self)
Returns True if any of the segment's locators has a +R signature
source code
 
_copy_remote_blocks(self, remote_blocks={})
Ask Keep to copy remote blocks and point to their local copies.
source code
 
segments(self) source code
 
clone(self, new_parent, new_name)
Make a copy of this file.
source code
 
replace_contents(self, other)
Replace segments of this file with segments from another `ArvadosFile` object.
source code
 
__eq__(self, other) source code
 
__ne__(self, other) source code
 
set_segments(self, segs) source code
 
set_committed(self, value=True)
Set committed flag.
source code
 
committed(self)
Get whether this is committed or not.
source code
 
add_writer(self, writer)
Add an ArvadosFileWriter reference to the list of writers
source code
 
remove_writer(self, writer, flush)
Called from ArvadosFileWriter.close().
source code
 
closed(self)
Get whether this is closed or not.
source code
 
truncate(self, size)
Shrink or expand the size of the file.
source code
 
readfrom(self, offset, size, num_retries, exact=False)
Read up to `size` bytes from the file starting at `offset`.
source code
 
writeto(self, offset, data, num_retries)
Write `data` to the file starting at `offset`.
source code
 
flush(self, sync=True, num_retries=0)
Flush the current bufferblock to Keep.
source code
 
add_segment(self, blocks, pos, size)
Add a segment to the end of the file.
source code
 
_add_segment(self, blocks, pos, size)
Internal implementation of add_segment.
source code
 
size(self)
Get the file size.
source code
 
manifest_text(self, stream_name=".", portable_locators=False, normalize=False, only_committed=False) source code
 
_reparent(self, newparent, newname) source code
Method Details [hide private]

__init__(self, parent, name, stream=[], segments=[])
(Constructor)

source code 

ArvadosFile constructor.

:stream:
  a list of Range objects representing a block stream

:segments:
  a list of Range objects representing segments

permission_expired(self, as_of_dt=None)

source code 

Returns True if any of the segment's locators is expired

Decorators:
  • @synchronized

has_remote_blocks(self)

source code 

Returns True if any of the segment's locators has a +R signature

Decorators:
  • @synchronized

_copy_remote_blocks(self, remote_blocks={})

source code 
Ask Keep to copy remote blocks and point to their local copies.

This is called from the parent Collection.

:remote_blocks:
    Shared cache of remote to local block mappings. This is used to avoid
    doing extra work when blocks are shared by more than one file in
    different subdirectories.

Decorators:
  • @synchronized

segments(self)

source code 
Decorators:
  • @synchronized

clone(self, new_parent, new_name)

source code 

Make a copy of this file.

Decorators:
  • @synchronized

replace_contents(self, other)

source code 

Replace segments of this file with segments from another `ArvadosFile` object.

Decorators:
  • @must_be_writable
  • @synchronized

set_segments(self, segs)

source code 
Decorators:
  • @synchronized

set_committed(self, value=True)

source code 

Set committed flag.

If value is True, set committed to be True.

If value is False, set committed to be False for this and all parents.

Decorators:
  • @synchronized

committed(self)

source code 

Get whether this is committed or not.

Decorators:
  • @synchronized

add_writer(self, writer)

source code 

Add an ArvadosFileWriter reference to the list of writers

Decorators:
  • @synchronized

remove_writer(self, writer, flush)

source code 

Called from ArvadosFileWriter.close(). Remove a writer reference from the list and do some block maintenance tasks.

Decorators:
  • @synchronized

closed(self)

source code 

Get whether this is closed or not. When the writers list is empty, the file is supposed to be closed.

truncate(self, size)

source code 

Shrink or expand the size of the file.

If `size` is less than the size of the file, the file contents after `size` will be discarded. If `size` is greater than the current size of the file, it will be filled with zero bytes.

Decorators:
  • @must_be_writable
  • @synchronized

readfrom(self, offset, size, num_retries, exact=False)

source code 
Read up to `size` bytes from the file starting at `offset`.

:exact:
 If False (default), return less data than requested if the read
 crosses a block boundary and the next block isn't cached.  If True,
 only return less data than requested when hitting EOF.

writeto(self, offset, data, num_retries)

source code 

Write `data` to the file starting at `offset`.

This will update existing bytes and/or extend the size of the file as necessary.

Decorators:
  • @must_be_writable
  • @synchronized

flush(self, sync=True, num_retries=0)

source code 
Flush the current bufferblock to Keep.

:sync:
  If True, commit block synchronously, wait until buffer block has been written.
  If False, commit block asynchronously, return immediately after putting block into
  the keep put queue.

Decorators:
  • @synchronized

add_segment(self, blocks, pos, size)

source code 

Add a segment to the end of the file.

`pos` and `offset` reference a section of the stream described by `blocks` (a list of Range objects)

Decorators:
  • @must_be_writable
  • @synchronized

size(self)

source code 

Get the file size.

Decorators:
  • @synchronized

manifest_text(self, stream_name=".", portable_locators=False, normalize=False, only_committed=False)

source code 
Decorators:
  • @synchronized

_reparent(self, newparent, newname)

source code 
Decorators:
  • @must_be_writable
  • @synchronized