3.6. Replica Management¶
The replica management module provides an interface to (distributed) data replication services, like for example iRODS.
The basic usage of the replica module is as follows:
myfile = radical.saga.replica.LogicalFile("irods://localhost/"+TEMP_FILENAME)
myfile.add_location("irods:////data/cache/AGLT2_CE_2_FTPplaceholder/whatever?resource=AGLT2_CE_2_FTP")
mydir = radical.saga.replica.LogicalDirectory("irods://localhost/" + IRODS_DIRECTORY)
mydir.make_dir("anotherdir")
Like all SAGA modules, the replica module relies on middleware adaptors
to provide bindings to a specific resource manager. Adaptors are implicitly
selected via the scheme part of the URL, e.g., local://
in the example
above selects the local replica adaptor.
Note
A list of available adaptors and supported resource managers can be found in the Developer Documentation part of this documentation.
The rest of this section is structured as follows:
3.6.1. Flags¶
The following constants are defined as valid flags for logical file and directory methods:
- radical.saga.filesystem.OVERWRITE¶
- radical.saga.filesystem.RECURSIVE¶
- radical.saga.filesystem.CREATE¶
- radical.saga.filesystem.CREATE_PARENTS¶
- radical.saga.filesystem.LOCK¶
- radical.saga.filesystem.EXCLUSIVE¶
- radical.saga.filesystem.DEREFERENCE¶
3.6.2. Logical File – radical.saga.replica.LogicalFile
¶
- class radical.saga.replica.LogicalFile(url=None, flags=512, session=None, _adaptor=None, _adaptor_state={}, _ttype=None)[source]¶
- __init__(url=None, flags=READ, session=None)[source]¶
url: saga.Url flags: flags enum session: saga.Session ret: obj
- add_location(name)[source]¶
Add a physical location.
name: saga.Url ttype: saga.task.type enum ret: None / saga.Task
- download(name, src=None, flags=None)[source]¶
Download a physical file.
name: saga.Url src: saga.Url flags: flags enum ttype: saga.task.type enum ret: None / saga.Task
- get_size()[source]¶
Return the size of the file.
ttype: saga.task.type enum ret: int / saga.Task
Returns the size of the physical file represented by this logical file (in bytes)
Example:
# get a file handle lf = saga.replica.LogicalFile("irods://localhost/tmp/data.bin") # print the logical file's size print(lf.get_size ())
- list_locations()[source]¶
List all physical locations of a logical file.
ttype: saga.task.type enum ret: list [saga.Url] / saga.Task
- remove_location(name)[source]¶
Remove a physical location.
name: saga.Url ttype: saga.task.type enum ret: None / saga.Task
- replicate(name)[source]¶
Replicate a logical file.
name: saga.Url flags: flags enum ttype: saga.task.type enum ret: None / saga.Task
3.6.3. Logical Directory – radical.saga.replica.LogicalDirectory
¶
- class radical.saga.replica.LogicalDirectory(url=None, flags=512, session=None, _adaptor=None, _adaptor_state={}, _ttype=None)[source]¶
- __init__(url, flags=READ, session=None)[source]¶
Create a new Logical Directory instance.
url: saga.Url flags: flags enum session: saga.Session ret: obj
- find(name_pattern, attr_pattern=None, flags=RECURSIVE)[source]¶
name_pattern: string attr_pattern: string flags: flags enum ttype: saga.task.type enum ret: list [saga.Url] / saga.Task
- get_size(tgt)[source]¶
tgt: logical file to get size for ttype: saga.task.type enum ret: int / saga.Task
Returns the size of the physical file represented by the given logical file (in bytes)
Example:
# get a logical directory handle lf = saga.replica.LogicalFile("irods://localhost/tmp/data/") # print a logical file's size print(lf.get_size ('data.dat'))
- open(tgt, flags=READ)[source]¶
tgt: saga.Url flags: saga.namespace.flags enum ttype: saga.task.type enum ret: saga.namespace.Entry / saga.Task
- open_dir(tgt, flags=READ)[source]¶
- Parameters:
tgt – name/path of the directory to open
flags – directory creation flags
ttype: saga.task.type enum ret: saga.namespace.Directory / saga.Task
Open and return a new directoy
The call opens and returns a directory at the given location.
Example:
# create a subdir 'data' in /tmp dir = saga.namespace.Directory("sftp://localhost/tmp/") data = dir.open_dir ('data/', saga.namespace.Create)