3.6. Replica Management

The replica management module provides an interface to (distributed) data replication services, like for example iRODS.

The basic usage of the replica module is as follows:

myfile = radical.saga.replica.LogicalFile("irods://localhost/"+TEMP_FILENAME)
myfile.add_location("irods:////data/cache/AGLT2_CE_2_FTPplaceholder/whatever?resource=AGLT2_CE_2_FTP")

mydir = radical.saga.replica.LogicalDirectory("irods://localhost/" + IRODS_DIRECTORY)
mydir.make_dir("anotherdir")

Like all SAGA modules, the replica module relies on middleware adaptors to provide bindings to a specific resource manager. Adaptors are implicitly selected via the scheme part of the URL, e.g., local:// in the example above selects the local replica adaptor.

Note

A list of available adaptors and supported resource managers can be found in the Developer Documentation part of this documentation.

The rest of this section is structured as follows:

3.6.1. Flags

The following constants are defined as valid flags for logical file and directory methods:

radical.saga.filesystem.OVERWRITE

radical.saga.filesystem.RECURSIVE

radical.saga.filesystem.CREATE

radical.saga.filesystem.CREATE_PARENTS

radical.saga.filesystem.LOCK

radical.saga.filesystem.EXCLUSIVE

radical.saga.filesystem.DEREFERENCE

3.6.2. Logical File – radical.saga.replica.LogicalFile

class radical.saga.replica.LogicalFile(url=None, flags=512, session=None, _adaptor=None, _adaptor_state={}, _ttype=None)

__init__(url=None, flags=READ, session=None)

url: saga.Url flags: flags enum session: saga.Session ret: obj

add_location(name)

Add a physical location.

name: saga.Url ttype: saga.task.type enum ret: None / saga.Task

download(name, src=None, flags=None)

Download a physical file.

name: saga.Url src: saga.Url flags: flags enum ttype: saga.task.type enum ret: None / saga.Task

get_size()

Return the size of the file.

ttype: saga.task.type enum ret: int / saga.Task

Returns the size of the physical file represented by this logical file (in bytes)

Example:

# get a file handle
lf = saga.replica.LogicalFile("irods://localhost/tmp/data.bin")

# print the logical file's size
print(lf.get_size ())

is_file()

ttype: saga.task.type enum ret: bool / saga.Task

list_locations()

List all physical locations of a logical file.

ttype: saga.task.type enum ret: list [saga.Url] / saga.Task

remove_location(name)

Remove a physical location.

name: saga.Url ttype: saga.task.type enum ret: None / saga.Task

replicate(name)

Replicate a logical file.

name: saga.Url flags: flags enum ttype: saga.task.type enum ret: None / saga.Task

update_location(old, new)

Updates a physical location.

old: saga.Url new: saga.Url ttype: saga.task.type enum ret: None / saga.Task

upload(name, tgt=None, flags=None)

Upload a physical file.

name: saga.Url tgt: saga.Url flags: flags enum ttype: saga.task.type enum ret: None / saga.Task

3.6.3. Logical Directory – radical.saga.replica.LogicalDirectory

class radical.saga.replica.LogicalDirectory(url=None, flags=512, session=None, _adaptor=None, _adaptor_state={}, _ttype=None)

__init__(url, flags=READ, session=None)

Create a new Logical Directory instance.

url: saga.Url flags: flags enum session: saga.Session ret: obj

find(name_pattern, attr_pattern=None, flags=RECURSIVE)

name_pattern: string attr_pattern: string flags: flags enum ttype: saga.task.type enum ret: list [saga.Url] / saga.Task

get_size(tgt)

tgt: logical file to get size for ttype: saga.task.type enum ret: int / saga.Task

Returns the size of the physical file represented by the given logical file (in bytes)

Example:

# get a logical directory handle
lf = saga.replica.LogicalFile("irods://localhost/tmp/data/")

# print a logical file's size
print(lf.get_size ('data.dat'))

is_file(tgt=None)

tgt: saga.Url / string ttype: saga.task.type enum ret: bool / saga.Task

open(tgt, flags=READ)

tgt: saga.Url flags: saga.namespace.flags enum ttype: saga.task.type enum ret: saga.namespace.Entry / saga.Task

open_dir(tgt, flags=READ)

Parameters:

• tgt – name/path of the directory to open

• flags – directory creation flags

ttype: saga.task.type enum ret: saga.namespace.Directory / saga.Task

Open and return a new directoy

The call opens and returns a directory at the given location.

Example:

# create a subdir 'data' in /tmp
dir = saga.namespace.Directory("sftp://localhost/tmp/")
data = dir.open_dir ('data/', saga.namespace.Create)