ray.tune.syncer._BackgroundSyncer#

class ray.tune.syncer._BackgroundSyncer(sync_period: float = 300, sync_timeout: float = 1800)[source]#

Bases: ray.tune.syncer.Syncer

Syncer using a background process for asynchronous file transfer.

sync_up(local_dir: str, remote_dir: str, exclude: Optional[List] = None) → bool[source]#

Synchronize local directory to remote directory.

This function can spawn an asynchronous process that can be awaited in wait().

Parameters

  • local_dir – Local directory to sync from.

  • remote_dir – Remote directory to sync up to. This is an URI (protocol://remote/path).

  • exclude – Pattern of files to exclude, e.g. ["*/checkpoint_*] to exclude trial checkpoints.

Returns

True if sync process has been spawned, False otherwise.

sync_down(remote_dir: str, local_dir: str, exclude: Optional[List] = None) → bool[source]#

Synchronize remote directory to local directory.

This function can spawn an asynchronous process that can be awaited in wait().

Parameters

  • remote_dir – Remote directory to sync down from. This is an URI (protocol://remote/path).

  • local_dir – Local directory to sync to.

  • exclude – Pattern of files to exclude, e.g. ["*/checkpoint_*] to exclude trial checkpoints.

Returns

True if sync process has been spawned, False otherwise.

delete(remote_dir: str) → bool[source]#

Delete directory on remote storage.

This function can spawn an asynchronous process that can be awaited in wait().

Parameters

remote_dir – Remote directory to delete. This is an URI (protocol://remote/path).

Returns

True if sync process has been spawned, False otherwise.

wait()[source]#

Wait for asynchronous sync command to finish.

You should implement this method if you spawn asynchronous syncing processes. This method should timeout after the asynchronous command has run for sync_timeout seconds and raise a TimeoutError.

retry()[source]#

Retry the last sync up, sync down, or delete command.

You should implement this method if you spawn asynchronous syncing processes.