ray.train.tensorflow.TensorflowTrainer.restore

classmethod TensorflowTrainer.restore(path: str, train_loop_per_worker: Callable[[], None] | Callable[[Dict], None] | None = None, train_loop_config: Dict | None = None, **kwargs) → DataParallelTrainer

Restores a DataParallelTrainer from a previously interrupted/failed run.

Parameters:

• train_loop_per_worker – Optionally re-specified train loop function. This should be used to re-specify a function that is not restorable in a new Ray cluster (e.g., it holds onto outdated object references). This should be the same training loop that was passed to the original trainer constructor.

• train_loop_config – Optionally re-specified train config. This should similarly be used if the original train_loop_config contained outdated object references, and it should not be modified from what was originally passed in.

See BaseTrainer.restore() for descriptions of the other arguments.

Returns:

A restored instance of the DataParallelTrainer subclass that is calling this method.

Return type:

DataParallelTrainer