ray.train.tensorflow.TensorflowTrainer.restore#

classmethod TensorflowTrainer.restore(path: str, train_loop_per_worker: Callable[[], None] | Callable[[Dict], None] | None = None, train_loop_config: Dict | None = None, **kwargs) DataParallelTrainer#

Restores a DataParallelTrainer from a previously interrupted/failed run.

Parameters:
  • train_loop_per_worker – Optionally re-specified train loop function. This should be used to re-specify a function that is not restorable in a new Ray cluster (e.g., it holds onto outdated object references). This should be the same training loop that was passed to the original trainer constructor.

  • train_loop_config – Optionally re-specified train config. This should similarly be used if the original train_loop_config contained outdated object references, and it should not be modified from what was originally passed in.

See BaseTrainer.restore() for descriptions of the other arguments.

Returns:

A restored instance of the DataParallelTrainer subclass that is calling this method.

Return type:

DataParallelTrainer