get_preemption_info

ray.train.get_preemption_info() → PreemptionInfo | None

Return the imminent preemption info for the current worker, or None.

Returns None until a node hosting one of the workers is being preempted (e.g. a spot instance reclaim). The recommended reaction is to save a just-in-time checkpoint and keep training: when the node is actually preempted, Ray Train restarts the run and resumes it from the latest checkpoint, retrying against FailureConfig.max_preemption_failures. A run that returns cleanly always finishes, whether or not a preemption is in progress.

Example

import ray.train

def train_func(config):
    for step in range(config["total_steps"]):
        if ray.train.get_preemption_info() is not None:
            ray.train.report(metrics, checkpoint=checkpoint)
        # ... normal training step (with your usual periodic
        # checkpointing) ...

Returns:

A PreemptionInfo with the affected node ids / world ranks and the reclaim deadline, or None if no preemption has been detected.

PublicAPI (alpha): This API is in alpha and may change before becoming stable.