ray.tune.schedulers.HyperBandScheduler#

class ray.tune.schedulers.HyperBandScheduler(time_attr: str = 'training_iteration', metric: Optional[str] = None, mode: Optional[str] = None, max_t: int = 81, reduction_factor: float = 3, stop_last_trials: bool = True)[source]#

Bases: ray.tune.schedulers.trial_scheduler.FIFOScheduler

Implements the HyperBand early stopping algorithm.

HyperBandScheduler early stops trials using the HyperBand optimization algorithm. It divides trials into brackets of varying sizes, and periodically early stops low-performing trials within each bracket.

To use this implementation of HyperBand with Tune, all you need to do is specify the max length of time a trial can run max_t, the time units time_attr, the name of the reported objective value metric, and if metric is to be maximized or minimized (mode). We automatically determine reasonable values for the other HyperBand parameters based on the given values.

For example, to limit trials to 10 minutes and early stop based on the episode_mean_reward attr, construct:

HyperBand('time_total_s', 'episode_reward_mean', max_t=600)

Note that Tune’s stopping criteria will be applied in conjunction with HyperBand’s early stopping mechanisms.

See also: https://blog.ml.cmu.edu/2018/12/12/massively-parallel-hyperparameter-optimization/

Parameters

  • time_attr – The training result attr to use for comparing time. Note that you can pass in something non-temporal such as training_iteration as a measure of progress, the only requirement is that the attribute should increase monotonically.

  • metric – The training result objective value attribute. Stopping procedures will use this attribute. If None but a mode was passed, the ray.tune.result.DEFAULT_METRIC will be used per default.

  • mode – One of {min, max}. Determines whether objective is minimizing or maximizing the metric attribute.

  • max_t – max time units per trial. Trials will be stopped after max_t time units (determined by time_attr) have passed. The scheduler will terminate trials after this time has passed. Note that this is different from the semantics of max_t as mentioned in the original HyperBand paper.

  • reduction_factor – Same as eta. Determines how sharp the difference is between bracket space-time allocation ratios.

  • stop_last_trials – Whether to terminate the trials after reaching max_t. Defaults to True.

PublicAPI: This API is stable across Ray releases.

Methods

choose_trial_to_run(tune_controller)

Fair scheduling within iteration by completion percentage.

debug_string()

This provides a progress notification for the algorithm.

on_trial_add(tune_controller, trial)

Adds new trial.

on_trial_complete(tune_controller, trial, result)

Cleans up trial info from bracket if trial completed early.

on_trial_error(tune_controller, trial)

Cleans up trial info from bracket if trial errored early.

on_trial_remove(tune_controller, trial)

Notification when trial terminates.

on_trial_result(tune_controller, trial, result)

If bracket is finished, all trials will be stopped.

restore(checkpoint_path)

Restore trial scheduler from checkpoint.

save(checkpoint_path)

Save trial scheduler to a checkpoint

Attributes

CONTINUE

Status for continuing trial execution

NOOP

PAUSE

Status for pausing trial execution

STOP

Status for stopping trial execution

metric

supports_buffered_results