ray.tune.schedulers.HyperBandScheduler#

class ray.tune.schedulers.HyperBandScheduler(time_attr: str = 'training_iteration', metric: str | None = None, mode: str | None = None, max_t: int = 81, reduction_factor: float = 3, stop_last_trials: bool = True)[source]#

Bases: FIFOScheduler

Implements the HyperBand early stopping algorithm.

HyperBandScheduler early stops trials using the HyperBand optimization algorithm. It divides trials into brackets of varying sizes, and periodically early stops low-performing trials within each bracket.

To use this implementation of HyperBand with Tune, all you need to do is specify the max length of time a trial can run max_t, the time units time_attr, the name of the reported objective value metric, and if metric is to be maximized or minimized (mode). We automatically determine reasonable values for the other HyperBand parameters based on the given values.

For example, to limit trials to 10 minutes and early stop based on the episode_mean_reward attr, construct:

HyperBand('time_total_s', 'episode_reward_mean', max_t=600)

Note that Tune’s stopping criteria will be applied in conjunction with HyperBand’s early stopping mechanisms.

See also: https://blog.ml.cmu.edu/2018/12/12/massively-parallel-hyperparameter-optimization/

Parameters:
  • time_attr – The training result attr to use for comparing time. Note that you can pass in something non-temporal such as training_iteration as a measure of progress, the only requirement is that the attribute should increase monotonically.

  • metric – The training result objective value attribute. Stopping procedures will use this attribute. If None but a mode was passed, the ray.tune.result.DEFAULT_METRIC will be used per default.

  • mode – One of {min, max}. Determines whether objective is minimizing or maximizing the metric attribute.

  • max_t – max time units per trial. Trials will be stopped after max_t time units (determined by time_attr) have passed. The scheduler will terminate trials after this time has passed. Note that this is different from the semantics of max_t as mentioned in the original HyperBand paper.

  • reduction_factor – Same as eta. Determines how sharp the difference is between bracket space-time allocation ratios.

  • stop_last_trials – Whether to terminate the trials after reaching max_t. Defaults to True.

Methods

choose_trial_to_run

Fair scheduling within iteration by completion percentage.

debug_string

This provides a progress notification for the algorithm.

on_trial_add

Adds new trial.

on_trial_complete

Cleans up trial info from bracket if trial completed early.

on_trial_error

Cleans up trial info from bracket if trial errored early.

on_trial_remove

Notification when trial terminates.

on_trial_result

If bracket is finished, all trials will be stopped.

restore

Restore trial scheduler from checkpoint.

save

Save trial scheduler to a checkpoint

Attributes

CONTINUE

Status for continuing trial execution

NOOP

PAUSE

Status for pausing trial execution

STOP

Status for stopping trial execution

metric

supports_buffered_results