ray.tune.search.hyperopt.HyperOptSearch#

class ray.tune.search.hyperopt.HyperOptSearch(space: Dict | None = None, metric: str | None = None, mode: str | None = None, points_to_evaluate: List[Dict] | None = None, n_initial_points: int = 20, random_state_seed: int | None = None, gamma: float = 0.25)[source]#

Bases: Searcher

A wrapper around HyperOpt to provide trial suggestions.

HyperOpt a Python library for serial and parallel optimization over awkward search spaces, which may include real-valued, discrete, and conditional dimensions. More info can be found at http://hyperopt.github.io/hyperopt.

HyperOptSearch uses the Tree-structured Parzen Estimators algorithm, though it can be trivially extended to support any algorithm HyperOpt supports.

To use this search algorithm, you will need to install HyperOpt:

pip install -U hyperopt
Parameters:
  • space – HyperOpt configuration. Parameters will be sampled from this configuration and will be used to override parameters generated in the variant generation process.

  • metric – The training result objective value attribute. If None but a mode was passed, the anonymous metric _metric will be used per default.

  • mode – One of {min, max}. Determines whether objective is minimizing or maximizing the metric attribute.

  • points_to_evaluate – Initial parameter suggestions to be run first. This is for when you already have some good parameters you want to run first to help the algorithm make better suggestions for future parameters. Needs to be a list of dicts containing the configurations.

  • n_initial_points – number of random evaluations of the objective function before starting to aproximate it with tree parzen estimators. Defaults to 20.

  • random_state_seed – seed for reproducible results. Defaults to None.

  • gamma – parameter governing the tree parzen estimators suggestion algorithm. Defaults to 0.25.

Tune automatically converts search spaces to HyperOpt’s format:

config = {
    'width': tune.uniform(0, 20),
    'height': tune.uniform(-100, 100),
    'activation': tune.choice(["relu", "tanh"])
}

current_best_params = [{
    'width': 10,
    'height': 0,
    'activation': "relu",
}]

hyperopt_search = HyperOptSearch(
    metric="mean_loss", mode="min",
    points_to_evaluate=current_best_params)

tuner = tune.Tuner(
    trainable,
    tune_config=tune.TuneConfig(
        search_alg=hyperopt_search
    ),
    param_space=config
)
tuner.fit()

If you would like to pass the search space manually, the code would look like this:

space = {
    'width': hp.uniform('width', 0, 20),
    'height': hp.uniform('height', -100, 100),
    'activation': hp.choice("activation", ["relu", "tanh"])
}

current_best_params = [{
    'width': 10,
    'height': 0,
    'activation': "relu",
}]

hyperopt_search = HyperOptSearch(
    space, metric="mean_loss", mode="min",
    points_to_evaluate=current_best_params)

tuner = tune.Tuner(
    trainable,
    tune_config=tune.TuneConfig(
        search_alg=hyperopt_search
    ),
)
tuner.fit()

Methods

add_evaluated_point

Pass results from a point that has been evaluated separately.

add_evaluated_trials

Pass results from trials that have been evaluated separately.

on_trial_complete

Notification for the completion of trial.

restore_from_dir

Restores the state of a searcher from a given checkpoint_dir.

save_to_dir

Automatically saves the given searcher to the checkpoint_dir.

set_max_concurrency

Set max concurrent trials this searcher can run.

Attributes

CKPT_FILE_TMPL

FINISHED

metric

The training result objective value attribute.

mode

Specifies if minimizing or maximizing the metric.