ray.tune.search.hyperopt.HyperOptSearch#

class ray.tune.search.hyperopt.HyperOptSearch(space: Dict | None = None, metric: str | None = None, mode: str | None = None, points_to_evaluate: List[Dict] | None = None, n_initial_points: int = 20, random_state_seed: int | None = None, gamma: float = 0.25)[source]#

Bases: Searcher

A wrapper around HyperOpt to provide trial suggestions.

HyperOpt a Python library for serial and parallel optimization over awkward search spaces, which may include real-valued, discrete, and conditional dimensions. More info can be found at http://hyperopt.github.io/hyperopt.

HyperOptSearch uses the Tree-structured Parzen Estimators algorithm, though it can be trivially extended to support any algorithm HyperOpt supports.

To use this search algorithm, you will need to install HyperOpt:

pip install -U hyperopt

Parameters:

space – HyperOpt configuration. Parameters will be sampled from this configuration and will be used to override parameters generated in the variant generation process.
metric – The training result objective value attribute. If None but a mode was passed, the anonymous metric _metric will be used per default.
mode – One of {min, max}. Determines whether objective is minimizing or maximizing the metric attribute.
points_to_evaluate – Initial parameter suggestions to be run first. This is for when you already have some good parameters you want to run first to help the algorithm make better suggestions for future parameters. Needs to be a list of dicts containing the configurations.
n_initial_points – number of random evaluations of the objective function before starting to aproximate it with tree parzen estimators. Defaults to 20.
random_state_seed – seed for reproducible results. Defaults to None.
gamma – parameter governing the tree parzen estimators suggestion algorithm. Defaults to 0.25.

Tune automatically converts search spaces to HyperOpt’s format:

config = {
    'width': tune.uniform(0, 20),
    'height': tune.uniform(-100, 100),
    'activation': tune.choice(["relu", "tanh"])
}

current_best_params = [{
    'width': 10,
    'height': 0,
    'activation': "relu",
}]

hyperopt_search = HyperOptSearch(
    metric="mean_loss", mode="min",
    points_to_evaluate=current_best_params)

tuner = tune.Tuner(
    trainable,
    tune_config=tune.TuneConfig(
        search_alg=hyperopt_search
    ),
    param_space=config
)
tuner.fit()

If you would like to pass the search space manually, the code would look like this:

space = {
    'width': hp.uniform('width', 0, 20),
    'height': hp.uniform('height', -100, 100),
    'activation': hp.choice("activation", ["relu", "tanh"])
}

current_best_params = [{
    'width': 10,
    'height': 0,
    'activation': "relu",
}]

hyperopt_search = HyperOptSearch(
    space, metric="mean_loss", mode="min",
    points_to_evaluate=current_best_params)

tuner = tune.Tuner(
    trainable,
    tune_config=tune.TuneConfig(
        search_alg=hyperopt_search
    ),
)
tuner.fit()

Methods

`add_evaluated_point`	Pass results from a point that has been evaluated separately.
`add_evaluated_trials`	Pass results from trials that have been evaluated separately.
`on_trial_complete`	Notification for the completion of trial.
`restore_from_dir`	Restores the state of a searcher from a given checkpoint_dir.
`save_to_dir`	Automatically saves the given searcher to the checkpoint_dir.
`set_max_concurrency`	Set max concurrent trials this searcher can run.

Attributes

`CKPT_FILE_TMPL`
`FINISHED`
`metric`	The training result objective value attribute.
`mode`	Specifies if minimizing or maximizing the metric.