ray.tune.search.hyperopt.HyperOptSearch

class ray.tune.search.hyperopt.HyperOptSearch(space: Optional[Dict] = None, metric: Optional[str] = None, mode: Optional[str] = None, points_to_evaluate: Optional[List[Dict]] = None, n_initial_points: int = 20, random_state_seed: Optional[int] = None, gamma: float = 0.25)

Bases: ray.tune.search.searcher.Searcher

A wrapper around HyperOpt to provide trial suggestions.

HyperOpt a Python library for serial and parallel optimization over awkward search spaces, which may include real-valued, discrete, and conditional dimensions. More info can be found at http://hyperopt.github.io/hyperopt.

HyperOptSearch uses the Tree-structured Parzen Estimators algorithm, though it can be trivially extended to support any algorithm HyperOpt supports.

To use this search algorithm, you will need to install HyperOpt:

pip install -U hyperopt

Parameters

• space – HyperOpt configuration. Parameters will be sampled from this configuration and will be used to override parameters generated in the variant generation process.

• metric – The training result objective value attribute. If None but a mode was passed, the anonymous metric _metric will be used per default.

• mode – One of {min, max}. Determines whether objective is minimizing or maximizing the metric attribute.

• points_to_evaluate – Initial parameter suggestions to be run first. This is for when you already have some good parameters you want to run first to help the algorithm make better suggestions for future parameters. Needs to be a list of dicts containing the configurations.

• n_initial_points – number of random evaluations of the objective function before starting to aproximate it with tree parzen estimators. Defaults to 20.

• random_state_seed – seed for reproducible results. Defaults to None.

• gamma – parameter governing the tree parzen estimators suggestion algorithm. Defaults to 0.25.

Tune automatically converts search spaces to HyperOpt’s format:

config = {
    'width': tune.uniform(0, 20),
    'height': tune.uniform(-100, 100),
    'activation': tune.choice(["relu", "tanh"])
}

current_best_params = [{
    'width': 10,
    'height': 0,
    'activation': "relu",
}]

hyperopt_search = HyperOptSearch(
    metric="mean_loss", mode="min",
    points_to_evaluate=current_best_params)

tuner = tune.Tuner(
    trainable,
    tune_config=tune.TuneConfig(
        search_alg=hyperopt_search
    ),
    param_space=config
)
tuner.fit()

If you would like to pass the search space manually, the code would look like this:

space = {
    'width': hp.uniform('width', 0, 20),
    'height': hp.uniform('height', -100, 100),
    'activation': hp.choice("activation", ["relu", "tanh"])
}

current_best_params = [{
    'width': 10,
    'height': 0,
    'activation': "relu",
}]

hyperopt_search = HyperOptSearch(
    space, metric="mean_loss", mode="min",
    points_to_evaluate=current_best_params)

tuner = tune.Tuner(
    trainable,
    tune_config=tune.TuneConfig(
        search_alg=hyperopt_search
    ),
)
tuner.fit()

set_search_properties(metric: Optional[str], mode: Optional[str], config: Dict, **spec) → bool

Pass search properties to searcher.

This method acts as an alternative to instantiating search algorithms with their own specific search spaces. Instead they can accept a Tune config through this method. A searcher should return True if setting the config was successful, or False if it was unsuccessful, e.g. when the search space has already been set.

Parameters

• metric – Metric to optimize

• mode – One of [“min”, “max”]. Direction to optimize.

• config – Tune config dict.

• **spec – Any kwargs for forward compatiblity. Info like Experiment.PUBLIC_KEYS is provided through here.

suggest(trial_id: str) → Optional[Dict]

Queries the algorithm to retrieve the next set of parameters.

Parameters

trial_id – Trial ID used for subsequent notifications.

Returns

Configuration for a trial, if possible.

If FINISHED is returned, Tune will be notified that no more suggestions/configurations will be provided. If None is returned, Tune will skip the querying of the searcher for this step.

Return type

dict | FINISHED | None

on_trial_result(trial_id: str, result: Dict) → None

Optional notification for result during training.

Note that by default, the result dict may include NaNs or may not include the optimization metric. It is up to the subclass implementation to preprocess the result to avoid breaking the optimization process.

Parameters

• trial_id – A unique string ID for the trial.

• result – Dictionary of metrics for current training progress. Note that the result dict may include NaNs or may not include the optimization metric. It is up to the subclass implementation to preprocess the result to avoid breaking the optimization process.

on_trial_complete(trial_id: str, result: Optional[Dict] = None, error: bool = False) → None

Notification for the completion of trial.

The result is internally negated when interacting with HyperOpt so that HyperOpt can “maximize” this value, as it minimizes on default.

save(checkpoint_path: str) → None

Save state to path for this search algorithm.

Parameters

checkpoint_path – File where the search algorithm state is saved. This path should be used later when restoring from file.

Example:

search_alg = Searcher(...)

tuner = tune.Tuner(
    cost,
    tune_config=tune.TuneConfig(
        search_alg=search_alg,
        num_samples=5
    ),
    param_space=config
)
results = tuner.fit()

search_alg.save("./my_favorite_path.pkl")

Changed in version 0.8.7: Save is automatically called by Tuner().fit(). You can use Tuner().restore() to restore from an experiment directory such as /ray_results/trainable.

restore(checkpoint_path: str) → None

Restore state for this search algorithm

Parameters

checkpoint_path – File where the search algorithm state is saved. This path should be the same as the one provided to “save”.

Example:

search_alg.save("./my_favorite_path.pkl")

search_alg2 = Searcher(...)
search_alg2 = ConcurrencyLimiter(search_alg2, 1)
search_alg2.restore(checkpoint_path)
tuner = tune.Tuner(
    cost,
    tune_config=tune.TuneConfig(
        search_alg=search_alg2,
        num_samples=5
    ),
)
tuner.fit()