Tune Search Space API#

This section covers the functions you can use to define your search spaces.

Caution

Not all Search Algorithms support all distributions. In particular, tune.sample_from and tune.grid_search are often unsupported. The default Random search and grid search (tune.search.basic_variant.BasicVariantGenerator) supports all distributions.

Tip

Avoid passing large objects as values in the search space, as that will incur a performance overhead. Use tune.with_parameters to pass large objects in or load them inside your trainable from disk (making sure that all nodes have access to the files) or cloud storage. See How can I avoid bottlenecks? for more information.

For a high-level overview, see this example:

config = {
    # Sample a float uniformly between -5.0 and -1.0
    "uniform": tune.uniform(-5, -1),

    # Sample a float uniformly between 3.2 and 5.4,
    # rounding to multiples of 0.2
    "quniform": tune.quniform(3.2, 5.4, 0.2),

    # Sample a float uniformly between 0.0001 and 0.01, while
    # sampling in log space
    "loguniform": tune.loguniform(1e-4, 1e-2),

    # Sample a float uniformly between 0.0001 and 0.1, while
    # sampling in log space and rounding to multiples of 0.00005
    "qloguniform": tune.qloguniform(1e-4, 1e-1, 5e-5),

    # Sample a random float from a normal distribution with
    # mean=10 and sd=2
    "randn": tune.randn(10, 2),

    # Sample a random float from a normal distribution with
    # mean=10 and sd=2, rounding to multiples of 0.2
    "qrandn": tune.qrandn(10, 2, 0.2),

    # Sample a integer uniformly between -9 (inclusive) and 15 (exclusive)
    "randint": tune.randint(-9, 15),

    # Sample a random uniformly between -21 (inclusive) and 12 (inclusive (!))
    # rounding to multiples of 3 (includes 12)
    # if q is 1, then randint is called instead with the upper bound exclusive
    "qrandint": tune.qrandint(-21, 12, 3),

    # Sample a integer uniformly between 1 (inclusive) and 10 (exclusive),
    # while sampling in log space
    "lograndint": tune.lograndint(1, 10),

    # Sample a integer uniformly between 1 (inclusive) and 10 (inclusive (!)),
    # while sampling in log space and rounding to multiples of 2
    # if q is 1, then lograndint is called instead with the upper bound exclusive
    "qlograndint": tune.qlograndint(1, 10, 2),

    # Sample an option uniformly from the specified choices
    "choice": tune.choice(["a", "b", "c"]),

    # Sample from a random function, in this case one that
    # depends on another value from the search space
    "func": tune.sample_from(lambda spec: spec.config.uniform * 0.01),

    # Do a grid search over these values. Every value will be sampled
    # ``num_samples`` times (``num_samples`` is the parameter you pass to ``tune.TuneConfig``,
    # which is taken in by ``Tuner``)
    "grid": tune.grid_search([32, 64, 128])
}

Random Distributions API#

tune.uniform

Sample a float value uniformly between lower and upper.

tune.quniform

Sample a quantized float value uniformly between lower and upper.

tune.loguniform

Sugar for sampling in different orders of magnitude.

tune.qloguniform

Sugar for sampling in different orders of magnitude.

tune.randn

Sample a float value normally with mean and sd.

tune.qrandn

Sample a float value normally with mean and sd.

tune.randint

Sample an integer value uniformly between lower and upper.

tune.qrandint

Sample an integer value uniformly between lower and upper.

tune.lograndint

Sample an integer value log-uniformly between lower and upper, with base being the base of logarithm.

tune.qlograndint

Sample an integer value log-uniformly between lower and upper, with base being the base of logarithm.

tune.choice

Sample a categorical value.

Grid Search and Custom Function APIs#

tune.grid_search

Specify a grid of values to search over.

tune.sample_from

Specify that tune should sample configuration values from this function.

References#

See also Random search and grid search (tune.search.basic_variant.BasicVariantGenerator).