Tune Search Space API#

This section covers the functions you can use to define your search spaces.

Caution

Not all Search Algorithms support all distributions. In particular, tune.sample_from and tune.grid_search are often unsupported. The default Random search and grid search (tune.search.basic_variant.BasicVariantGenerator) supports all distributions.

Tip

Avoid passing large objects as values in the search space, as that will incur a performance overhead. Use tune.with_parameters to pass large objects in or load them inside your trainable from disk (making sure that all nodes have access to the files) or cloud storage. See How can I avoid bottlenecks? for more information.

For a high-level overview, see this example:

config = {
    # Sample a float uniformly between -5.0 and -1.0
    "uniform": tune.uniform(-5, -1),

    # Sample a float uniformly between 3.2 and 5.4,
    # rounding to multiples of 0.2
    "quniform": tune.quniform(3.2, 5.4, 0.2),

    # Sample a float uniformly between 0.0001 and 0.01, while
    # sampling in log space
    "loguniform": tune.loguniform(1e-4, 1e-2),

    # Sample a float uniformly between 0.0001 and 0.1, while
    # sampling in log space and rounding to multiples of 0.00005
    "qloguniform": tune.qloguniform(1e-4, 1e-1, 5e-5),

    # Sample a random float from a normal distribution with
    # mean=10 and sd=2
    "randn": tune.randn(10, 2),

    # Sample a random float from a normal distribution with
    # mean=10 and sd=2, rounding to multiples of 0.2
    "qrandn": tune.qrandn(10, 2, 0.2),

    # Sample a integer uniformly between -9 (inclusive) and 15 (exclusive)
    "randint": tune.randint(-9, 15),

    # Sample a random uniformly between -21 (inclusive) and 12 (inclusive (!))
    # rounding to multiples of 3 (includes 12)
    # if q is 1, then randint is called instead with the upper bound exclusive
    "qrandint": tune.qrandint(-21, 12, 3),

    # Sample a integer uniformly between 1 (inclusive) and 10 (exclusive),
    # while sampling in log space
    "lograndint": tune.lograndint(1, 10),

    # Sample a integer uniformly between 1 (inclusive) and 10 (inclusive (!)),
    # while sampling in log space and rounding to multiples of 2
    # if q is 1, then lograndint is called instead with the upper bound exclusive
    "qlograndint": tune.qlograndint(1, 10, 2),

    # Sample an option uniformly from the specified choices
    "choice": tune.choice(["a", "b", "c"]),

    # Sample from a random function, in this case one that
    # depends on another value from the search space
    "func": tune.sample_from(lambda spec: spec.config.uniform * 0.01),

    # Do a grid search over these values. Every value will be sampled
    # ``num_samples`` times (``num_samples`` is the parameter you pass to ``tune.TuneConfig``,
    # which is taken in by ``Tuner``)
    "grid": tune.grid_search([32, 64, 128])
}

Random Distributions API#

tune.uniform(lower, upper)

Sample a float value uniformly between lower and upper.

tune.quniform(lower, upper, q)

Sample a quantized float value uniformly between lower and upper.

tune.loguniform(lower, upper[, base])

Sugar for sampling in different orders of magnitude.

tune.qloguniform(lower, upper, q[, base])

Sugar for sampling in different orders of magnitude.

tune.randn([mean, sd])

Sample a float value normally with mean and sd.

tune.qrandn(mean, sd, q)

Sample a float value normally with mean and sd.

tune.randint(lower, upper)

Sample an integer value uniformly between lower and upper.

tune.qrandint(lower, upper[, q])

Sample an integer value uniformly between lower and upper.

tune.lograndint(lower, upper[, base])

Sample an integer value log-uniformly between lower and upper, with base being the base of logarithm.

tune.qlograndint(lower, upper, q[, base])

Sample an integer value log-uniformly between lower and upper, with base being the base of logarithm.

tune.choice(categories)

Sample a categorical value.

Grid Search and Custom Function APIs#

tune.grid_search(values)

Specify a grid of values to search over.

tune.sample_from(func)

Specify that tune should sample configuration values from this function.

References#

See also Random search and grid search (tune.search.basic_variant.BasicVariantGenerator).