Search Space API#

Random Distributions API#

This section covers the functions you can use to define your search spaces.

Caution

Not all Search Algorithms support all distributions. In particular, tune.sample_from and tune.grid_search are often unsupported. The default Random search and grid search (tune.search.basic_variant.BasicVariantGenerator) supports all distributions.

For a high-level overview, see this example:

config = {
    # Sample a float uniformly between -5.0 and -1.0
    "uniform": tune.uniform(-5, -1),

    # Sample a float uniformly between 3.2 and 5.4,
    # rounding to multiples of 0.2
    "quniform": tune.quniform(3.2, 5.4, 0.2),

    # Sample a float uniformly between 0.0001 and 0.01, while
    # sampling in log space
    "loguniform": tune.loguniform(1e-4, 1e-2),

    # Sample a float uniformly between 0.0001 and 0.1, while
    # sampling in log space and rounding to multiples of 0.00005
    "qloguniform": tune.qloguniform(1e-4, 1e-1, 5e-5),

    # Sample a random float from a normal distribution with
    # mean=10 and sd=2
    "randn": tune.randn(10, 2),

    # Sample a random float from a normal distribution with
    # mean=10 and sd=2, rounding to multiples of 0.2
    "qrandn": tune.qrandn(10, 2, 0.2),

    # Sample a integer uniformly between -9 (inclusive) and 15 (exclusive)
    "randint": tune.randint(-9, 15),

    # Sample a random uniformly between -21 (inclusive) and 12 (inclusive (!))
    # rounding to multiples of 3 (includes 12)
    # if q is 1, then randint is called instead with the upper bound exclusive
    "qrandint": tune.qrandint(-21, 12, 3),

    # Sample a integer uniformly between 1 (inclusive) and 10 (exclusive),
    # while sampling in log space
    "lograndint": tune.lograndint(1, 10),

    # Sample a integer uniformly between 1 (inclusive) and 10 (inclusive (!)),
    # while sampling in log space and rounding to multiples of 2
    # if q is 1, then lograndint is called instead with the upper bound exclusive
    "qlograndint": tune.qlograndint(1, 10, 2),

    # Sample an option uniformly from the specified choices
    "choice": tune.choice(["a", "b", "c"]),

    # Sample from a random function, in this case one that
    # depends on another value from the search space
    "func": tune.sample_from(lambda spec: spec.config.uniform * 0.01),

    # Do a grid search over these values. Every value will be sampled
    # ``num_samples`` times (``num_samples`` is the parameter you pass to ``tune.TuneConfig``,
    # which is taken in by ``Tuner``)
    "grid": tune.grid_search([32, 64, 128])
}

tune.uniform#

ray.tune.uniform(lower: float, upper: float)[source]#

Sample a float value uniformly between lower and upper.

Sampling from tune.uniform(1, 10) is equivalent to sampling from np.random.uniform(1, 10))

PublicAPI: This API is stable across Ray releases.

tune.quniform#

ray.tune.quniform(lower: float, upper: float, q: float)[source]#

Sample a quantized float value uniformly between lower and upper.

Sampling from tune.uniform(1, 10) is equivalent to sampling from np.random.uniform(1, 10))

The value will be quantized, i.e. rounded to an integer increment of q. Quantization makes the upper bound inclusive.

PublicAPI: This API is stable across Ray releases.

tune.loguniform#

ray.tune.loguniform(lower: float, upper: float, base: float = 10)[source]#

Sugar for sampling in different orders of magnitude.

Parameters

  • lower – Lower boundary of the output interval (e.g. 1e-4)

  • upper – Upper boundary of the output interval (e.g. 1e-2)

  • base – Base of the log. Defaults to 10.

PublicAPI: This API is stable across Ray releases.

tune.qloguniform#

ray.tune.qloguniform(lower: float, upper: float, q: float, base: float = 10)[source]#

Sugar for sampling in different orders of magnitude.

The value will be quantized, i.e. rounded to an integer increment of q.

Quantization makes the upper bound inclusive.

Parameters

  • lower – Lower boundary of the output interval (e.g. 1e-4)

  • upper – Upper boundary of the output interval (e.g. 1e-2)

  • q – Quantization number. The result will be rounded to an integer increment of this value.

  • base – Base of the log. Defaults to 10.

PublicAPI: This API is stable across Ray releases.

tune.randn#

ray.tune.randn(mean: float = 0.0, sd: float = 1.0)[source]#

Sample a float value normally with mean and sd.

Parameters

  • mean – Mean of the normal distribution. Defaults to 0.

  • sd – SD of the normal distribution. Defaults to 1.

PublicAPI: This API is stable across Ray releases.

tune.qrandn#

ray.tune.qrandn(mean: float, sd: float, q: float)[source]#

Sample a float value normally with mean and sd.

The value will be quantized, i.e. rounded to an integer increment of q.

Parameters

  • mean – Mean of the normal distribution.

  • sd – SD of the normal distribution.

  • q – Quantization number. The result will be rounded to an integer increment of this value.

PublicAPI: This API is stable across Ray releases.

tune.randint#

ray.tune.randint(lower: int, upper: int)[source]#

Sample an integer value uniformly between lower and upper.

lower is inclusive, upper is exclusive.

Sampling from tune.randint(10) is equivalent to sampling from np.random.randint(10)

Changed in version 1.5.0: When converting Ray Tune configs to searcher-specific search spaces, the lower and upper limits are adjusted to keep compatibility with the bounds stated in the docstring above.

PublicAPI: This API is stable across Ray releases.

tune.qrandint#

ray.tune.qrandint(lower: int, upper: int, q: int = 1)[source]#

Sample an integer value uniformly between lower and upper.

lower is inclusive, upper is also inclusive (!).

The value will be quantized, i.e. rounded to an integer increment of q. Quantization makes the upper bound inclusive.

Changed in version 1.5.0: When converting Ray Tune configs to searcher-specific search spaces, the lower and upper limits are adjusted to keep compatibility with the bounds stated in the docstring above.

PublicAPI: This API is stable across Ray releases.

tune.lograndint#

ray.tune.lograndint(lower: int, upper: int, base: float = 10)[source]#

Sample an integer value log-uniformly between lower and upper, with base being the base of logarithm.

lower is inclusive, upper is exclusive.

Changed in version 1.5.0: When converting Ray Tune configs to searcher-specific search spaces, the lower and upper limits are adjusted to keep compatibility with the bounds stated in the docstring above.

PublicAPI: This API is stable across Ray releases.

tune.qlograndint#

ray.tune.qlograndint(lower: int, upper: int, q: int, base: float = 10)[source]#

Sample an integer value log-uniformly between lower and upper, with base being the base of logarithm.

lower is inclusive, upper is also inclusive (!).

The value will be quantized, i.e. rounded to an integer increment of q. Quantization makes the upper bound inclusive.

Changed in version 1.5.0: When converting Ray Tune configs to searcher-specific search spaces, the lower and upper limits are adjusted to keep compatibility with the bounds stated in the docstring above.

PublicAPI: This API is stable across Ray releases.

tune.choice#

ray.tune.choice(categories: Sequence)[source]#

Sample a categorical value.

Sampling from tune.choice([1, 2]) is equivalent to sampling from np.random.choice([1, 2])

PublicAPI: This API is stable across Ray releases.

tune.sample_from#

ray.tune.sample_from(func: Callable[[Dict], Any])[source]#

Specify that tune should sample configuration values from this function.

Parameters

func – An callable function to draw a sample from.

PublicAPI: This API is stable across Ray releases.

Grid Search API#

Convenience method for specifying grid search over a value.

Parameters

values – An iterable whose parameters will be gridded.

PublicAPI (beta): This API is in beta and may change before becoming stable.

References#

See also Random search and grid search (tune.search.basic_variant.BasicVariantGenerator).