Histogram

class ray.serve.metrics.Histogram(name: str, description: str = '', boundaries: List[float] = None, tag_keys: Tuple[str] | None = None)

Bases: Histogram

Tracks the size and number of events in buckets.

Histograms allow you to calculate aggregate quantiles such as 25, 50, 95, 99 percentile latency for an RPC.

This corresponds to Prometheus’ histogram metric: https://prometheus.io/docs/concepts/metric_types/#histogram

The “deployment”, “replica” and “application” tags are added automatically when the metric is created inside a deployment replica. The “route” tag is not added automatically: declare it in tag_keys and its value is filled in from the current request context on every record call, unless you pass an explicit value.

@serve.deployment
class MyDeployment:
    def __init__(self):
        self.my_histogram = Histogram(
            "my_histogram",
            description=("Histogram of the __call__ method running time."),
            boundaries=[1,2,4,8,16,32,64],
            tag_keys=("model",),
        )
        self.my_histogram.set_default_tags({"model": "123"})

    def __call__(self):
        start = time.time()
        self.my_histogram.observe(time.time() - start)

Parameters:

• name (str) – Name of the metric.

• description (str) – Description of the metric.

• boundaries (List[float]) – Boundaries of histogram buckets.

• tag_keys (Tuple[str] | None) – Tag keys of the metric.

PublicAPI (beta): This API is in beta and may change before becoming stable.

set_default_tags(default_tags: Dict[str, str])

Set default tags of metrics.

Example

>>> from ray.util.metrics import Counter
>>> # Note that set_default_tags returns the instance itself.
>>> counter = Counter("name", tag_keys=("a",))
>>> counter2 = counter.set_default_tags({"a": "b"})
>>> assert counter is counter2
>>> # this means you can instantiate it in this way.
>>> counter = Counter("name", tag_keys=("a",)).set_default_tags({"a": "b"})

Parameters:

default_tags (Dict[str, str]) – Default tags that are used for every record method.

Returns:

it returns the instance itself.

Return type:

Metric

observe(value: int | float, tags: Dict[str, str] = None)

Observe the given value, add serve context tag values to the tags