ray.serve.metrics.Histogram

class ray.serve.metrics.Histogram(name: str, description: str = '', boundaries: List[float] | None = None, tag_keys: Tuple[str] | None = None)

Bases: Histogram

Tracks the size and number of events in buckets.

Histograms allow you to calculate aggregate quantiles such as 25, 50, 95, 99 percentile latency for an RPC.

This corresponds to Prometheus’ histogram metric: https://prometheus.io/docs/concepts/metric_types/#histogram

Serve-related tags (“deployment”, “replica”, “application”, “route”) are added automatically if not provided.

@serve.deployment
class MyDeployment:
    def __init__(self):
        self.my_histogram = Histogram(
            "my_histogram",
            description=("Histogram of the __call__ method running time."),
            boundaries=[1,2,4,8,16,32,64],
            tag_keys=("model",),
        )
        self.my_histogram.set_default_tags({"model": "123"})

    def __call__(self):
        start = time.time()
        self.my_histogram.observe(time.time() - start)

Parameters:

• name – Name of the metric.

• description – Description of the metric.

• boundaries – Boundaries of histogram buckets.

• tag_keys – Tag keys of the metric.

PublicAPI (beta): This API is in beta and may change before becoming stable.

set_default_tags(default_tags: Dict[str, str])

Set default tags of metrics.

Example

>>> from ray.util.metrics import Counter
>>> # Note that set_default_tags returns the instance itself.
>>> counter = Counter("name", tag_keys=("a",))
>>> counter2 = counter.set_default_tags({"a": "b"})
>>> assert counter is counter2
>>> # this means you can instantiate it in this way.
>>> counter = Counter("name", tag_keys=("a",)).set_default_tags({"a": "b"})

Parameters:

default_tags – Default tags that are used for every record method.

Returns:

it returns the instance itself.

Return type:

Metric

observe(value: int | float, tags: Dict[str, str] | None = None)

Observe the given value, add serve context tag values to the tags