ray.serve.metrics.Histogram

class ray.serve.metrics.Histogram(name: str, description: str = '', boundaries: List[float] = None, tag_keys: Tuple[str] | None = None)

Bases: Histogram

Tracks the size and number of events in buckets.

Histograms allow you to calculate aggregate quantiles such as 25, 50, 95, 99 percentile latency for an RPC.

This corresponds to Prometheus’ histogram metric: https://prometheus.io/docs/concepts/metric_types/#histogram

Serve-related tags (“deployment”, “replica”, “application”, “route”) are added automatically if not provided.

@serve.deployment
class MyDeployment:
    def __init__(self):
        self.my_histogram = Histogram(
            "my_histogram",
            description=("Histogram of the __call__ method running time."),
            boundaries=[1,2,4,8,16,32,64],
            tag_keys=("model",),
        )
        self.my_histogram.set_default_tags({"model": "123"})

    def __call__(self):
        start = time.time()
        self.my_histogram.observe(time.time() - start)

Parameters:

• name – Name of the metric.

• description – Description of the metric.

• boundaries – Boundaries of histogram buckets.

• tag_keys – Tag keys of the metric.

PublicAPI (beta): This API is in beta and may change before becoming stable.

Methods

observe

Observe the given value, add serve context tag values to the tags

Attributes

info	Return information about histogram metric.