ApproximateQuantile#

class ray.data.aggregate.ApproximateQuantile(on: str, quantiles: List[float], quantile_precision: int = 800, alias_name: str | None = None)[source]#

Bases: AggregateFnV2

Computes the approximate quantiles of a column by using a datasketches kll_floats_sketch. https://datasketches.apache.org/docs/KLL/KLLSketch.html

The accuracy of the KLL quantile sketch is a function of the configured quantile precision, which also affects the overall size of the sketch. The KLL Sketch has absolute error. For example, a specified rank accuracy of 1% at the median (rank = 0.50) means that the true quantile (if you could extract it from the set) should be between getQuantile(0.49) and getQuantile(0.51). This same 1% error applied at a rank of 0.95 means that the true quantile should be between getQuantile(0.94) and getQuantile(0.96). In other words, the error is a fixed +/- epsilon for the entire range of ranks.

Typical single-sided rank error by quantile_precision (use for getQuantile/getRank):

  • quantile_precision=100 → ~2.61%

  • quantile_precision=200 → ~1.33%

  • quantile_precision=400 → ~0.68%

  • quantile_precision=800 → ~0.35%

See https://datasketches.apache.org/docs/KLL/KLLAccuracyAndSize.html for details on accuracy and size.

Null values in the target column are ignored when constructing the sketch.

Examples

>>> import ray
>>> from ray.data.aggregate import ApproximateQuantile
>>> # Create a dataset with some values
>>> ds = ray.data.from_items(
...     [{"value": 20.0}, {"value": 40.0}, {"value": 60.0},
...     {"value": 80.0}, {"value": 100.0}]
... )
>>> result = ds.aggregate(ApproximateQuantile(on="value", quantiles=[0.1, 0.5, 0.9]))
>>> # Result: {'approx_quantile(value)': [20.0, 60.0, 100.0]}
Parameters:

  • on (str) – The name of the column to calculate the quantile on. Must be a numeric column.

  • quantiles (List[float]) – The list of quantiles to compute. Must be between 0 and 1 inclusive. For example, quantiles=[0.5] computes the median. Null entries in the source column are skipped.

  • quantile_precision (int) – Controls the accuracy and memory footprint of the sketch (K in KLL); higher values yield lower error but use more memory. Defaults to 800. See https://datasketches.apache.org/docs/KLL/KLLAccuracyAndSize.html for details on accuracy and size.

  • alias_name (str | None) – Optional name for the resulting column. If not provided, defaults to “approx_quantile({column_name})”.

PublicAPI (alpha): This API is in alpha and may change before becoming stable.

Methods

get_agg_name

Return the agg name (e.g., 'sum', 'mean', 'count').

get_target_column

Return the name of the column this aggregation reads.

output_field

Return the PyArrow Field this aggregator produces.

zero

Create an empty KLL sketch.