Unique#

class ray.data.aggregate.Unique(on: str | None = None, ignore_nulls: bool = False, alias_name: str | None = None, encode_lists: bool | ListEncodingMode | None = None)[source]#

Bases: VectorizedAggregateFnV2[Set[Any], List[Any]]

Defines unique aggregation.

Examples

>>> import ray
>>> from ray.data.aggregate import Unique
>>> ds = ray.data.range(100)
>>> ds = ds.add_column(
...     "group_key", lambda batch: batch["id"].astype("int64") % 3
... )
>>> # Calculating the unique values per group:
>>> result = ds.groupby("group_key").aggregate(Unique(on="id")).take_all()
>>> # result: [{'group_key': 0, 'unique(id)': ...},
>>> #          {'group_key': 1, 'unique(id)': ...},
>>> #          {'group_key': 2, 'unique(id)': ...}]
Parameters:

  • on (str | None) – The name of the column from which to collect unique values.

  • ignore_nulls (bool) – Whether to ignore null values when collecting unique items. Default is True (nulls are excluded).

  • alias_name (str | None) – Optional name for the resulting column.

  • encode_lists (bool | ListEncodingMode | None) – If True, encode list elements. If False, encode whole lists (i.e., the entire list is considered as a single object). False by default. Note that this is a top-level flatten (not a recursive flatten) operation.

Values that the vectorized path can’t handle (column types without an Arrow unique kernel, such as tensors, or list-typed columns when the optional polars package isn’t installed) are collected in Python instead; lists and arrays become tuples.

Methods

finalize

Transforms the final accumulated state into the desired output.

get_agg_name

Return the agg name (e.g., 'sum', 'mean', 'count').

get_target_column

Return the name of the column this aggregation reads.

output_field

Return the PyArrow Field this aggregator produces.