ray.data.Dataset.mean#

Dataset.mean(on: Union[None, str, Callable[[ray.data.block.T], Any], List[Union[None, str, Callable[[ray.data.block.T], Any]]]] = None, ignore_nulls: bool = True) → ray.data.block.U[source]#

Compute mean over entire dataset.

Note

This operation will trigger execution of the lazy transformations performed on this dataset, and will block until execution completes.

Examples

>>> import ray
>>> ray.data.range(100).mean()
49.5
>>> ray.data.from_items([
...     (i, i**2)
...     for i in range(100)]).mean(lambda x: x[1])
3283.5
>>> ray.data.range_table(100).mean("value")
49.5
>>> ray.data.from_items([
...     {"A": i, "B": i**2}
...     for i in range(100)]).mean(["A", "B"])
{'mean(A)': 49.5, 'mean(B)': 3283.5}
Parameters

  • on –

    The data subset on which to compute the mean.

    • For a simple dataset: it can be a callable or a list thereof, and the default is to return a scalar mean of all rows.

    • For an Arrow dataset: it can be a column name or a list thereof, and the default is to return an ArrowRow containing the column-wise mean of all columns.

  • ignore_nulls – Whether to ignore null values. If True, null values will be ignored when computing the mean; if False, if a null value is encountered, the output will be None. We consider np.nan, None, and pd.NaT to be null values. Default is True.

Returns

The mean result.

For a simple dataset, the output is:

  • on=None: a scalar representing the mean of all rows,

  • on=callable: a scalar representing the mean of the outputs of the callable called on each row,

  • on=[callable_1, ..., calalble_n]: a tuple of (mean_1, ..., mean_n) representing the mean of the outputs of the corresponding callables called on each row.

For an Arrow dataset, the output is:

  • on=None: an ArrowRow containing the column-wise mean of all columns,

  • on="col": a scalar representing the mean of all items in column "col",

  • on=["col_1", ..., "col_n"]: an n-column ArrowRow containing the column-wise mean of the provided columns.

If the dataset is empty, all values are null, or any value is null AND ignore_nulls is False, then the output will be None.