with_columns#

Dataset.with_columns(exprs: Mapping[str, Expr], *, compute: ComputeStrategy | None = None, **ray_remote_args) → Dataset[source]#

Add or overwrite multiple columns via expressions in a single projection.

This is the multi-column counterpart of with_column(). All expressions are evaluated within one projection over the existing columns, which avoids the repeated work of chaining several with_column calls.

Examples

>>> import ray
>>> from ray.data.expressions import col
>>> ds = ray.data.range(100)
>>> ds.with_columns({
...     "id_2": col("id") * 2,
...     "id_3": col("id") * 3,
... }).show(2)
{'id': 0, 'id_2': 0, 'id_3': 0}
{'id': 1, 'id_2': 2, 'id_3': 3}

Parameters:

exprs – A mapping from new column name to the expression that defines its values. Column order follows the mapping’s insertion order.
compute – The compute strategy to use for the projection operation.
**ray_remote_args – Additional resource requirements to request from Ray for the map tasks (e.g., num_gpus=1).

Returns:

A new dataset with the added or overwritten columns.

PublicAPI (alpha): This API is in alpha and may change before becoming stable.