ray.data.from_arrow_refs

ray.data.from_arrow_refs(tables: ObjectRef[pyarrow.Table | bytes] | List[ObjectRef[pyarrow.Table | bytes]]) → MaterializedDataset

Create a Dataset from a list of Ray object references to PyArrow tables.

Examples

>>> import pyarrow as pa
>>> import ray
>>> table_ref = ray.put(pa.table({"x": [1]}))
>>> ray.data.from_arrow_refs(table_ref)
MaterializedDataset(num_blocks=1, num_rows=1, schema={x: int64})

Create a Ray Dataset from a list of PyArrow table references

>>> ray.data.from_arrow_refs([table_ref, table_ref])
MaterializedDataset(num_blocks=2, num_rows=2, schema={x: int64})

Parameters:

tables – A Ray object reference to Arrow table, or list of Ray object references to Arrow tables, or its streaming format in bytes.

Returns:

Dataset holding data read from the tables.

DeveloperAPI: This API may change across minor Ray releases.