ray.data.from_arrow_refs
ray.data.from_arrow_refs#
- ray.data.from_arrow_refs(tables: Union[ray.types.ObjectRef[Union[pyarrow.Table, bytes]], List[ray.types.ObjectRef[Union[pyarrow.Table, bytes]]]]) ray.data.dataset.MaterializedDataset [source]#
Create a
Dataset
from a list of Ray object references to PyArrow tables.Examples
>>> import pyarrow as pa >>> import ray >>> table_ref = ray.put(pa.table({"x": [1]})) >>> ray.data.from_arrow_refs(table_ref) MaterializedDataset(num_blocks=1, num_rows=1, schema={x: int64})
Create a Ray Dataset from a list of PyArrow table references
>>> ray.data.from_arrow_refs([table_ref, table_ref]) MaterializedDataset(num_blocks=2, num_rows=2, schema={x: int64})
- Parameters
tables – A Ray object reference to Arrow table, or list of Ray object references to Arrow tables, or its streaming format in bytes.
- Returns
Dataset
holding data read from the tables.
DeveloperAPI: This API may change across minor Ray releases.