ray.data.Dataset.to_arrow_refs

Dataset.to_arrow_refs() → List[ray.types.ObjectRef[pyarrow.Table]]

Convert this dataset into a distributed set of Arrow tables.

This is only supported for datasets convertible to Arrow records. This function is zero-copy if the existing data is already in Arrow format. Otherwise, the data will be converted to Arrow format.

Time complexity: O(1) unless conversion is required.

Returns

A list of remote Arrow tables created from this dataset.

DeveloperAPI: This API may change across minor Ray releases.