ray.data.Dataset.to_arrow_refs
ray.data.Dataset.to_arrow_refs#
- Dataset.to_arrow_refs() List[ray.types.ObjectRef[pyarrow.Table]] [source]#
Convert this dataset into a distributed set of Arrow tables.
This is only supported for datasets convertible to Arrow records. This function is zero-copy if the existing data is already in Arrow format. Otherwise, the data will be converted to Arrow format.
Time complexity: O(1) unless conversion is required.
- Returns
A list of remote Arrow tables created from this dataset.
DeveloperAPI: This API may change across minor Ray releases.