ray.data.Dataset.to_pandas_refs#

Dataset.to_pandas_refs() List[ray.types.ObjectRef[pandas.DataFrame]][source]#

Convert this dataset into a distributed set of Pandas dataframes.

This is only supported for datasets convertible to Arrow records. This function induces a copy of the data. For zero-copy access to the underlying data, consider using .to_arrow() or .get_internal_block_refs().

Time complexity: O(dataset size / parallelism)

Returns

A list of remote Pandas dataframes created from this dataset.

DeveloperAPI: This API may change across minor Ray releases.