ray.data.Dataset.to_pandas_refs#

Dataset.to_pandas_refs() → List[ray.types.ObjectRef[pandas.DataFrame]][source]#

Convert this dataset into a distributed set of Pandas dataframes.

This is only supported for datasets convertible to Arrow records. This function induces a copy of the data. For zero-copy access to the underlying data, consider using .to_arrow() or .get_internal_block_refs().

Note

This operation will trigger execution of the lazy transformations performed on this dataset, and will block until execution completes.

Time complexity: O(dataset size / parallelism)

Returns

A list of remote Pandas dataframes created from this dataset.

DeveloperAPI: This API may change across minor Ray releases.