ray.data.Dataset.to_numpy_refs#

Dataset.to_numpy_refs(*, column: Optional[str] = None) → List[ray.types.ObjectRef[numpy.ndarray]][source]#

Convert this dataset into a distributed set of NumPy ndarrays.

This is only supported for datasets convertible to NumPy ndarrays. This function induces a copy of the data. For zero-copy access to the underlying data, consider using .to_arrow() or .get_internal_block_refs().

Time complexity: O(dataset size / parallelism)

Parameters

  • column – The name of the column to convert to numpy, or None to specify the

  • blocks (entire row. If not specified for Arrow or Pandas) –

  • returned (each) –

  • ndarrays. (future will represent a dict of column) –

Returns

A list of remote NumPy ndarrays created from this dataset.

DeveloperAPI: This API may change across minor Ray releases.