ray.data.from_torch(dataset: torch.utils.data.Dataset) ray.data.dataset.MaterializedDataset[source]#

Create a dataset from a Torch dataset.

This function is inefficient. Use it to read small datasets or prototype.


If your dataset is large, this function may execute slowly or raise an out-of-memory error. To avoid issues, read the underyling data with a function like read_images().


This function isn’t paralellized. It loads the entire dataset into the head node’s memory before moving the data to the distributed object store.


>>> import ray
>>> from torchvision import datasets
>>> dataset = datasets.MNIST("data", download=True)  
>>> ds = ray.data.from_torch(dataset)  
>>> ds  
Dataset(num_blocks=200, num_rows=60000, schema={item: object})
>>> ds.take(1)  
{"item": (<PIL.Image.Image image mode=L size=28x28 at 0x...>, 5)}

dataset – A Torch dataset.


A MaterializedDataset containing the Torch dataset samples.

PublicAPI: This API is stable across Ray releases.