ray.air.util.check_ingest.make_local_dataset_iterator#

ray.air.util.check_ingest.make_local_dataset_iterator(dataset: ray.data.dataset.Dataset, preprocessor: ray.data.preprocessor.Preprocessor, dataset_config: ray.air.config.DatasetConfig) → ray.data.dataset_iterator.DatasetIterator[source]#

A helper function to create a local DatasetIterator, like the one returned by get_dataset_shard().

This function should only be used for development and debugging. It will raise an exception if called by a worker instead of the driver.

Parameters

  • dataset – The input Dataset.

  • preprocessor – The preprocessor that will be applied to the input dataset.

  • dataset_config – The dataset config normally passed to the trainer.

DeveloperAPI: This API may change across minor Ray releases.