ray.train.DataConfig

class ray.train.DataConfig(datasets_to_split: Literal['all'] | List[str] = 'all', execution_options: ExecutionOptions | None = None, enable_shard_locality: bool = True)

Class responsible for configuring Train dataset preprocessing.

For advanced use cases, this class can be subclassed and the configure() method overriden for custom data preprocessing.

Methods

__init__	Construct a DataConfig.
configure	Configure how Train datasets should be assigned to workers.
default_ingest_options	The default Ray Data options used for data ingest.
set_train_total_resources	Set the total number of CPUs and GPUs used by training.