ray.data.preprocessors.Concatenator.transform

Concatenator.transform(ds: Dataset, *, batch_size: int | None = None, num_cpus: float | None = None, memory: float | None = None, concurrency: int | None = None) → Dataset

Transform the given dataset.

Parameters:

• ds – Input Dataset.

• batch_size – [experimental] Advanced configuration for adjusting input size for each worker.

• num_cpus – [experimental] The number of CPUs to reserve for each parallel map worker.

• memory – [experimental] The heap memory in bytes to reserve for each parallel map worker.

• concurrency – [experimental] The maximum number of Ray workers to use concurrently.

Returns:

The transformed Dataset.

Return type:

ray.data.Dataset

Raises:

PreprocessorNotFittedException – if fit is not called yet.