ray.data.Dataset.write_datasource

Dataset.write_datasource(datasource: ray.data.datasource.datasource.Datasource, *, ray_remote_args: Optional[Dict[str, Any]] = None, **write_args) → None

Writes the dataset to a custom Datasource.

For an example of how to use this method, see Implementing a Custom Datasource.

Note

This operation will trigger execution of the lazy transformations performed on this dataset.

Time complexity: O(dataset size / parallelism)

Parameters

• datasource – The Datasource to write to.

• ray_remote_args – Kwargs passed to ray.remote in the write tasks.

• write_args – Additional write args to pass to the Datasource.