Dataset.write_datasource(datasource: ray.data.datasource.datasource.Datasource, *, ray_remote_args: Optional[Dict[str, Any]] = None, **write_args) None[source]#

Writes the dataset to a custom Datasource.

For an example of how to use this method, see Implementing a Custom Datasource.


This operation will trigger execution of the lazy transformations performed on this dataset.

Time complexity: O(dataset size / parallelism)

  • datasource – The Datasource to write to.

  • ray_remote_args – Kwargs passed to ray.remote in the write tasks.

  • write_args – Additional write args to pass to the Datasource.