ray.data.Dataset.write_datasource
ray.data.Dataset.write_datasource#
- Dataset.write_datasource(datasource: ray.data.datasource.datasource.Datasource, *, ray_remote_args: Optional[Dict[str, Any]] = None, **write_args) None [source]#
Writes the dataset to a custom
Datasource
.For an example of how to use this method, see Implementing a Custom Datasource.
Note
This operation will trigger execution of the lazy transformations performed on this dataset.
Time complexity: O(dataset size / parallelism)
- Parameters
datasource – The
Datasource
to write to.ray_remote_args – Kwargs passed to
ray.remote
in the write tasks.write_args – Additional write args to pass to the
Datasource
.