ray.data.DatasetPipeline.write_parquet

DatasetPipeline.write_parquet(path: str, *, filesystem: Optional[pyarrow.fs.FileSystem] = None, try_create_dir: bool = True, arrow_open_stream_args: Optional[Dict[str, Any]] = None, block_path_provider: ray.data.datasource.file_based_datasource.BlockWritePathProvider = <ray.data.datasource.file_based_datasource.DefaultBlockWritePathProvider object>, arrow_parquet_args_fn: Callable[[], Dict[str, Any]] = <function DatasetPipeline.<lambda>>, ray_remote_args: Dict[str, Any] = None, **arrow_parquet_args) → None

Call Dataset.write_parquet on each output dataset of this pipeline.