ray.data.Datasink.on_write_complete#

Datasink.on_write_complete(write_result_blocks: List[pyarrow.Table | pandas.DataFrame]) → WriteResult[source]#

Callback for when a write job completes.

This can be used to “commit” a write output. This method must succeed prior to write_datasink() returning to the user. If this method fails, then on_write_failed() is called.

Parameters:

  • write_result_blocks – The blocks resulting from executing

  • operator (the Write)

  • stats. (containing write results and)

Returns:

A WriteResult object containing the aggregated stats of all the input write results.