ray.data.Dataset.to_spark#

Dataset.to_spark(spark: pyspark.sql.SparkSession) → pyspark.sql.DataFrame[source]#

Convert this dataset into a Spark dataframe.

Time complexity: O(dataset size / parallelism)

Returns

A Spark dataframe created from this dataset.