ray.data.Dataset.serialize_lineage

Dataset.serialize_lineage() → bytes

Serialize this dataset’s lineage, not the actual data or the existing data futures, to bytes that can be stored and later deserialized, possibly on a different cluster.

Note that this will drop all computed data, and that everything will be recomputed from scratch after deserialization.

Use Dataset.deserialize_lineage() to deserialize the serialized bytes returned from this method into a Dataset.

Note

Unioned and zipped datasets, produced by :py:meth`Dataset.union` and Dataset.zip(), are not lineage-serializable.

Returns

Serialized bytes containing the lineage of this dataset.

DeveloperAPI: This API may change across minor Ray releases.