ray.data.Dataset.serialize_lineage
ray.data.Dataset.serialize_lineage#
- Dataset.serialize_lineage() bytes [source]#
Serialize this dataset’s lineage, not the actual data or the existing data futures, to bytes that can be stored and later deserialized, possibly on a different cluster.
Note that this will drop all computed data, and that everything will be recomputed from scratch after deserialization.
Use
Dataset.deserialize_lineage()
to deserialize the serialized bytes returned from this method into a Dataset.Note
Unioned and zipped datasets, produced by :py:meth`Dataset.union` and
Dataset.zip()
, are not lineage-serializable.- Returns
Serialized bytes containing the lineage of this dataset.
DeveloperAPI: This API may change across minor Ray releases.