ray.data.Dataset.has_serializable_lineage#

Dataset.has_serializable_lineage() bool[source]#

Whether this dataset’s lineage is able to be serialized for storage and later deserialized, possibly on a different cluster.

Only datasets that are created from data that we know will still exist at deserialization time, e.g. data external to this Ray cluster such as persistent cloud object stores, support lineage-based serialization. All of the ray.data.read_*() APIs support lineage-based serialization.

Examples

>>> import ray
>>> ray.data.from_items(list(range(10))).has_serializable_lineage()
False
>>> ray.data.read_csv("s3://anonymous@ray-example-data/iris.csv").has_serializable_lineage()
True