ray.data.Datasource

class ray.data.Datasource

Bases: _DatasourceProjectionPushdownMixin

Interface for defining a custom Dataset datasource.

To read a datasource into a dataset, use read_datasource().

Methods

__init__
create_reader	Deprecated: Implement get_read_tasks() and estimate_inmemory_data_size() instead.
estimate_inmemory_data_size	Return an estimate of the in-memory data size, or None if unknown.
get_current_projection	Retrurns current projection
get_name	Return a human-readable name for this datasource.
get_read_tasks	Execute the read and return read tasks.
prepare_read	Deprecated: Implement get_read_tasks() and estimate_inmemory_data_size() instead.
supports_projection_pushdown	Returns True in case Datasource supports projection operation being pushed down into the reading layer

Attributes

should_create_reader
supports_distributed_reads	If False, only launch read tasks on the driver's node.