ray.data.Datasource#
- class ray.data.Datasource[source]#
Interface for defining a custom
Dataset
datasource.To read a datasource into a dataset, use
read_datasource()
. To write to a writable datasource, usewrite_datasource()
.See
RangeDatasource
andDummyOutputDatasource
for examples of how to implement readable and writable datasources.Note
Datasource instances must be serializable, since
write()
is called in remote tasks.Methods
Return a Reader for the given read arguments.
Return an estimate of the in-memory data size, or None if unknown.
Return a human-readable name for this datasource.
Execute the read and return read tasks.
Callback for when a write job completes.
Callback for when a write job fails.
Callback for when a write job starts.
Deprecated: Please implement create_reader() instead.
Write blocks out to the datasource.
Attributes
If
False
, only launch read tasks on the driver's node.