ray.data.datasource.NumpyDatasource#

class ray.data.datasource.NumpyDatasource(*args, **kwds)[source]#

Numpy datasource, for reading and writing Numpy files.

Examples

>>> import ray
>>> from ray.data.datasource import NumpyDatasource
>>> source = NumpyDatasource() 
>>> ray.data.read_datasource( 
...     source, paths="/path/to/dir").take()
[array([0., 1., 2.]), ...]

PublicAPI: This API is stable across Ray releases.

__init__()#

Methods

__init__()

create_reader(**kwargs)

Return a Reader for the given read arguments.

do_write(blocks, metadata, path, dataset_uuid)

Creates and returns write tasks for a file-based datasource.

file_extension_filter()

on_write_complete(write_results, **kwargs)

Callback for when a write job completes.

on_write_failed(write_results, error, **kwargs)

Callback for when a write job fails.

prepare_read(parallelism, **read_args)

Deprecated: Please implement create_reader() instead.