ray.data.datasource.DefaultFileMetadataProvider#

class ray.data.datasource.DefaultFileMetadataProvider[source]#

Bases: ray.data.datasource.file_meta_provider.BaseFileMetadataProvider

Default metadata provider for FileBasedDatasource implementations that reuse the base prepare_read method.

Calculates block size in bytes as the sum of its constituent file sizes, and assumes a fixed number of rows per file.

DeveloperAPI: This API may change across minor Ray releases.

Methods

__init__