ray.data.datasource.BaseFileMetadataProvider

class ray.data.datasource.BaseFileMetadataProvider

Bases: FileMetadataProvider

Abstract callable that provides metadata for FileBasedDatasource implementations that reuse the base prepare_read() method.

Also supports file and file size discovery in input directory paths.

Current subclasses:

• DefaultFileMetadataProvider

DeveloperAPI: This API may change across minor Ray releases.

Methods

__init__
expand_paths	Expands all paths into concrete file paths by walking directories.