GravitinoCatalog

class ray.data.GravitinoCatalog(*, uri: str, metalake: str, iceberg_rest_uri: str | None = None, iceberg_rest_catalog_kwargs: Dict[str, Any] | None = None)

Bases: Catalog

Resolve Gravitino tables through their format-specific endpoints.

This adapter maps Gravitino table names to Ray Data reader/writer configuration. Iceberg uses an explicitly configured PyIceberg REST catalog. It performs no metadata requests and does not require the Gravitino Python SDK.

Resolving tables requires PyIceberg 0.10.0 or later for REST authentication, including the default noop authentication manager. Install it with pip install -U "pyiceberg>=0.10.0".

Parameters:

• uri (str) – Gravitino main API base URI, for example "http://gravitino:8090". Reserved for future format-specific resolution; Iceberg does not contact this endpoint.

• metalake (str) – Non-empty metalake name retained for future format-specific resolution. The Iceberg REST endpoint must serve this metalake; this adapter does not verify that binding.

• iceberg_rest_uri (str | None) – Explicit Iceberg REST endpoint, for example "http://gravitino:9001/iceberg/". Required when resolving Iceberg tables; endpoint discovery is not implemented.

• iceberg_rest_catalog_kwargs (Dict[str, Any] | None) – Optional PyIceberg authentication and storage properties for the REST endpoint. Supply authentication explicitly in an auth dictionary with a non-empty type and all settings required by that authentication manager. If auth, token, and credential are all absent, the provider sets auth={"type": "noop"}. Legacy top-level token or credential properties require an explicit auth dictionary and do not override that authentication manager. For a static Bearer token, use auth={"type": "noop"} with header.Authorization="Bearer <token>". Other properties still follow PyIceberg’s configuration merging rules; this is not general configuration isolation. Don’t rely on configuration files or environment variables to supply authentication settings.

Example

>>> import ray
>>> catalog = ray.data.GravitinoCatalog(
...     uri="http://gravitino:8090",
...     metalake="lakehouse",
...     iceberg_rest_uri="http://gravitino:9001/iceberg/",
... )
>>> ds = ray.data.read_iceberg(
...     table_identifier="iceberg_catalog.sales.orders", catalog=catalog
... )

Note

The initial scope is existing-table reads and same-schema append writes. Other native Iceberg write modes have not been validated through this provider. Delta and Parquet access are unsupported. The REST service’s dynamic catalog configuration binds the metalake; the catalog component of catalog.schema.table selects its warehouse. Table existence is checked during actual REST operations. Catalog authorization depends on the REST service’s deployment and configuration. For Gravitino 1.3.1, catalog access control requires auxiliary REST mode, gravitino.authorization.enable=true, and the dynamic catalog configuration provider. Standalone mode doesn’t provide catalog authorization. The default representation hides authentication properties, but this object is not a credential-free serialization boundary. Ray execution state may carry the configuration needed by PyIceberg tasks.

PublicAPI (alpha): This API is in alpha and may change before becoming stable.

Methods

resolve

Resolve a Gravitino table to an Iceberg REST configuration.

Attributes

iceberg_rest_catalog_kwargs
iceberg_rest_uri
uri
metalake