Note

From Ray 2.6.0 onwards, RLlib is adopting a new stack for training and model customization, gradually replacing the ModelV2 API and some convoluted parts of Policy API with the RLModule API. Click here for details.

Note

This doc is related to the RLModule API and therefore experimental.

Catalog API#

Basic usage#

Use the following basic API to get a default encoder or action distribution out of Catalog. To change the catalog behavior, modify the following methods. Algorithm-specific implementations of Catalog have additional methods, for example, for building heads.

Catalog

Describes the sub-module-architectures to be used in RLModules.

Catalog.build_encoder

Builds the encoder.

Catalog.get_action_dist_cls

Get the action distribution class.

Catalog.get_tokenizer_config

Returns a tokenizer config for the given space.

Advanced usage#

The following methods and attributes are used internally by the Catalog to build the default models. Only override them when you need more granular control.

Catalog.latent_dims

Returns the latent dimensions of the encoder.

Catalog._determine_components_hook

Decision tree hook for subclasses to override.

Catalog._get_encoder_config

Returns an EncoderConfig for the given input_space and model_config_dict.

Catalog._get_dist_cls_from_action_space

Returns a distribution class for the given action space.