Ray 2.10.0 introduces the alpha stage of RLlib’s “new API stack”. The Ray Team plans to transition algorithms, example scripts, and documentation to the new code base thereby incrementally replacing the “old API stack” (e.g., ModelV2, Policy, RolloutWorker) throughout the subsequent minor releases leading up to Ray 3.0.

Note, however, that so far only PPO (single- and multi-agent) and SAC (single-agent only) support the “new API stack” and continue to run by default with the old APIs. You can continue to use the existing custom (old stack) classes.

See here for more details on how to use the new API stack.


This doc is related to RLlib’s new API stack and therefore experimental.

Catalog API#

Basic usage#

Use the following basic API to get a default encoder or action distribution out of Catalog. To change the catalog behavior, modify the following methods. Algorithm-specific implementations of Catalog have additional methods, for example, for building heads.


Describes the sub-module-architectures to be used in RLModules.


Builds the encoder.


Get the action distribution class.


Returns a tokenizer config for the given space.

Advanced usage#

The following methods and attributes are used internally by the Catalog to build the default models. Only override them when you need more granular control.


Returns the latent dimensions of the encoder.


Decision tree hook for subclasses to override.


Returns an EncoderConfig for the given input_space and model_config_dict.


Returns a distribution class for the given action space.