Note

Ray 2.10.0 introduces the alpha stage of RLlib’s “new API stack”. The team is currently transitioning algorithms, example scripts, and documentation to the new code base throughout the subsequent minor releases leading up to Ray 3.0.

See here for more details on how to activate and use the new API stack.

RLModule API#

RL Module specifications and configurations#

Single Agent#

`RLModuleSpec`	Utility spec class to make constructing RLModules (in single-agent case) easier.
`RLModuleSpec.build`	Builds the RLModule from this spec.
`RLModuleSpec.get_rl_module_config`

RLModule Configuration#

`RLModuleConfig`
`RLModuleConfig.to_dict`
`RLModuleConfig.from_dict`
`RLModuleConfig.get_catalog`

Multi RLModule (multi-agent)#

`MultiRLModuleSpec`	A utility spec class to make it constructing MultiRLModules easier.
`MultiRLModuleSpec.build`	Builds either the MultiRLModule or a (single) sub-RLModule under `module_id`.
`MultiRLModuleSpec.get_multi_rl_module_config`

RL Module API#

Constructor#

`RLModule`	Base class for RLlib modules.
`RLModule.as_multi_rl_module`	Returns a multi-agent wrapper around this module.

Forward methods#

`forward_train`	DO NOT OVERRIDE! Forward-pass during training called from the learner.
`forward_exploration`	DO NOT OVERRIDE! Forward-pass during exploration, called from the sampler.
`forward_inference`	DO NOT OVERRIDE! Forward-pass during evaluation, called from the sampler.
`_forward_train`	Forward-pass used before the loss computation (training).
`_forward_exploration`	Forward-pass used for action computation with exploration behavior.
`_forward_inference`	Forward-pass used for action computation without exploration behavior.

IO specifications#

`input_specs_inference`	Returns the input specs of the forward_inference method.
`input_specs_exploration`	Returns the input specs of the forward_exploration method.
`input_specs_train`	Returns the input specs of the forward_train method.
`output_specs_inference`
`output_specs_exploration`
`output_specs_train`	Returns the output specs of the forward_train method.

Saving and Loading#

`get_state`	Returns the state dict of the module.
`set_state`	Sets the implementing class' state to the given state dict.
`save_to_path`	Saves the state of the implementing class (or `state`) to `path`.
`restore_from_path`	Restores the state of the implementing class from the given path.
`from_checkpoint`	Creates a new Checkpointable instance from the given location and returns it.

Multi Agent RL Module API#

Constructor#

`MultiRLModule`	Base class for an RLModule that contains n sub-RLModules.
`MultiRLModule.setup`	Sets up the underlying, individual RLModules.
`MultiRLModule.as_multi_rl_module`	Returns self in order to match `RLModule.as_multi_rl_module()` behavior.

Modifying the underlying RL modules#

`add_module`	Adds a module at run time to the multi-agent module.
`remove_module`	Removes a module at runtime from the multi-agent module.

Saving and Loading#

`save_to_path`	Saves the state of the implementing class (or `state`) to `path`.
`restore_from_path`	Restores the state of the implementing class from the given path.