ray.rllib.core.rl_module.multi_rl_module.MultiRLModuleSpec#

class ray.rllib.core.rl_module.multi_rl_module.MultiRLModuleSpec(multi_rl_module_class: ~typing.Type[~ray.rllib.core.rl_module.multi_rl_module.MultiRLModule] = <class 'ray.rllib.core.rl_module.multi_rl_module.MultiRLModule'>, observation_space: gymnasium.Space | None = None, action_space: gymnasium.Space | None = None, inference_only: bool | None = None, model_config: dict | None = None, rl_module_specs: ~ray.rllib.core.rl_module.rl_module.RLModuleSpec | ~typing.Dict[str, ~ray.rllib.core.rl_module.rl_module.RLModuleSpec] = None, load_state_path: str | None = None, modules_to_load: ~typing.Set[str] | None = None, module_specs: ~ray.rllib.core.rl_module.rl_module.RLModuleSpec | ~typing.Dict[str, ~ray.rllib.core.rl_module.rl_module.RLModuleSpec] | None = None)[source]#

A utility spec class to make it constructing MultiRLModules easier.

Users can extend this class to modify the behavior of base class. For example to share neural networks across the modules, the build method can be overridden to create the shared module first and then pass it to custom module classes that would then use it as a shared module.

Parameters:

multi_rl_module_class – The class of the MultiRLModule to construct. By default, this is the base MultiRLModule class.
observation_space – Optional global observation space for the MultiRLModule. Useful for shared network components that live only inside the MultiRLModule and don’t have their own ModuleID and own RLModule within self._rl_modules.
action_space – Optional global action space for the MultiRLModule. Useful for shared network components that live only inside the MultiRLModule and don’t have their own ModuleID and own RLModule within self._rl_modules.
inference_only – An optional global inference_only flag. If not set (None by default), considers the MultiRLModule to be inference_only=True, only if all submodules also have their own inference_only flags set to True.
model_config – An optional global model_config dict. Useful to configure shared network components that only live inside the MultiRLModule and don’t have their own ModuleID and own RLModule within self._rl_modules.
rl_module_specs – The module specs for each individual module. It can be either a RLModuleSpec used for all module_ids or a dictionary mapping from module IDs to RLModuleSpecs for each individual module.
load_state_path – The path to the module state to load from. NOTE: This must be an absolute path. NOTE: If the load_state_path of this spec is set, and the load_state_path of one of the RLModuleSpecs’ is also set, the weights of that RL Module will be loaded from the path specified in the RLModuleSpec. This is useful if you want to load the weights of a MultiRLModule and also manually load the weights of some of the RL modules within that MultiRLModule from other checkpoints.
modules_to_load – A set of module ids to load from the checkpoint. This is only used if load_state_path is set. If this is None, all modules are loaded.

PublicAPI (alpha): This API is in alpha and may change before becoming stable.

Methods

`add_modules`	Add new module specs to the spec or updates existing ones.
`as_multi_rl_module_spec`	Returns self in order to match `RLModuleSpec.as_multi_rl_module_spec()`.
`build`	Builds either the multi-agent module or the single-agent module.
`from_dict`	Creates a MultiRLModuleSpec from a dictionary.
`from_module`	Creates a MultiRLModuleSpec from a MultiRLModule.
`remove_modules`	Removes the provided ModuleIDs from this MultiRLModuleSpec.
`to_dict`	Converts the MultiRLModuleSpec to a dictionary.
`update`	Updates this spec with the other spec.

Attributes

`action_space`
`inference_only`
`load_state_path`
`model_config`
`module_specs`
`modules_to_load`
`observation_space`
`rl_module_specs`