ray.rllib.core.rl_module.marl_module.MultiAgentRLModuleSpec#

class ray.rllib.core.rl_module.marl_module.MultiAgentRLModuleSpec(marl_module_class: ~typing.Type[~ray.rllib.core.rl_module.marl_module.MultiAgentRLModule] = <class 'ray.rllib.core.rl_module.marl_module.MultiAgentRLModule'>, module_specs: ~ray.rllib.core.rl_module.rl_module.SingleAgentRLModuleSpec | ~typing.Dict[str, ~ray.rllib.core.rl_module.rl_module.SingleAgentRLModuleSpec] = None, load_state_path: str | None = None, modules_to_load: ~typing.Set[str] | None = None)[source]#

A utility spec class to make it constructing MARL modules easier.

Users can extend this class to modify the behavior of base class. For example to share neural networks across the modules, the build method can be overriden to create the shared module first and then pass it to custom module classes that would then use it as a shared module.

Parameters:
  • marl_module_class – The class of the multi-agent RLModule to construct. By default it is set to MultiAgentRLModule class. This class simply loops throught each module and calls their foward methods.

  • module_specs – The module specs for each individual module. It can be either a SingleAgentRLModuleSpec used for all module_ids or a dictionary mapping from module IDs to SingleAgentRLModuleSpecs for each individual module.

  • load_state_path – The path to the module state to load from. NOTE: This must be an absolute path. NOTE: If the load_state_path of this spec is set, and the load_state_path of one of the SingleAgentRLModuleSpecs’ is also set, the weights of that RL Module will be loaded from the path specified in the SingleAgentRLModuleSpec. This is useful if you want to load the weights of a MARL module and also manually load the weights of some of the RL modules within that MARL module from other checkpoints.

  • modules_to_load – A set of module ids to load from the checkpoint. This is only used if load_state_path is set. If this is None, all modules are loaded.

PublicAPI (alpha): This API is in alpha and may change before becoming stable.

Methods

add_modules

Add new module specs to the spec or updates existing ones.

as_multi_agent

Returns self to match SingleAgentRLModuleSpec.as_multi_agent().

build

Builds either the multi-agent module or the single-agent module.

from_dict

Creates a MultiAgentRLModuleSpec from a dictionary.

from_module

Creates a MultiAgentRLModuleSpec from a MultiAgentRLModule.

get_marl_config

Returns the MultiAgentRLModuleConfig for this spec.

to_dict

Converts the MultiAgentRLModuleSpec to a dictionary.

update

Updates this spec with the other spec.

Attributes

load_state_path

module_specs

modules_to_load