ray.rllib.algorithms.algorithm.Algorithm.add_module#

Algorithm.add_module(module_id: str, module_spec: RLModuleSpec, *, config_overrides: Dict | None = None, new_agent_to_module_mapping_fn: Callable[[Any, SingleAgentEpisode | MultiAgentEpisode], str] | None = None, new_should_module_be_updated: Sequence[str] | Callable[[str, MultiAgentBatch | None], bool] | None = None, add_to_learners: bool = True, add_to_env_runners: bool = True, add_to_eval_env_runners: bool = True) → MultiRLModuleSpec[source]#

Adds a new (single-agent) RLModule to this Algorithm’s MARLModule.

Note that an Algorithm has up to 3 different components to which to add the new module to: The LearnerGroup (with n Learners), the EnvRunnerGroup (with m EnvRunners plus a local one) and - if applicable - the eval EnvRunnerGroup (with o EnvRunners plus a local one).

Parameters:

  • module_id – ID of the RLModule to add to the MARLModule. IMPORTANT: Must not contain characters that are also not allowed in Unix/Win filesystems, such as: <>:"/|?*, or a dot, space or backslash at the end of the ID.

  • module_spec – The SingleAgentRLModuleSpec to use for constructing the new RLModule.

  • config_overrides – The AlgorithmConfig overrides that should apply to the new Module, if any.

  • new_agent_to_module_mapping_fn – An optional (updated) AgentID to ModuleID mapping function to use from here on. Note that already ongoing episodes will not change their mapping but will use the old mapping till the end of the episode.

  • new_should_module_be_updated – An optional sequence of ModuleIDs or a callable taking ModuleID and SampleBatchType and returning whether the ModuleID should be updated (trained). If None, will keep the existing setup in place. RLModules, whose IDs are not in the list (or for which the callable returns False) will not be updated.

  • add_to_learners – Whether to add the new RLModule to the LearnerGroup (with its n Learners).

  • add_to_env_runners – Whether to add the new RLModule to the EnvRunnerGroup (with its m EnvRunners plus the local one).

  • add_to_eval_env_runners – Whether to add the new RLModule to the eval EnvRunnerGroup (with its o EnvRunners plus the local one).

Returns:

The new MultiAgentRLModuleSpec (after the RLModule has been added).