ray.rllib.core.rl_module.marl_module.MultiAgentRLModule#

class ray.rllib.core.rl_module.marl_module.MultiAgentRLModule(*args, **kwargs)[source]#

Bases: RLModule

Base class for multi-agent RLModules.

This class holds a mapping from module_ids to the underlying RLModules. It provides a convenient way of accessing each individual module, as well as accessing all of them with only one API call. Whether or not a given module is trainable is determined by the caller of this class (not the instance of this class itself).

The extension of this class can include any arbitrary neural networks as part of the multi-agent module. For example, a multi-agent module can include a shared encoder network that is used by all the individual RLModules. It is up to the user to decide how to implement this class.

The default implementation assumes the data communicated as input and output of the APIs in this class are MultiAgentBatch types. The MultiAgentRLModule simply loops through each module_id, and runs the forward pass of the corresponding RLModule object with the associated SampleBatch within the MultiAgentBatch. It also assumes that the underlying RLModules do not share any parameters or communication with one another. The behavior of modules with such advanced communication would be undefined by default. To share parameters or communication between the underlying RLModules, you should implement your own MultiAgentRLModule subclass.

PublicAPI (alpha): This API is in alpha and may change before becoming stable.

Methods

add_module

Adds a module at run time to the multi-agent module.

as_multi_agent

Returns a multi-agent wrapper around this module.

foreach_module

Calls the given function with each (module_id, module).

forward_exploration

Forward-pass during exploration, called from the sampler.

forward_inference

Forward-pass during evaluation, called from the sampler.

forward_train

Forward-pass during training called from the learner.

get_exploration_action_dist_cls

Returns the action distribution class for this RLModule used for exploration.

get_inference_action_dist_cls

Returns the action distribution class for this RLModule used for inference.

get_state

Returns the state of the multi-agent module.

get_train_action_dist_cls

Returns the action distribution class for this RLModule used for training.

input_specs_exploration

Returns the input specs of the forward_exploration method.

input_specs_inference

Returns the input specs of the forward_inference method.

input_specs_train

Returns the input specs of the forward_train method.

keys

Returns a keys view over the module IDs in this MultiAgentRLModule.

load_state

Loads the weights of an MultiAgentRLModule from dir.

remove_module

Removes a module at run time from the multi-agent module.

save_state

Saves the weights of this MultiAgentRLModule to dir.

set_state

Sets the state of the multi-agent module.

setup

Sets up the underlying RLModules.

unwrapped

Returns the underlying module if this module is a wrapper.

update_default_view_requirements

Updates default view requirements with the view requirements of this module.

Attributes

framework