ray.rllib.evaluation.worker_set.WorkerSet.add_policy#

WorkerSet.add_policy(policy_id: str, policy_cls: Type[Policy] | None = None, policy: Policy | None = None, *, observation_space: gymnasium.spaces.Space | None = None, action_space: gymnasium.spaces.Space | None = None, config: AlgorithmConfig | dict | None = None, policy_state: Dict[str, numpy.array | jnp.ndarray | tf.Tensor | torch.Tensor | dict | tuple] | None = None, policy_mapping_fn: Callable[[Any, int], str] | None = None, policies_to_train: Container[str] | Callable[[str, SampleBatch | MultiAgentBatch | None], bool] | None = None, module_spec: SingleAgentRLModuleSpec | None = None, workers: List[EnvRunner | ActorHandle] | None = -1) None[source]#

Adds a policy to this WorkerSet’s workers or a specific list of workers.

Parameters:
  • policy_id – ID of the policy to add.

  • policy_cls – The Policy class to use for constructing the new Policy. Note: Only one of policy_cls or policy must be provided.

  • policy – The Policy instance to add to this WorkerSet. If not None, the given Policy object will be directly inserted into the local worker and clones of that Policy will be created on all remote workers. Note: Only one of policy_cls or policy must be provided.

  • observation_space – The observation space of the policy to add. If None, try to infer this space from the environment.

  • action_space – The action space of the policy to add. If None, try to infer this space from the environment.

  • config – The config object or overrides for the policy to add.

  • policy_state – Optional state dict to apply to the new policy instance, right after its construction.

  • policy_mapping_fn – An optional (updated) policy mapping function to use from here on. Note that already ongoing episodes will not change their mapping but will use the old mapping till the end of the episode.

  • policies_to_train – An optional list of policy IDs to be trained or a callable taking PolicyID and SampleBatchType and returning a bool (trainable or not?). If None, will keep the existing setup in place. Policies, whose IDs are not in the list (or for which the callable returns False) will not be updated.

  • module_spec – In the new RLModule API we need to pass in the module_spec for the new module that is supposed to be added. Knowing the policy spec is not sufficient.

  • workers – A list of EnvRunner/ActorHandles (remote EnvRunners) to add this policy to. If defined, will only add the given policy to these workers.

Raises:

KeyError – If the given policy_id already exists in this WorkerSet.