ray.rllib.env.multi_agent_episode.MultiAgentEpisode.set_extra_model_outputs#

MultiAgentEpisode.set_extra_model_outputs(*, key, new_data: Dict[Hashable, Any], at_indices: int | List[int] | slice | None = None, neg_index_as_lookback: bool = False) → None[source]#

Overwrites all or some of this Episode’s extra model outputs with new_data.

This is a helper method to batch SingleAgentEpisode.set_extra_model_outputs. For more detail, see SingleAgentEpisode.set_extra_model_outputs.

Parameters:

  • key – The key within self.extra_model_outputs to override data on or to insert as a new key into self.extra_model_outputs.

  • new_data – A dict mapping agent IDs to new extra model outputs data. Each value in the dict is the new extra model outputs data to overwrite existing data with. This may be a list of individual reward(s) in case this episode is still not numpy’ized yet. In case this episode has already been numpy’ized, this should be a np.ndarray with a length exactly the size of the to-be-overwritten slice or segment (provided by at_indices).

  • at_indices – A single int is interpreted as one index, which to overwrite with new_data (which is expected to be a single extra model output). A list of ints is interpreted as a list of indices, all of which to overwrite with new_data (which is expected to be of the same size as len(at_indices)). A slice object is interpreted as a range of indices to be overwritten with new_data (which is expected to be of the same size as the provided slice). Thereby, negative indices by default are interpreted as “before the end” unless the neg_index_as_lookback=True option is used, in which case negative indices are interpreted as “before ts=0”, meaning going back into the lookback buffer.

  • neg_index_as_lookback – If True, negative values in at_indices are interpreted as “before ts=0”, meaning going back into the lookback buffer. For example, an episode with extra_model_outputs[key][agent_id] = [4, 5, 6, 7, 8, 9], where [4, 5, 6] is the lookback buffer range (ts=0 item is 7), will handle a call to set_extra_model_outputs(key, individual_output, -1, neg_index_as_lookback=True) by overwriting the value of 6 in our extra_model_outputs[key][agent_id] buffer with the provided “individual_output”.

Raises:

IndexError – If the provided at_indices do not match the size of new_data.