ray.rllib.env.single_agent_episode.SingleAgentEpisode.set_observations#

SingleAgentEpisode.set_observations(*, new_data, at_indices: int | slice | List[int] | None = None, neg_index_as_lookback: bool = False) → None[source]#

Overwrites all or some of this Episode’s observations with the provided data.

Note that an episode’s observation data cannot be written to directly as it is managed by a InfiniteLookbackBuffer object. Normally, individual, current observations are added to the episode either by calling self.add_env_step or more directly (and manually) via self.observations.append|extend(). However, for certain postprocessing steps, the entirety (or a slice) of an episode’s observations might have to be rewritten, which is when self.set_observations() should be used.

Parameters:

  • new_data – The new observation data to overwrite existing data with. This may be a list of individual observation(s) in case this episode is still not numpy’ized yet. In case this episode has already been numpy’ized, this should be (possibly complex) struct matching the observation space and with a batch size of its leafs exactly the size of the to-be-overwritten slice or segment (provided by at_indices).

  • at_indices – A single int is interpreted as one index, which to overwrite with new_data (which is expected to be a single observation). A list of ints is interpreted as a list of indices, all of which to overwrite with new_data (which is expected to be of the same size as len(at_indices)). A slice object is interpreted as a range of indices to be overwritten with new_data (which is expected to be of the same size as the provided slice). Thereby, negative indices by default are interpreted as “before the end” unless the neg_index_as_lookback=True option is used, in which case negative indices are interpreted as “before ts=0”, meaning going back into the lookback buffer.

  • neg_index_as_lookback – If True, negative values in at_indices are interpreted as “before ts=0”, meaning going back into the lookback buffer. For example, an episode with observations = [4, 5, 6, 7, 8, 9], where [4, 5, 6] is the lookback buffer range (ts=0 item is 7), will handle a call to set_observations(individual_observation, -1, neg_index_as_lookback=True) by overwriting the value of 6 in our observations buffer with the provided “individual_observation”.

Raises:

IndexError – If the provided at_indices do not match the size of new_data.