ray.rllib.policy.sample_batch.SampleBatch#

class ray.rllib.policy.sample_batch.SampleBatch(*args, **kwargs)[source]#

Bases: dict

Wrapper around a dictionary with string keys and array-like values.

For example, {“obs”: [1, 2, 3], “reward”: [0, -1, 1]} is a batch of three samples, each with an “obs” and “reward” attribute.

Methods

__init__

Constructs a sample batch (same params as dict constructor).

agent_steps

Returns the same as len(self) (number of steps in this batch).

as_multi_agent

Returns the respective MultiAgentBatch

clear

columns

Returns a list of the batch-data in the specified columns.

compress

Compresses the data buffers (by column) in place.

concat

Concatenates other to this one and returns a new SampleBatch.

copy

Creates a deep or shallow copy of this SampleBatch and returns it.

decompress_if_needed

Decompresses data buffers (per column if not compressed) in place.

env_steps

Returns the same as len(self) (number of steps in this batch).

fromkeys

Create a new dictionary with keys from iterable and values set to value.

get

Returns one column (by key) from the data or a default value.

get_single_step_input_dict

Creates single ts SampleBatch at given index from self.

is_single_trajectory

Returns True if this SampleBatch only contains one trajectory.

is_terminated_or_truncated

Returns True if self is either terminated or truncated at idx -1.

items

keys

pop

If the key is not found, return the default if given; otherwise, raise a KeyError.

popitem

Remove and return a (key, value) pair as a 2-tuple.

right_zero_pad

Right (adding zeros at end) zero-pads this SampleBatch in-place.

rows

Returns an iterator over data rows, i.e. dicts with column values.

set_get_interceptor

Sets a function to be called on every getitem.

set_training

Sets the is_training flag for this SampleBatch.

setdefault

Insert key with a value of default if key is not in the dictionary.

shuffle

Shuffles the rows of this batch in-place.

size_bytes

Returns sum over number of bytes of all data buffers.

slice

Returns a slice of the row data of this batch (w/o copying).

split_by_episode

Splits by eps_id column and returns list of new batches.

timeslices

Returns SampleBatches, each one representing a k-slice of this one.

to_device

TODO: transfer batch to given device as framework tensor.

update

If E is present and has a .keys() method, then does: for k in E: D[k] = E[k] If E is present and lacks a .keys() method, then does: for k, v in E: D[k] = v In either case, this is followed by: for k in F: D[k] = F[k]

values

Attributes

ACTIONS

ACTION_DIST

ACTION_DIST_INPUTS

ACTION_LOGP

ACTION_PROB

AGENT_INDEX

ATTENTION_MASKS

CUR_OBS

DONES

ENV_ID

EPS_ID

INFOS

NEXT_OBS

OBS

OBS_EMBEDS

PREV_ACTIONS

PREV_REWARDS

RETURNS_TO_GO

REWARDS

SEQ_LENS

T

TERMINATEDS

TRUNCATEDS

UNROLL_ID

VALUES_BOOTSTRAPPED

VF_PREDS

is_training