flatten_inputs_to_1d_tensor#

ray.rllib.utils.numpy.flatten_inputs_to_1d_tensor(inputs: NDArray[Any] | jnp.ndarray | tf.Tensor | torch.Tensor | dict | tuple, spaces_struct: gymnasium.spaces.Space | Dict[str, gymnasium.spaces.Space] | Tuple[gymnasium.spaces.Space, ...] | None = None, time_axis: bool = False, batch_axis: bool = True) → NDArray[Any] | jnp.ndarray | tf.Tensor | torch.Tensor[source]#

Flattens arbitrary input structs according to the given spaces struct.

Returns a single 1D tensor resulting from the different input components’ values.

Thereby: - Boxes (any shape) get flattened to (B, [T]?, -1). Note that image boxes are not treated differently from other types of Boxes and get flattened as well. - Discrete (int) values are one-hot’d, e.g. a batch of [1, 0, 3] (B=3 with Discrete(4) space) results in [[0, 1, 0, 0], [1, 0, 0, 0], [0, 0, 0, 1]]. - MultiDiscrete values are multi-one-hot’d, e.g. a batch of [[0, 2], [1, 4]] (B=2 with MultiDiscrete([2, 5]) space) results in [[1, 0, 0, 0, 1, 0, 0], [0, 1, 0, 0, 0, 0, 1]].

Parameters:
  • inputs (NDArray[Any] | jnp.ndarray | tf.Tensor | torch.Tensor | dict | tuple) – The inputs to be flattened.

  • spaces_struct (gymnasium.spaces.Space | Dict[str, gymnasium.spaces.Space] | Tuple[gymnasium.spaces.Space, ...] | None) – The (possibly nested) structure of the spaces that inputs belongs to.

  • time_axis (bool) – Whether all inputs have a time-axis (after the batch axis). If True, will keep not only the batch axis (0th), but the time axis (1st) as-is and flatten everything from the 2nd axis up.

  • batch_axis (bool) – Whether all inputs have a batch axis. If True, will keep that batch axis as-is and flatten everything from the other dims up.

Returns:

A single 1D tensor resulting from concatenating all flattened/one-hot’d input components. Depending on the time_axis flag, the shape is (B, n) or (B, T, n).

Return type:

NDArray[Any] | jnp.ndarray | tf.Tensor | torch.Tensor

Examples

>>> # B=2
>>> import numpy as np
>>> from ray.rllib.utils.numpy import flatten_inputs_to_1d_tensor
>>> from gymnasium.spaces import Discrete, Box
>>> out = flatten_inputs_to_1d_tensor(
...     {
...         "a": np.array([1, 0]),
...         "b": np.array([[[0.0], [0.1]], [[1.0], [1.1]]]),
...     },
...     spaces_struct=dict(a=Discrete(2), b=Box(-1.0, 1.0, shape=(2, 1)))
... )
>>> print(out)
[[0.  1.  0.  0.1]
 [1.  0.  1.  1.1]]
>>> # B=2; T=2
>>> out = flatten_inputs_to_1d_tensor(
...     (np.array([[1, 0], [0, 1]]),
...      np.array([[[0.0, 0.1], [1.0, 1.1]], [[2.0, 2.1], [3.0, 3.1]]])),
...     spaces_struct=tuple([Discrete(2), Box(-1.0, 1.0, shape=(2, ))]),
...     time_axis=True
... )
>>> print(out)
[[[0.  1.  0.  0.1]
  [1.  0.  1.  1.1]]

 [[1.  0.  2.  2.1]
  [0.  1.  3.  3.1]]]