VectorEnv API

rllib.env.vector_env.VectorEnv

class ray.rllib.env.vector_env.VectorEnv(observation_space: <Mock name='mock.Space' id='139804197671056'>, action_space: <Mock name='mock.Space' id='139804197671056'>, num_envs: int)[source]

An environment that supports batch evaluation using clones of sub-envs.

__init__(observation_space: <Mock name='mock.Space' id='139804197671056'>, action_space: <Mock name='mock.Space' id='139804197671056'>, num_envs: int)[source]

Initializes a VectorEnv instance.

Parameters
  • observation_space – The observation Space of a single sub-env.

  • action_space – The action Space of a single sub-env.

  • num_envs – The number of clones to make of the given sub-env.

static vectorize_gym_envs(make_env: Optional[Callable[[int], Any]] = None, existing_envs: Optional[List[<Mock name='mock.Env' id='139804191431376'>]] = None, num_envs: int = 1, action_space: Optional[<Mock name='mock.Space' id='139804197671056'>] = None, observation_space: Optional[<Mock name='mock.Space' id='139804197671056'>] = None, env_config=None, policy_config=None)ray.rllib.env.vector_env._VectorizedGymEnv[source]

Translates any given gym.Env(s) into a VectorizedEnv object.

Parameters
  • make_env – Factory that produces a new gym.Env taking the sub-env’s vector index as only arg. Must be defined if the number of existing_envs is less than num_envs.

  • existing_envs – Optional list of already instantiated sub environments.

  • num_envs – Total number of sub environments in this VectorEnv.

  • action_space – The action space. If None, use existing_envs[0]’s action space.

  • observation_space – The observation space. If None, use existing_envs[0]’s action space.

Returns

The resulting _VectorizedGymEnv object (subclass of VectorEnv).

vector_reset() → List[Any][source]

Resets all sub-environments.

Returns

List of observations from each environment.

reset_at(index: Optional[int] = None) → Any[source]

Resets a single environment.

Parameters

index – An optional sub-env index to reset.

Returns

Observations from the reset sub environment.

vector_step(actions: List[Any]) → Tuple[List[Any], List[float], List[bool], List[dict]][source]

Performs a vectorized step on all sub environments using actions.

Parameters

actions – List of actions (one for each sub-env).

Returns

A tuple consisting of 1) New observations for each sub-env. 2) Reward values for each sub-env. 3) Done values for each sub-env. 4) Info values for each sub-env.

get_sub_environments() → List[Any][source]

Returns the underlying sub environments.

Returns

List of all underlying sub environments.

try_render_at(index: Optional[int] = None) → Optional[numpy.ndarray][source]

Renders a single environment.

Parameters

index – An optional sub-env index to render.

Returns

Either a numpy RGB image (shape=(w x h x 3) dtype=uint8) or None in case rendering is handled directly by this method.

to_base_env(make_env: Callable[[int], Any] = None, num_envs: int = 1, remote_envs: bool = False, remote_env_batch_wait_ms: int = 0)ray.rllib.env.base_env.BaseEnv[source]

Converts an RLlib MultiAgentEnv into a BaseEnv object.

The resulting BaseEnv is always vectorized (contains n sub-environments) to support batched forward passes, where n may also be 1. BaseEnv also supports async execution via the poll and send_actions methods and thus supports external simulators.

Parameters
  • make_env – A callable taking an int as input (which indicates the number of individual sub-environments within the final vectorized BaseEnv) and returning one individual sub-environment.

  • num_envs – The number of sub-environments to create in the resulting (vectorized) BaseEnv. The already existing env will be one of the num_envs.

  • remote_envs – Whether each sub-env should be a @ray.remote actor. You can set this behavior in your config via the remote_worker_envs=True option.

  • remote_env_batch_wait_ms – The wait time (in ms) to poll remote sub-environments for, if applicable. Only used if remote_envs is True.

Returns

The resulting BaseEnv object.

Gym.Env to VectorEnv

Internally, RLlib uses the following wrapper class to convert your provided gym.Env class into a VectorEnv first. After that, RLlib will convert the resulting objects into a BaseEnv.

class ray.rllib.env.vector_env._VectorizedGymEnv(make_env: Optional[Callable[[int], Any]] = None, existing_envs: Optional[List[<Mock name='mock.Env' id='139804191431376'>]] = None, num_envs: int = 1, *, observation_space: Optional[<Mock name='mock.Space' id='139804197671056'>] = None, action_space: Optional[<Mock name='mock.Space' id='139804197671056'>] = None, env_config=None, policy_config=None)[source]

Internal wrapper to translate any gym.Envs into a VectorEnv object.

__init__(make_env: Optional[Callable[[int], Any]] = None, existing_envs: Optional[List[<Mock name='mock.Env' id='139804191431376'>]] = None, num_envs: int = 1, *, observation_space: Optional[<Mock name='mock.Space' id='139804197671056'>] = None, action_space: Optional[<Mock name='mock.Space' id='139804197671056'>] = None, env_config=None, policy_config=None)[source]

Initializes a _VectorizedGymEnv object.

Parameters
  • make_env – Factory that produces a new gym.Env taking the sub-env’s vector index as only arg. Must be defined if the number of existing_envs is less than num_envs.

  • existing_envs – Optional list of already instantiated sub environments.

  • num_envs – Total number of sub environments in this VectorEnv.

  • action_space – The action space. If None, use existing_envs[0]’s action space.

  • observation_space – The observation space. If None, use existing_envs[0]’s action space.

vector_reset()[source]

Resets all sub-environments.

Returns

List of observations from each environment.

reset_at(index: Optional[int] = None) → Any[source]

Resets a single environment.

Parameters

index – An optional sub-env index to reset.

Returns

Observations from the reset sub environment.

vector_step(actions)[source]

Performs a vectorized step on all sub environments using actions.

Parameters

actions – List of actions (one for each sub-env).

Returns

A tuple consisting of 1) New observations for each sub-env. 2) Reward values for each sub-env. 3) Done values for each sub-env. 4) Info values for each sub-env.

get_sub_environments()[source]

Returns the underlying sub environments.

Returns

List of all underlying sub environments.

try_render_at(index: Optional[int] = None)[source]

Renders a single environment.

Parameters

index – An optional sub-env index to render.

Returns

Either a numpy RGB image (shape=(w x h x 3) dtype=uint8) or None in case rendering is handled directly by this method.