VectorEnv API¶

rllib.env.vector_env.VectorEnv¶

class ray.rllib.env.vector_env.VectorEnv(observation_space: <MagicMock name='mock.Space' id='139805173999760'>, action_space: <MagicMock name='mock.Space' id='139805173999760'>, num_envs: int)[source]¶

An environment that supports batch evaluation using clones of sub-envs.

__init__(observation_space: <MagicMock name='mock.Space' id='139805173999760'>, action_space: <MagicMock name='mock.Space' id='139805173999760'>, num_envs: int)[source]¶

Initializes a VectorEnv instance.

Parameters

observation_space – The observation Space of a single sub-env.
action_space – The action Space of a single sub-env.
num_envs – The number of clones to make of the given sub-env.

static vectorize_gym_envs(make_env: Optional[Callable[[int], Any]] = None, existing_envs: Optional[List[<MagicMock name='mock.Env' id='139806551590480'>]] = None, num_envs: int = 1, action_space: Optional[<MagicMock name='mock.Space' id='139805173999760'>] = None, observation_space: Optional[<MagicMock name='mock.Space' id='139805173999760'>] = None, restart_failed_sub_environments: bool = False, env_config=None, policy_config=None) → ray.rllib.env.vector_env._VectorizedGymEnv[source]¶

Translates any given gym.Env(s) into a VectorizedEnv object.

Parameters

make_env – Factory that produces a new gym.Env taking the sub-env’s vector index as only arg. Must be defined if the number of existing_envs is less than num_envs.
existing_envs – Optional list of already instantiated sub environments.
num_envs – Total number of sub environments in this VectorEnv.
action_space – The action space. If None, use existing_envs[0]’s action space.
observation_space – The observation space. If None, use existing_envs[0]’s observation space.
restart_failed_sub_environments – If True and any sub-environment (within a vectorized env) throws any error during env stepping, the Sampler will try to restart the faulty sub-environment. This is done without disturbing the other (still intact) sub-environment and without the RolloutWorker crashing.

Returns

The resulting _VectorizedGymEnv object (subclass of VectorEnv).

vector_reset() → List[Any][source]¶

Resets all sub-environments.

Returns: List of observations from each environment.

reset_at(index: Optional[int] = None) → Any[source]¶

Resets a single sub-environment.

Parameters: index – An optional sub-env index to reset.
Returns: Observations from the reset sub environment.

restart_at(index: Optional[int] = None) → None[source]¶

Restarts a single sub-environment.

Parameters: index – An optional sub-env index to restart.

vector_step(actions: List[Any]) → Tuple[List[Any], List[float], List[bool], List[dict]][source]¶

Performs a vectorized step on all sub environments using actions.

Parameters: actions – List of actions (one for each sub-env).
Returns: A tuple consisting of 1) New observations for each sub-env. 2) Reward values for each sub-env. 3) Done values for each sub-env. 4) Info values for each sub-env.

get_sub_environments() → List[Any][source]¶

Returns the underlying sub environments.

Returns: List of all underlying sub environments.

try_render_at(index: Optional[int] = None) → Optional[numpy.ndarray][source]¶

Renders a single environment.

Parameters: index – An optional sub-env index to render.
Returns: Either a numpy RGB image (shape=(w x h x 3) dtype=uint8) or None in case rendering is handled directly by this method.

to_base_env(make_env: Optional[Callable[[int], Any]] = None, num_envs: int = 1, remote_envs: bool = False, remote_env_batch_wait_ms: int = 0, restart_failed_sub_environments: bool = False) → ray.rllib.env.base_env.BaseEnv[source]¶

Converts an RLlib MultiAgentEnv into a BaseEnv object.

The resulting BaseEnv is always vectorized (contains n sub-environments) to support batched forward passes, where n may also be 1. BaseEnv also supports async execution via the poll and send_actions methods and thus supports external simulators.

Parameters

make_env – A callable taking an int as input (which indicates the number of individual sub-environments within the final vectorized BaseEnv) and returning one individual sub-environment.
num_envs – The number of sub-environments to create in the resulting (vectorized) BaseEnv. The already existing env will be one of the num_envs.
remote_envs – Whether each sub-env should be a @ray.remote actor. You can set this behavior in your config via the remote_worker_envs=True option.
remote_env_batch_wait_ms – The wait time (in ms) to poll remote sub-environments for, if applicable. Only used if remote_envs is True.

Returns

The resulting BaseEnv object.

Gym.Env to VectorEnv¶

Internally, RLlib uses the following wrapper class to convert your provided gym.Env class into a VectorEnv first. After that, RLlib will convert the resulting objects into a BaseEnv.

class ray.rllib.env.vector_env._VectorizedGymEnv(make_env: Optional[Callable[[int], Any]] = None, existing_envs: Optional[List[<MagicMock name='mock.Env' id='139806551590480'>]] = None, num_envs: int = 1, *, observation_space: Optional[<MagicMock name='mock.Space' id='139805173999760'>] = None, action_space: Optional[<MagicMock name='mock.Space' id='139805173999760'>] = None, restart_failed_sub_environments: bool = False, env_config=None, policy_config=None)[source]¶

Internal wrapper to translate any gym.Envs into a VectorEnv object.

__init__(make_env: Optional[Callable[[int], Any]] = None, existing_envs: Optional[List[<MagicMock name='mock.Env' id='139806551590480'>]] = None, num_envs: int = 1, *, observation_space: Optional[<MagicMock name='mock.Space' id='139805173999760'>] = None, action_space: Optional[<MagicMock name='mock.Space' id='139805173999760'>] = None, restart_failed_sub_environments: bool = False, env_config=None, policy_config=None)[source]¶

Initializes a _VectorizedGymEnv object.

Parameters

make_env – Factory that produces a new gym.Env taking the sub-env’s vector index as only arg. Must be defined if the number of existing_envs is less than num_envs.
existing_envs – Optional list of already instantiated sub environments.
num_envs – Total number of sub environments in this VectorEnv.
action_space – The action space. If None, use existing_envs[0]’s action space.
observation_space – The observation space. If None, use existing_envs[0]’s observation space.
restart_failed_sub_environments – If True and any sub-environment (within a vectorized env) throws any error during env stepping, we will try to restart the faulty sub-environment. This is done without disturbing the other (still intact) sub-environments.

vector_reset()[source]¶

Resets all sub-environments.

Returns: List of observations from each environment.

reset_at(index: Optional[int] = None) → Any[source]¶

Resets a single sub-environment.

Parameters: index – An optional sub-env index to reset.
Returns: Observations from the reset sub environment.

restart_at(index: Optional[int] = None) → None[source]¶

Restarts a single sub-environment.

Parameters: index – An optional sub-env index to restart.

vector_step(actions)[source]¶

Performs a vectorized step on all sub environments using actions.

Parameters: actions – List of actions (one for each sub-env).
Returns: A tuple consisting of 1) New observations for each sub-env. 2) Reward values for each sub-env. 3) Done values for each sub-env. 4) Info values for each sub-env.

get_sub_environments()[source]¶

Returns the underlying sub environments.

Returns: List of all underlying sub environments.

try_render_at(index: Optional[int] = None)[source]¶

Renders a single environment.

Parameters: index – An optional sub-env index to render.
Returns: Either a numpy RGB image (shape=(w x h x 3) dtype=uint8) or None in case rendering is handled directly by this method.

Ray 3.0.0.dev0

VectorEnv API¶

rllib.env.vector_env.VectorEnv¶

Gym.Env to VectorEnv¶