VectorEnv API¶
rllib.env.vector_env.VectorEnv¶
- class ray.rllib.env.vector_env.VectorEnv(observation_space: <MagicMock name='mock.Space' id='140067862243472'>, action_space: <MagicMock name='mock.Space' id='140067862243472'>, num_envs: int)[source]¶
An environment that supports batch evaluation using clones of sub-envs.
- __init__(observation_space: <MagicMock name='mock.Space' id='140067862243472'>, action_space: <MagicMock name='mock.Space' id='140067862243472'>, num_envs: int)[source]¶
Initializes a VectorEnv instance.
- Parameters
observation_space – The observation Space of a single sub-env.
action_space – The action Space of a single sub-env.
num_envs – The number of clones to make of the given sub-env.
- static vectorize_gym_envs(make_env: Optional[Callable[[int], Any]] = None, existing_envs: Optional[List[<MagicMock name='mock.Env' id='140067862289360'>]] = None, num_envs: int = 1, action_space: Optional[<MagicMock name='mock.Space' id='140067862243472'>] = None, observation_space: Optional[<MagicMock name='mock.Space' id='140067862243472'>] = None, env_config=None, policy_config=None) ray.rllib.env.vector_env._VectorizedGymEnv[source]¶
Translates any given gym.Env(s) into a VectorizedEnv object.
- Parameters
make_env – Factory that produces a new gym.Env taking the sub-env’s vector index as only arg. Must be defined if the number of existing_envs is less than num_envs.
existing_envs – Optional list of already instantiated sub environments.
num_envs – Total number of sub environments in this VectorEnv.
action_space – The action space. If None, use existing_envs[0]’s action space.
observation_space – The observation space. If None, use existing_envs[0]’s observation space.
- Returns
The resulting _VectorizedGymEnv object (subclass of VectorEnv).
- vector_reset() List[Any][source]¶
Resets all sub-environments.
- Returns
List of observations from each environment.
- reset_at(index: Optional[int] = None) Any[source]¶
Resets a single environment.
- Parameters
index – An optional sub-env index to reset.
- Returns
Observations from the reset sub environment.
- vector_step(actions: List[Any]) Tuple[List[Any], List[float], List[bool], List[dict]][source]¶
Performs a vectorized step on all sub environments using actions.
- Parameters
actions – List of actions (one for each sub-env).
- Returns
A tuple consisting of 1) New observations for each sub-env. 2) Reward values for each sub-env. 3) Done values for each sub-env. 4) Info values for each sub-env.
- get_sub_environments() List[Any][source]¶
Returns the underlying sub environments.
- Returns
List of all underlying sub environments.
- try_render_at(index: Optional[int] = None) Optional[numpy.ndarray][source]¶
Renders a single environment.
- Parameters
index – An optional sub-env index to render.
- Returns
Either a numpy RGB image (shape=(w x h x 3) dtype=uint8) or None in case rendering is handled directly by this method.
- to_base_env(make_env: Optional[Callable[[int], Any]] = None, num_envs: int = 1, remote_envs: bool = False, remote_env_batch_wait_ms: int = 0) ray.rllib.env.base_env.BaseEnv[source]¶
Converts an RLlib MultiAgentEnv into a BaseEnv object.
The resulting BaseEnv is always vectorized (contains n sub-environments) to support batched forward passes, where n may also be 1. BaseEnv also supports async execution via the poll and send_actions methods and thus supports external simulators.
- Parameters
make_env – A callable taking an int as input (which indicates the number of individual sub-environments within the final vectorized BaseEnv) and returning one individual sub-environment.
num_envs – The number of sub-environments to create in the resulting (vectorized) BaseEnv. The already existing env will be one of the num_envs.
remote_envs – Whether each sub-env should be a @ray.remote actor. You can set this behavior in your config via the remote_worker_envs=True option.
remote_env_batch_wait_ms – The wait time (in ms) to poll remote sub-environments for, if applicable. Only used if remote_envs is True.
- Returns
The resulting BaseEnv object.
Gym.Env to VectorEnv¶
Internally, RLlib uses the following wrapper class to convert your provided gym.Env class into
a VectorEnv first. After that, RLlib will convert the resulting objects into a BaseEnv.
- class ray.rllib.env.vector_env._VectorizedGymEnv(make_env: Optional[Callable[[int], Any]] = None, existing_envs: Optional[List[<MagicMock name='mock.Env' id='140067862289360'>]] = None, num_envs: int = 1, *, observation_space: Optional[<MagicMock name='mock.Space' id='140067862243472'>] = None, action_space: Optional[<MagicMock name='mock.Space' id='140067862243472'>] = None, env_config=None, policy_config=None)[source]¶
Internal wrapper to translate any gym.Envs into a VectorEnv object.
- __init__(make_env: Optional[Callable[[int], Any]] = None, existing_envs: Optional[List[<MagicMock name='mock.Env' id='140067862289360'>]] = None, num_envs: int = 1, *, observation_space: Optional[<MagicMock name='mock.Space' id='140067862243472'>] = None, action_space: Optional[<MagicMock name='mock.Space' id='140067862243472'>] = None, env_config=None, policy_config=None)[source]¶
Initializes a _VectorizedGymEnv object.
- Parameters
make_env – Factory that produces a new gym.Env taking the sub-env’s vector index as only arg. Must be defined if the number of existing_envs is less than num_envs.
existing_envs – Optional list of already instantiated sub environments.
num_envs – Total number of sub environments in this VectorEnv.
action_space – The action space. If None, use existing_envs[0]’s action space.
observation_space – The observation space. If None, use existing_envs[0]’s observation space.
- vector_reset()[source]¶
Resets all sub-environments.
- Returns
List of observations from each environment.
- reset_at(index: Optional[int] = None) Any[source]¶
Resets a single environment.
- Parameters
index – An optional sub-env index to reset.
- Returns
Observations from the reset sub environment.
- vector_step(actions)[source]¶
Performs a vectorized step on all sub environments using actions.
- Parameters
actions – List of actions (one for each sub-env).
- Returns
A tuple consisting of 1) New observations for each sub-env. 2) Reward values for each sub-env. 3) Done values for each sub-env. 4) Info values for each sub-env.