VectorEnv API¶
rllib.env.vector_env.VectorEnv¶
-
class
ray.rllib.env.vector_env.VectorEnv(observation_space: <Mock name='mock.Space' id='139804197671056'>, action_space: <Mock name='mock.Space' id='139804197671056'>, num_envs: int)[source]¶ An environment that supports batch evaluation using clones of sub-envs.
-
__init__(observation_space: <Mock name='mock.Space' id='139804197671056'>, action_space: <Mock name='mock.Space' id='139804197671056'>, num_envs: int)[source]¶ Initializes a VectorEnv instance.
- Parameters
observation_space – The observation Space of a single sub-env.
action_space – The action Space of a single sub-env.
num_envs – The number of clones to make of the given sub-env.
-
static
vectorize_gym_envs(make_env: Optional[Callable[[int], Any]] = None, existing_envs: Optional[List[<Mock name='mock.Env' id='139804191431376'>]] = None, num_envs: int = 1, action_space: Optional[<Mock name='mock.Space' id='139804197671056'>] = None, observation_space: Optional[<Mock name='mock.Space' id='139804197671056'>] = None, env_config=None, policy_config=None) → ray.rllib.env.vector_env._VectorizedGymEnv[source]¶ Translates any given gym.Env(s) into a VectorizedEnv object.
- Parameters
make_env – Factory that produces a new gym.Env taking the sub-env’s vector index as only arg. Must be defined if the number of existing_envs is less than num_envs.
existing_envs – Optional list of already instantiated sub environments.
num_envs – Total number of sub environments in this VectorEnv.
action_space – The action space. If None, use existing_envs[0]’s action space.
observation_space – The observation space. If None, use existing_envs[0]’s action space.
- Returns
The resulting _VectorizedGymEnv object (subclass of VectorEnv).
-
vector_reset() → List[Any][source]¶ Resets all sub-environments.
- Returns
List of observations from each environment.
-
reset_at(index: Optional[int] = None) → Any[source]¶ Resets a single environment.
- Parameters
index – An optional sub-env index to reset.
- Returns
Observations from the reset sub environment.
-
vector_step(actions: List[Any]) → Tuple[List[Any], List[float], List[bool], List[dict]][source]¶ Performs a vectorized step on all sub environments using actions.
- Parameters
actions – List of actions (one for each sub-env).
- Returns
A tuple consisting of 1) New observations for each sub-env. 2) Reward values for each sub-env. 3) Done values for each sub-env. 4) Info values for each sub-env.
-
get_sub_environments() → List[Any][source]¶ Returns the underlying sub environments.
- Returns
List of all underlying sub environments.
-
try_render_at(index: Optional[int] = None) → Optional[numpy.ndarray][source]¶ Renders a single environment.
- Parameters
index – An optional sub-env index to render.
- Returns
Either a numpy RGB image (shape=(w x h x 3) dtype=uint8) or None in case rendering is handled directly by this method.
-
to_base_env(make_env: Callable[[int], Any] = None, num_envs: int = 1, remote_envs: bool = False, remote_env_batch_wait_ms: int = 0) → ray.rllib.env.base_env.BaseEnv[source]¶ Converts an RLlib MultiAgentEnv into a BaseEnv object.
The resulting BaseEnv is always vectorized (contains n sub-environments) to support batched forward passes, where n may also be 1. BaseEnv also supports async execution via the poll and send_actions methods and thus supports external simulators.
- Parameters
make_env – A callable taking an int as input (which indicates the number of individual sub-environments within the final vectorized BaseEnv) and returning one individual sub-environment.
num_envs – The number of sub-environments to create in the resulting (vectorized) BaseEnv. The already existing env will be one of the num_envs.
remote_envs – Whether each sub-env should be a @ray.remote actor. You can set this behavior in your config via the remote_worker_envs=True option.
remote_env_batch_wait_ms – The wait time (in ms) to poll remote sub-environments for, if applicable. Only used if remote_envs is True.
- Returns
The resulting BaseEnv object.
-
Gym.Env to VectorEnv¶
Internally, RLlib uses the following wrapper class to convert your provided gym.Env class into
a VectorEnv first. After that, RLlib will convert the resulting objects into a BaseEnv.
-
class
ray.rllib.env.vector_env._VectorizedGymEnv(make_env: Optional[Callable[[int], Any]] = None, existing_envs: Optional[List[<Mock name='mock.Env' id='139804191431376'>]] = None, num_envs: int = 1, *, observation_space: Optional[<Mock name='mock.Space' id='139804197671056'>] = None, action_space: Optional[<Mock name='mock.Space' id='139804197671056'>] = None, env_config=None, policy_config=None)[source]¶ Internal wrapper to translate any gym.Envs into a VectorEnv object.
-
__init__(make_env: Optional[Callable[[int], Any]] = None, existing_envs: Optional[List[<Mock name='mock.Env' id='139804191431376'>]] = None, num_envs: int = 1, *, observation_space: Optional[<Mock name='mock.Space' id='139804197671056'>] = None, action_space: Optional[<Mock name='mock.Space' id='139804197671056'>] = None, env_config=None, policy_config=None)[source]¶ Initializes a _VectorizedGymEnv object.
- Parameters
make_env – Factory that produces a new gym.Env taking the sub-env’s vector index as only arg. Must be defined if the number of existing_envs is less than num_envs.
existing_envs – Optional list of already instantiated sub environments.
num_envs – Total number of sub environments in this VectorEnv.
action_space – The action space. If None, use existing_envs[0]’s action space.
observation_space – The observation space. If None, use existing_envs[0]’s action space.
-
vector_reset()[source]¶ Resets all sub-environments.
- Returns
List of observations from each environment.
-
reset_at(index: Optional[int] = None) → Any[source]¶ Resets a single environment.
- Parameters
index – An optional sub-env index to reset.
- Returns
Observations from the reset sub environment.
-
vector_step(actions)[source]¶ Performs a vectorized step on all sub environments using actions.
- Parameters
actions – List of actions (one for each sub-env).
- Returns
A tuple consisting of 1) New observations for each sub-env. 2) Reward values for each sub-env. 3) Done values for each sub-env. 4) Info values for each sub-env.
-