Wrapping environments

Environment wrappers are a way to modify the behavior of an environment without modifying the environment itself. This can be used to apply functions to modify observations or rewards, record videos, enforce time limits, etc. A detailed description of the API is available in the gymnasium.Wrapper class.

At present, all RL environments inheriting from the RLTaskEnv class are compatible with gymnasium.Wrapper, since the base class implements the gymnasium.Env interface. In order to wrap an environment, you need to first initialize the base environment. After that, you can wrap it with as many wrappers as you want by calling env = wrapper(env, *args, **kwargs) repeatedly.

For example, here is how you would wrap an environment to enforce that reset is called before step or render:

"""Launch Isaac Sim Simulator first."""


from omni.isaac.orbit.app import AppLauncher

# launch omniverse app in headless mode
app_launcher = AppLauncher(headless=True)
simulation_app = app_launcher.app

"""Rest everything follows."""

import gymnasium as gym

import omni.isaac.orbit_tasks  # noqa: F401
from omni.isaac.orbit_tasks.utils import load_cfg_from_registry

# create base environment
cfg = load_cfg_from_registry("Isaac-Reach-Franka-v0", "env_cfg_entry_point")
env = gym.make("Isaac-Reach-Franka-v0", cfg=cfg)
# wrap environment to enforce that reset is called before step
env = gym.wrappers.OrderEnforcing(env)

Wrapper for recording videos

The gymnasium.wrappers.RecordVideo wrapper can be used to record videos of the environment. The wrapper takes a video_dir argument, which specifies where to save the videos. The videos are saved in mp4 format at specified intervals for specified number of environment steps or episodes.

To use the wrapper, you need to first install ffmpeg. On Ubuntu, you can install it by running:

sudo apt-get install ffmpeg

Attention

By default, when running an environment in headless mode, the Omniverse viewport is disabled. This is done to improve performance by avoiding unnecessary rendering.

We notice the following performance in different rendering modes with the Isaac-Reach-Franka-v0 environment using an RTX 3090 GPU:

• No GUI execution without off-screen rendering enabled: ~65,000 FPS

• No GUI execution with off-screen enabled: ~57,000 FPS

• GUI execution with full rendering: ~13,000 FPS

The viewport camera used for rendering is the default camera in the scene called "/OmniverseKit_Persp". The camera’s pose and image resolution can be configured through the ViewerCfg class.

Default parameters of the ViewerCfg class:

@configclass
class ViewerCfg:
    """Configuration of the scene viewport camera."""

    eye: tuple[float, float, float] = (7.5, 7.5, 7.5)
    """Initial camera position (in m). Default is (7.5, 7.5, 7.5)."""

    lookat: tuple[float, float, float] = (0.0, 0.0, 0.0)
    """Initial camera target position (in m). Default is (0.0, 0.0, 0.0)."""

    cam_prim_path: str = "/OmniverseKit_Persp"
    """The camera prim path to record images from. Default is "/OmniverseKit_Persp",
    which is the default camera in the viewport.
    """

    resolution: tuple[int, int] = (1280, 720)
    """The resolution (width, height) of the camera specified using :attr:`cam_prim_path`.
    Default is (1280, 720).
    """

    origin_type: Literal["world", "env", "asset_root"] = "world"
    """The frame in which the camera position (eye) and target (lookat) are defined in. Default is "world".

    Available options are:

    * ``"world"``: The origin of the world.
    * ``"env"``: The origin of the environment defined by :attr:`env_index`.
    * ``"asset_root"``: The center of the asset defined by :attr:`asset_name` in environment :attr:`env_index`.
    """

    env_index: int = 0
    """The environment index for frame origin. Default is 0.

    This quantity is only effective if :attr:`origin` is set to "env" or "asset_root".
    """

    asset_name: str | None = None
    """The asset name in the interactive scene for the frame origin. Default is None.

    This quantity is only effective if :attr:`origin` is set to "asset_root".
    """

After adjusting the parameters, you can record videos by wrapping the environment with the gymnasium.wrappers.RecordVideo wrapper and enabling the off-screen rendering flag. Additionally, you need to specify the render mode of the environment as "rgb_array".

As an example, the following code records a video of the Isaac-Reach-Franka-v0 environment for 200 steps, and saves it in the videos folder at a step interval of 1500 steps.

"""Launch Isaac Sim Simulator first."""


from omni.isaac.orbit.app import AppLauncher

# launch omniverse app in headless mode with off-screen rendering
app_launcher = AppLauncher(headless=True, offscreen_render=True)
simulation_app = app_launcher.app

"""Rest everything follows."""

import gymnasium as gym

# adjust camera resolution and pose
env_cfg.viewer.resolution = (640, 480)
env_cfg.viewer.eye = (1.0, 1.0, 1.0)
env_cfg.viewer.lookat = (0.0, 0.0, 0.0)
# create isaac-env instance
# set render mode to rgb_array to obtain images on render calls
env = gym.make(task_name, cfg=env_cfg, render_mode="rgb_array")
# wrap for video recording
video_kwargs = {
    "video_folder": "videos",
    "step_trigger": lambda step: step % 1500 == 0,
    "video_length": 200,
}
env = gym.wrappers.RecordVideo(env, **video_kwargs)

Wrapper for learning frameworks

Every learning framework has its own API for interacting with environments. For example, the Stable-Baselines3 library uses the gym.Env interface to interact with environments. However, libraries like RL-Games or RSL-RL use their own API for interfacing with a learning environments. Since there is no one-size-fits-all solution, we do not base the RLTaskEnv class on any particular learning framework’s environment definition. Instead, we implement wrappers to make it compatible with the learning framework’s environment definition.

As an example of how to use the RL task environment with Stable-Baselines3:

from omni.isaac.orbit_tasks.utils.wrappers.sb3 import Sb3VecEnvWrapper

# create isaac-env instance
env = gym.make(task_name, cfg=env_cfg)
# wrap around environment for stable baselines
env = Sb3VecEnvWrapper(env)

Caution

Wrapping the environment with the respective learning framework’s wrapper should happen in the end, i.e. after all other wrappers have been applied. This is because the learning framework’s wrapper modifies the interpretation of environment’s APIs which may no longer be compatible with gymnasium.Env.

Adding new wrappers

All new wrappers should be added to the omni.isaac.orbit_tasks.utils.wrappers module. They should check that the underlying environment is an instance of omni.isaac.orbit.envs.RLTaskEnv before applying the wrapper. This can be done by using the unwrapped() property.

We include a set of wrappers in this module that can be used as a reference to implement your own wrappers. If you implement a new wrapper, please consider contributing it to the framework by opening a pull request.