orbit.sensors

orbit.sensors#

Sub-package containing various sensor classes implementations.

This subpackage contains the sensor classes that are compatible with Isaac Sim. We include both USD-based and custom sensors:

USD-prim sensors: Available in Omniverse and require creating a USD prim for them. For instance, RTX ray tracing camera and lidar sensors.
USD-schema sensors: Available in Omniverse and require creating a USD schema on an existing prim. For instance, contact sensors and frame transformers.
Custom sensors: Implemented in Python and do not require creating any USD prim or schema. For instance, warp-based ray-casters.

Due to the above categorization, the prim paths passed to the sensor’s configuration class are interpreted differently based on the sensor type. The following table summarizes the interpretation of the prim paths for different sensor types:

Sensor Type	Example Prim Path	Pre-check
Camera	/World/robot/base/camera	Leaf is available, and it will spawn a USD camera
Contact Sensor	/World/robot/feet_*	Leaf is available and checks if the schema exists
Ray Caster	/World/robot/base	Leaf exists and is a physics body (Articulation / Rigid Body)
Frame Transformer	/World/robot/base	Leaf exists and is a physics body (Articulation / Rigid Body)

Submodules

patterns

Sub-module for ray-casting patterns used by the ray-caster.

Classes

`SensorBase`	The base class for implementing a sensor.
`SensorBaseCfg`	Configuration parameters for a sensor.
`Camera`	The camera sensor for acquiring visual data.
`CameraData`	Data container for the camera sensor.
`CameraCfg`	Configuration for a camera sensor.
`ContactSensor`	A contact reporting sensor.
`ContactSensorData`	Data container for the contact reporting sensor.
`ContactSensorCfg`	Configuration for the contact sensor.
`FrameTransformer`	A sensor for reporting frame transforms.
`FrameTransformerData`	Data container for the frame transformer sensor.
`FrameTransformerCfg`	Configuration for the frame transformer sensor.
`RayCaster`	A ray-casting sensor.
`RayCasterData`	Data container for the ray-cast sensor.
`RayCasterCfg`	Configuration for the ray-cast sensor.
`RayCasterCamera`	A ray-casting camera sensor.
`RayCasterCameraCfg`	Configuration for the ray-cast sensor.

Sensor Base#

class omni.isaac.orbit.sensors.SensorBase[source]#

The base class for implementing a sensor.

The implementation is based on lazy evaluation. The sensor data is only updated when the user tries accessing the data through the data property or sets force_compute=True in the update() method. This is done to avoid unnecessary computation when the sensor data is not used.

The sensor is updated at the specified update period. If the update period is zero, then the sensor is updated at every simulation step.

Methods:

`__init__`(cfg)	Initialize the sensor class.
`set_debug_vis`(debug_vis)	Sets whether to visualize the sensor data.
`reset`([env_ids])	Resets the sensor internals.

Attributes:

`num_instances`	Number of instances of the sensor.
`device`	Memory device for computation.
`data`	Data from the sensor.
`has_debug_vis_implementation`	Whether the sensor has a debug visualization implemented.

__init__(cfg: SensorBaseCfg)[source]#

Initialize the sensor class.

Parameters:: cfg – The configuration parameters for the sensor.

property num_instances: int#

Number of instances of the sensor.

This is equal to the number of sensors per environment multiplied by the number of environments.

property device: str#: Memory device for computation.

abstract property data: Any#

Data from the sensor.

This property is only updated when the user tries to access the data. This is done to avoid unnecessary computation when the sensor data is not used.

For updating the sensor when this property is accessed, you can use the following code snippet in your sensor implementation:

# update sensors if needed
self._update_outdated_buffers()
# return the data (where `_data` is the data for the sensor)
return self._data

property has_debug_vis_implementation: bool#: Whether the sensor has a debug visualization implemented.

set_debug_vis(debug_vis: bool) → bool[source]#

Sets whether to visualize the sensor data.

Parameters:: debug_vis – Whether to visualize the sensor data.
Returns:: Whether the debug visualization was successfully set. False if the sensor does not support debug visualization.

reset(env_ids: Sequence[int] | None = None)[source]#

Resets the sensor internals.

Parameters:: env_ids – The sensor ids to reset. Defaults to None.

class omni.isaac.orbit.sensors.SensorBaseCfg[source]#

Configuration parameters for a sensor.

Attributes:

`prim_path`	Prim path (or expression) to the sensor.
`update_period`	Update period of the sensor buffers (in seconds).
`history_length`	Number of past frames to store in the sensor buffers.
`debug_vis`	Whether to visualize the sensor.

prim_path: str#: Prim path (or expression) to the sensor.

Note

The expression can contain the environment namespace regex {ENV_REGEX_NS} which will be replaced with the environment namespace.

Example: {ENV_REGEX_NS}/Robot/sensor will be replaced with /World/envs/env_.*/Robot/sensor.

update_period: float#: Update period of the sensor buffers (in seconds). Defaults to 0.0 (update every step).

history_length: int#: Number of past frames to store in the sensor buffers. Defaults to 0, which means that only the current data is stored (no history).

debug_vis: bool#: Whether to visualize the sensor. Defaults to False.

USD Camera#

class omni.isaac.orbit.sensors.Camera[source]#

Bases: SensorBase

The camera sensor for acquiring visual data.

This class wraps over the UsdGeom Camera for providing a consistent API for acquiring visual data. It ensures that the camera follows the ROS convention for the coordinate system.

Summarizing from the replicator extension, the following sensor types are supported:

"rgb": A rendered color image.
"distance_to_camera": An image containing the distance to camera optical center.
"distance_to_image_plane": An image containing distances of 3D points from camera plane along camera’s z-axis.
"normals": An image containing the local surface normal vectors at each pixel.
"motion_vectors": An image containing the motion vector data at each pixel.
"semantic_segmentation": The semantic segmentation data.
"instance_segmentation_fast": The instance segmentation data.
"instance_id_segmentation_fast": The instance id segmentation data.

Note

Currently the following sensor types are not supported in a “view” format:

"instance_segmentation": The instance segmentation data. Please use the fast counterparts instead.
"instance_id_segmentation": The instance id segmentation data. Please use the fast counterparts instead.
"bounding_box_2d_tight": The tight 2D bounding box data (only contains non-occluded regions).
"bounding_box_2d_tight_fast": The tight 2D bounding box data (only contains non-occluded regions).
"bounding_box_2d_loose": The loose 2D bounding box data (contains occluded regions).
"bounding_box_2d_loose_fast": The loose 2D bounding box data (contains occluded regions).
"bounding_box_3d": The 3D view space bounding box data.
"bounding_box_3d_fast": The 3D view space bounding box data.

Attributes:

`cfg`	The configuration parameters.
`UNSUPPORTED_TYPES`	The set of sensor types that are not supported by the camera class.
`num_instances`	Number of instances of the sensor.
`data`	Data from the sensor.
`frame`	Frame number when the measurement took place.
`render_product_paths`	The path of the render products for the cameras.
`image_shape`	A tuple containing (height, width) of the camera sensor.
`device`	Memory device for computation.
`has_debug_vis_implementation`	Whether the sensor has a debug visualization implemented.

Methods:

`__init__`(cfg)	Initializes the camera sensor.
`set_intrinsic_matrices`(matrices[, ...])	Set parameters of the USD camera from its intrinsic matrix.
`set_world_poses`([positions, orientations, ...])	Set the pose of the camera w.r.t.
`set_world_poses_from_view`(eyes, targets[, ...])	Set the poses of the camera from the eye position and look-at target position.
`reset`([env_ids])	Resets the sensor internals.
`set_debug_vis`(debug_vis)	Sets whether to visualize the sensor data.

cfg: CameraCfg#: The configuration parameters.

UNSUPPORTED_TYPES: set[str] = {'bounding_box_2d_loose', 'bounding_box_2d_loose_fast', 'bounding_box_2d_tight', 'bounding_box_2d_tight_fast', 'bounding_box_3d', 'bounding_box_3d_fast', 'instance_id_segmentation', 'instance_segmentation'}#: The set of sensor types that are not supported by the camera class.

__init__(cfg: CameraCfg)[source]#

Initializes the camera sensor.

Parameters:

cfg – The configuration parameters.

Raises:

RuntimeError – If no camera prim is found at the given path.
ValueError – If the provided data types are not supported by the camera.

property num_instances: int#

Number of instances of the sensor.

This is equal to the number of sensors per environment multiplied by the number of environments.

property data: CameraData#

Data from the sensor.

This property is only updated when the user tries to access the data. This is done to avoid unnecessary computation when the sensor data is not used.

For updating the sensor when this property is accessed, you can use the following code snippet in your sensor implementation:

# update sensors if needed
self._update_outdated_buffers()
# return the data (where `_data` is the data for the sensor)
return self._data

property frame: torch.tensor#: Frame number when the measurement took place.

property render_product_paths: list[str]#

The path of the render products for the cameras.

This can be used via replicator interfaces to attach to writes or external annotator registry.

property image_shape: tuple[int, int]#: A tuple containing (height, width) of the camera sensor.

set_intrinsic_matrices(matrices: torch.Tensor, focal_length: float = 1.0, env_ids: Sequence[int] | None = None)[source]#

Set parameters of the USD camera from its intrinsic matrix.

The intrinsic matrix and focal length are used to set the following parameters to the USD camera:

focal_length: The focal length of the camera.
horizontal_aperture: The horizontal aperture of the camera.
vertical_aperture: The vertical aperture of the camera.
horizontal_aperture_offset: The horizontal offset of the camera.
vertical_aperture_offset: The vertical offset of the camera.

Warning

Due to limitations of Omniverse camera, we need to assume that the camera is a spherical lens, i.e. has square pixels, and the optical center is centered at the camera eye. If this assumption is not true in the input intrinsic matrix, then the camera will not set up correctly.

Parameters:

matrices – The intrinsic matrices for the camera. Shape is (N, 3, 3).
focal_length – Focal length to use when computing aperture values. Defaults to 1.0.
env_ids – A sensor ids to manipulate. Defaults to None, which means all sensor indices.

set_world_poses(positions: torch.Tensor | None = None, orientations: torch.Tensor | None = None, env_ids: Sequence[int] | None = None, convention: Literal['opengl', 'ros', 'world'] = 'ros')[source]#

Set the pose of the camera w.r.t. the world frame using specified convention.

Since different fields use different conventions for camera orientations, the method allows users to set the camera poses in the specified convention. Possible conventions are:

"opengl" - forward axis: -Z - up axis +Y - Offset is applied in the OpenGL (Usd.Camera) convention
"ros" - forward axis: +Z - up axis -Y - Offset is applied in the ROS convention
"world" - forward axis: +X - up axis +Z - Offset is applied in the World Frame convention

See omni.isaac.orbit.sensors.camera.utils.convert_orientation_convention() for more details on the conventions.

Parameters:

positions – The cartesian coordinates (in meters). Shape is (N, 3). Defaults to None, in which case the camera position in not changed.
orientations – The quaternion orientation in (w, x, y, z). Shape is (N, 4). Defaults to None, in which case the camera orientation in not changed.
env_ids – A sensor ids to manipulate. Defaults to None, which means all sensor indices.
convention – The convention in which the poses are fed. Defaults to “ros”.

Raises:

RuntimeError – If the camera prim is not set. Need to call initialize() method first.

set_world_poses_from_view(eyes: torch.Tensor, targets: torch.Tensor, env_ids: Sequence[int] | None = None)[source]#

Set the poses of the camera from the eye position and look-at target position.

Parameters:

eyes – The positions of the camera’s eye. Shape is (N, 3).
targets – The target locations to look at. Shape is (N, 3).
env_ids – A sensor ids to manipulate. Defaults to None, which means all sensor indices.

Raises:

RuntimeError – If the camera prim is not set. Need to call initialize() method first.
NotImplementedError – If the stage up-axis is not “Y” or “Z”.

reset(env_ids: Sequence[int] | None = None)[source]#

Resets the sensor internals.

Parameters:: env_ids – The sensor ids to reset. Defaults to None.

property device: str#: Memory device for computation.

property has_debug_vis_implementation: bool#: Whether the sensor has a debug visualization implemented.

set_debug_vis(debug_vis: bool) → bool#

Sets whether to visualize the sensor data.

Parameters:: debug_vis – Whether to visualize the sensor data.
Returns:: Whether the debug visualization was successfully set. False if the sensor does not support debug visualization.

class omni.isaac.orbit.sensors.CameraData[source]#

Data container for the camera sensor.

Attributes:

`pos_w`	Position of the sensor origin in world frame, following ROS convention.
`quat_w_world`	Quaternion orientation (w, x, y, z) of the sensor origin in world frame, following the world coordinate frame
`image_shape`	A tuple containing (height, width) of the camera sensor.
`intrinsic_matrices`	The intrinsic matrices for the camera.
`output`	The retrieved sensor data with sensor types as key.
`info`	The retrieved sensor info with sensor types as key.
`quat_w_ros`	Quaternion orientation (w, x, y, z) of the sensor origin in the world frame, following ROS convention.
`quat_w_opengl`	Quaternion orientation (w, x, y, z) of the sensor origin in the world frame, following Opengl / USD Camera convention.

pos_w: torch.Tensor = None#

Position of the sensor origin in world frame, following ROS convention.

Shape is (N, 3) where N is the number of sensors.

quat_w_world: torch.Tensor = None#

Quaternion orientation (w, x, y, z) of the sensor origin in world frame, following the world coordinate frame

Note

World frame convention follows the camera aligned with forward axis +X and up axis +Z.

Shape is (N, 4) where N is the number of sensors.

image_shape: tuple[int, int] = None#: A tuple containing (height, width) of the camera sensor.

intrinsic_matrices: torch.Tensor = None#

The intrinsic matrices for the camera.

Shape is (N, 3, 3) where N is the number of sensors.

output: tensordict.TensorDict = None#

The retrieved sensor data with sensor types as key.

The format of the data is available in the Replicator Documentation. For semantic-based data, this corresponds to the "data" key in the output of the sensor.

info: list[dict[str, Any]] = None#

The retrieved sensor info with sensor types as key.

This contains extra information provided by the sensor such as semantic segmentation label mapping, prim paths. For semantic-based data, this corresponds to the "info" key in the output of the sensor. For other sensor types, the info is empty.

property quat_w_ros: torch.Tensor#

Quaternion orientation (w, x, y, z) of the sensor origin in the world frame, following ROS convention.

Note

ROS convention follows the camera aligned with forward axis +Z and up axis -Y.

Shape is (N, 4) where N is the number of sensors.

property quat_w_opengl: torch.Tensor#

Quaternion orientation (w, x, y, z) of the sensor origin in the world frame, following Opengl / USD Camera convention.

Note

OpenGL convention follows the camera aligned with forward axis -Z and up axis +Y.

Shape is (N, 4) where N is the number of sensors.

class omni.isaac.orbit.sensors.CameraCfg[source]#

Bases: SensorBaseCfg

Configuration for a camera sensor.

Classes:

OffsetCfg

The offset pose of the sensor's frame from the sensor's parent frame.

Attributes:

`offset`	The offset pose of the sensor's frame from the sensor's parent frame.
`spawn`	Spawn configuration for the asset.
`data_types`	List of sensor names/types to enable for the camera.
`width`	Width of the image in pixels.
`height`	Height of the image in pixels.
`semantic_filter`	A string or a list specifying a semantic filter predicate.
`colorize_semantic_segmentation`	Whether to colorize the semantic segmentation images.
`colorize_instance_id_segmentation`	Whether to colorize the instance ID segmentation images.
`colorize_instance_segmentation`	Whether to colorize the instance ID segmentation images.

class OffsetCfg[source]#

Bases: object

The offset pose of the sensor’s frame from the sensor’s parent frame.

Attributes:

`pos`	Translation w.r.t.
`rot`	Quaternion rotation (w, x, y, z) w.r.t.
`convention`	The convention in which the frame offset is applied.

pos: tuple[float, float, float]#: Translation w.r.t. the parent frame. Defaults to (0.0, 0.0, 0.0).

rot: tuple[float, float, float, float]#: Quaternion rotation (w, x, y, z) w.r.t. the parent frame. Defaults to (1.0, 0.0, 0.0, 0.0).

convention: Literal['opengl', 'ros', 'world']#

The convention in which the frame offset is applied. Defaults to “ros”.

"opengl" - forward axis: -Z - up axis: +Y - Offset is applied in the OpenGL (Usd.Camera) convention.
"ros" - forward axis: +Z - up axis: -Y - Offset is applied in the ROS convention.
"world" - forward axis: +X - up axis: +Z - Offset is applied in the World Frame convention.

offset: OffsetCfg#: The offset pose of the sensor’s frame from the sensor’s parent frame. Defaults to identity.

Note

The parent frame is the frame the sensor attaches to. For example, the parent frame of a camera at path /World/envs/env_0/Robot/Camera is /World/envs/env_0/Robot.

spawn: PinholeCameraCfg | FisheyeCameraCfg | None#

Spawn configuration for the asset.

If None, then the prim is not spawned by the asset. Instead, it is assumed that the asset is already present in the scene.

data_types: list[str]#

List of sensor names/types to enable for the camera. Defaults to [“rgb”].

Please refer to the Camera class for a list of available data types.

prim_path: str#: Prim path (or expression) to the sensor.

Note

The expression can contain the environment namespace regex {ENV_REGEX_NS} which will be replaced with the environment namespace.

Example: {ENV_REGEX_NS}/Robot/sensor will be replaced with /World/envs/env_.*/Robot/sensor.

update_period: float#: Update period of the sensor buffers (in seconds). Defaults to 0.0 (update every step).

history_length: int#: Number of past frames to store in the sensor buffers. Defaults to 0, which means that only the current data is stored (no history).

debug_vis: bool#: Whether to visualize the sensor. Defaults to False.

width: int#: Width of the image in pixels.

height: int#: Height of the image in pixels.

semantic_filter: str | list[str]#

A string or a list specifying a semantic filter predicate. Defaults to "*:*".

If a string, it should be a disjunctive normal form of (semantic type, labels). For examples:

"typeA : labelA & !labelB | labelC , typeB: labelA ; typeC: labelE": All prims with semantic type “typeA” and label “labelA” but not “labelB” or with label “labelC”. Also, all prims with semantic type “typeB” and label “labelA”, or with semantic type “typeC” and label “labelE”.
"typeA : * ; * : labelA": All prims with semantic type “typeA” or with label “labelA”

If a list of strings, each string should be a semantic type. The segmentation for prims with semantics of the specified types will be retrieved. For example, if the list is [“class”], only the segmentation for prims with semantics of type “class” will be retrieved.

Contact Sensor#

class omni.isaac.orbit.sensors.ContactSensor[source]#

Bases: SensorBase

A contact reporting sensor.

The contact sensor reports the normal contact forces on a rigid body in the world frame. It relies on the PhysX ContactReporter API to be activated on the rigid bodies.

To enable the contact reporter on a rigid body, please make sure to enable the omni.isaac.orbit.sim.spawner.RigidObjectSpawnerCfg.activate_contact_sensors on your asset spawner configuration. This will enable the contact reporter on all the rigid bodies in the asset.

The sensor can be configured to report the contact forces on a set of bodies with a given filter pattern. Please check the documentation on RigidContactView for more details.

Attributes:

`cfg`	The configuration parameters.
`num_instances`	Number of instances of the sensor.
`data`	Data from the sensor.
`num_bodies`	Number of bodies with contact sensors attached.
`body_names`	Ordered names of bodies with contact sensors attached.
`body_physx_view`	View for the rigid bodies captured (PhysX).
`contact_physx_view`	Contact reporter view for the bodies (PhysX).
`device`	Memory device for computation.
`has_debug_vis_implementation`	Whether the sensor has a debug visualization implemented.

Methods:

`__init__`(cfg)	Initializes the contact sensor object.
`reset`([env_ids])	Resets the sensor internals.
`find_bodies`(name_keys[, preserve_order])	Find bodies in the articulation based on the name keys.
`compute_first_contact`(dt[, abs_tol])	Checks if bodies that have established contact within the last `dt` seconds.
`compute_first_air`(dt[, abs_tol])	Checks if bodies that have broken contact within the last `dt` seconds.
`set_debug_vis`(debug_vis)	Sets whether to visualize the sensor data.

cfg: ContactSensorCfg#: The configuration parameters.

__init__(cfg: ContactSensorCfg)[source]#

Initializes the contact sensor object.

Parameters:: cfg – The configuration parameters.

property num_instances: int#

Number of instances of the sensor.

This is equal to the number of sensors per environment multiplied by the number of environments.

property data: ContactSensorData#

Data from the sensor.

This property is only updated when the user tries to access the data. This is done to avoid unnecessary computation when the sensor data is not used.

For updating the sensor when this property is accessed, you can use the following code snippet in your sensor implementation:

# update sensors if needed
self._update_outdated_buffers()
# return the data (where `_data` is the data for the sensor)
return self._data

property num_bodies: int#: Number of bodies with contact sensors attached.

property body_names: list[str]#: Ordered names of bodies with contact sensors attached.

property body_physx_view: omni.physics.tensors.impl.api.RigidBodyView#: View for the rigid bodies captured (PhysX).

Note

Use this view with caution. It requires handling of tensors in a specific way.

property contact_physx_view: omni.physics.tensors.impl.api.RigidContactView#: Contact reporter view for the bodies (PhysX).

Note

Use this view with caution. It requires handling of tensors in a specific way.

reset(env_ids: Sequence[int] | None = None)[source]#

Resets the sensor internals.

Parameters:: env_ids – The sensor ids to reset. Defaults to None.

find_bodies(name_keys: str | Sequence[str], preserve_order: bool = False) → tuple[list[int], list[str]][source]#

Find bodies in the articulation based on the name keys.

Parameters:

name_keys – A regular expression or a list of regular expressions to match the body names.
preserve_order – Whether to preserve the order of the name keys in the output. Defaults to False.

Returns:

A tuple of lists containing the body indices and names.

compute_first_contact(dt: float, abs_tol: float = 1e-08) → torch.Tensor[source]#

Checks if bodies that have established contact within the last dt seconds.

This function checks if the bodies have established contact within the last dt seconds by comparing the current contact time with the given time period. If the contact time is less than the given time period, then the bodies are considered to be in contact.

Note

The function assumes that dt is a factor of the sensor update time-step. In other words \(dt / dt_sensor = n\), where \(n\) is a natural number. This is always true if the sensor is updated by the physics or the environment stepping time-step and the sensor is read by the environment stepping time-step.

Parameters:

dt – The time period since the contact was established.
abs_tol – The absolute tolerance for the comparison.

Returns:

A boolean tensor indicating the bodies that have established contact within the last dt seconds. Shape is (N, B), where N is the number of sensors and B is the number of bodies in each sensor.

Raises:

RuntimeError – If the sensor is not configured to track contact time.

compute_first_air(dt: float, abs_tol: float = 1e-08) → torch.Tensor[source]#

Checks if bodies that have broken contact within the last dt seconds.

This function checks if the bodies have broken contact within the last dt seconds by comparing the current air time with the given time period. If the air time is less than the given time period, then the bodies are considered to not be in contact.

Note

It assumes that dt is a factor of the sensor update time-step. In other words, \(dt / dt_sensor = n\), where \(n\) is a natural number. This is always true if the sensor is updated by the physics or the environment stepping time-step and the sensor is read by the environment stepping time-step.

Parameters:

dt – The time period since the contract is broken.
abs_tol – The absolute tolerance for the comparison.

Returns:

A boolean tensor indicating the bodies that have broken contact within the last dt seconds. Shape is (N, B), where N is the number of sensors and B is the number of bodies in each sensor.

Raises:

RuntimeError – If the sensor is not configured to track contact time.

property device: str#: Memory device for computation.

property has_debug_vis_implementation: bool#: Whether the sensor has a debug visualization implemented.

set_debug_vis(debug_vis: bool) → bool#

Sets whether to visualize the sensor data.

Parameters:: debug_vis – Whether to visualize the sensor data.
Returns:: Whether the debug visualization was successfully set. False if the sensor does not support debug visualization.

class omni.isaac.orbit.sensors.ContactSensorData[source]#

Data container for the contact reporting sensor.

Attributes:

`pos_w`	Position of the sensor origin in world frame.
`quat_w`	Orientation of the sensor origin in quaternion (w, x, y, z) in world frame.
`net_forces_w`	The net contact forces in world frame.
`net_forces_w_history`	The net contact forces in world frame.
`force_matrix_w`	The contact forces filtered between the sensor bodies and filtered bodies in world frame.
`last_air_time`	Time spent (in s) in the air before the last contact.
`current_air_time`	Time spent (in s) in the air since the last detach.
`last_contact_time`	Time spent (in s) in contact before the last detach.
`current_contact_time`	Time spent (in s) in contact since the last contact.

pos_w: torch.Tensor | None = None#

Position of the sensor origin in world frame.

Shape is (N, 3), where N is the number of sensors.

Note

If the ContactSensorCfg.track_pose is False, then this qunatity is None.

quat_w: torch.Tensor | None = None#

Orientation of the sensor origin in quaternion (w, x, y, z) in world frame.

Shape is (N, 4), where N is the number of sensors.

Note

If the ContactSensorCfg.track_pose is False, then this qunatity is None.

net_forces_w: torch.Tensor | None = None#

The net contact forces in world frame.

Shape is (N, B, 3), where N is the number of sensors and B is the number of bodies in each sensor.

net_forces_w_history: torch.Tensor | None = None#

The net contact forces in world frame.

Shape is (N, T, B, 3), where N is the number of sensors, T is the configured history length and B is the number of bodies in each sensor.

In the history dimension, the first index is the most recent and the last index is the oldest.

force_matrix_w: torch.Tensor | None = None#

The contact forces filtered between the sensor bodies and filtered bodies in world frame.

Shape is (N, B, M, 3), where N is the number of sensors, B is number of bodies in each sensor and M is the number of filtered bodies.

Note

If the ContactSensorCfg.filter_prim_paths_expr is empty, then this quantity is None.

last_air_time: torch.Tensor | None = None#

Time spent (in s) in the air before the last contact.

Shape is (N, B), where N is the number of sensors and B is the number of bodies in each sensor.

Note

If the ContactSensorCfg.track_air_time is False, then this quantity is None.

current_air_time: torch.Tensor | None = None#

Time spent (in s) in the air since the last detach.

Shape is (N, B), where N is the number of sensors and B is the number of bodies in each sensor.

Note

If the ContactSensorCfg.track_air_time is False, then this quantity is None.

last_contact_time: torch.Tensor | None = None#

Time spent (in s) in contact before the last detach.

Shape is (N, B), where N is the number of sensors and B is the number of bodies in each sensor.

Note

If the ContactSensorCfg.track_air_time is False, then this quantity is None.

current_contact_time: torch.Tensor | None = None#

Time spent (in s) in contact since the last contact.

Shape is (N, B), where N is the number of sensors and B is the number of bodies in each sensor.

Note

If the ContactSensorCfg.track_air_time is False, then this quantity is None.

class omni.isaac.orbit.sensors.ContactSensorCfg[source]#

Bases: SensorBaseCfg

Configuration for the contact sensor.

Attributes:

`track_pose`	Whether to track the pose of the sensor's origin.
`track_air_time`	Whether to track the air/contact time of the bodies (time between contacts).
`force_threshold`	The threshold on the norm of the contact force that determines whether two bodies are in collision or not.
`filter_prim_paths_expr`	The list of primitive paths (or expressions) to filter contacts with.
`visualizer_cfg`	The configuration object for the visualization markers.

track_pose: bool#: Whether to track the pose of the sensor’s origin. Defaults to False.

track_air_time: bool#: Whether to track the air/contact time of the bodies (time between contacts). Defaults to False.

prim_path: str#: Prim path (or expression) to the sensor.

Note

The expression can contain the environment namespace regex {ENV_REGEX_NS} which will be replaced with the environment namespace.

Example: {ENV_REGEX_NS}/Robot/sensor will be replaced with /World/envs/env_.*/Robot/sensor.

update_period: float#: Update period of the sensor buffers (in seconds). Defaults to 0.0 (update every step).

history_length: int#: Number of past frames to store in the sensor buffers. Defaults to 0, which means that only the current data is stored (no history).

debug_vis: bool#: Whether to visualize the sensor. Defaults to False.

force_threshold: float#

The threshold on the norm of the contact force that determines whether two bodies are in collision or not.

This value is only used for tracking the mode duration (the time in contact or in air), if track_air_time is True.

filter_prim_paths_expr: list[str]#

The list of primitive paths (or expressions) to filter contacts with. Defaults to an empty list, in which case no filtering is applied.

The contact sensor allows reporting contacts between the primitive specified with prim_path and other primitives in the scene. For instance, in a scene containing a robot, a ground plane and an object, you can obtain individual contact reports of the base of the robot with the ground plane and the object.

Note

The expression in the list can contain the environment namespace regex {ENV_REGEX_NS} which will be replaced with the environment namespace.

Example: {ENV_REGEX_NS}/Object will be replaced with /World/envs/env_.*/Object.

visualizer_cfg: VisualizationMarkersCfg#: The configuration object for the visualization markers. Defaults to CONTACT_SENSOR_MARKER_CFG.

Note

This attribute is only used when debug visualization is enabled.

Frame Transformer#

class omni.isaac.orbit.sensors.FrameTransformer[source]#

Bases: SensorBase

A sensor for reporting frame transforms.

This class provides an interface for reporting the transform of one or more frames (target frames) with respect to another frame (source frame). The source frame is specified by the user as a prim path (FrameTransformerCfg.prim_path) and the target frames are specified by the user as a list of prim paths (FrameTransformerCfg.target_frames).

The source frame and target frames are assumed to be rigid bodies. The transform of the target frames with respect to the source frame is computed by first extracting the transform of the source frame and target frames from the physics engine and then computing the relative transform between the two.

Additionally, the user can specify an offset for the source frame and each target frame. This is useful for specifying the transform of the desired frame with respect to the body’s center of mass, for instance.

A common example of using this sensor is to track the position and orientation of the end effector of a robotic manipulator. In this case, the source frame would be the body corresponding to the base frame of the manipulator, and the target frame would be the body corresponding to the end effector. Since the end-effector is typically a fictitious body, the user may need to specify an offset from the end-effector to the body of the manipulator.

Note

Currently, this implementation only handles frames within an articulation. This is because the frame regex expressions are resolved based on their parent prim path. This can be extended to handle frames outside of articulation by using the frame prim path instead. However, this would require additional checks to ensure that the user-specified frames are valid which is not currently implemented.

Warning

The implementation assumes that the parent body of a target frame is not the same as that of the source frame (i.e. FrameTransformerCfg.prim_path). While a corner case, this can occur if the user specifies the same prim path for both the source frame and target frame. In this case, the target frame will be ignored and not reported. This is a limitation of the current implementation and will be fixed in a future release.

Attributes:

`cfg`	The configuration parameters.
`data`	Data from the sensor.
`device`	Memory device for computation.
`has_debug_vis_implementation`	Whether the sensor has a debug visualization implemented.
`num_instances`	Number of instances of the sensor.

Methods:

`__init__`(cfg)	Initializes the frame transformer object.
`reset`([env_ids])	Resets the sensor internals.
`set_debug_vis`(debug_vis)	Sets whether to visualize the sensor data.

cfg: FrameTransformerCfg#: The configuration parameters.

__init__(cfg: FrameTransformerCfg)[source]#

Initializes the frame transformer object.

Parameters:: cfg – The configuration parameters.

property data: FrameTransformerData#

Data from the sensor.

This property is only updated when the user tries to access the data. This is done to avoid unnecessary computation when the sensor data is not used.

For updating the sensor when this property is accessed, you can use the following code snippet in your sensor implementation:

# update sensors if needed
self._update_outdated_buffers()
# return the data (where `_data` is the data for the sensor)
return self._data

reset(env_ids: Sequence[int] | None = None)[source]#

Resets the sensor internals.

Parameters:: env_ids – The sensor ids to reset. Defaults to None.

property device: str#: Memory device for computation.

property has_debug_vis_implementation: bool#: Whether the sensor has a debug visualization implemented.

property num_instances: int#

Number of instances of the sensor.

This is equal to the number of sensors per environment multiplied by the number of environments.

set_debug_vis(debug_vis: bool) → bool#

Sets whether to visualize the sensor data.

Parameters:: debug_vis – Whether to visualize the sensor data.
Returns:: Whether the debug visualization was successfully set. False if the sensor does not support debug visualization.

class omni.isaac.orbit.sensors.FrameTransformerData[source]#

Data container for the frame transformer sensor.

Attributes:

`target_frame_names`	Target frame names (this denotes the order in which that frame data is ordered).
`target_pos_source`	Position of the target frame(s) relative to source frame.
`target_quat_source`	Orientation of the target frame(s) relative to source frame quaternion (w, x, y, z).
`target_pos_w`	Position of the target frame(s) after offset (in world frame).
`target_quat_w`	Orientation of the target frame(s) after offset (in world frame) quaternion (w, x, y, z).
`source_pos_w`	Position of the source frame after offset (in world frame).
`source_quat_w`	Orientation of the source frame after offset (in world frame) quaternion (w, x, y, z).
`target_rot_source`	Alias for `target_quat_source`.
`target_rot_w`	Alias for `target_quat_w`.
`source_rot_w`	Alias for `source_quat_w`.

target_frame_names: list[str] = None#

Target frame names (this denotes the order in which that frame data is ordered).

The frame names are resolved from the FrameTransformerCfg.FrameCfg.name field. This usually follows the order in which the frames are defined in the config. However, in the case of regex matching, the order may be different.

target_pos_source: torch.Tensor = None#

Position of the target frame(s) relative to source frame.

Shape is (N, M, 3), where N is the number of environments, and M is the number of target frames.

target_quat_source: torch.Tensor = None#

Orientation of the target frame(s) relative to source frame quaternion (w, x, y, z).

Shape is (N, M, 4), where N is the number of environments, and M is the number of target frames.

target_pos_w: torch.Tensor = None#

Position of the target frame(s) after offset (in world frame).

Shape is (N, M, 3), where N is the number of environments, and M is the number of target frames.

target_quat_w: torch.Tensor = None#

Orientation of the target frame(s) after offset (in world frame) quaternion (w, x, y, z).

Shape is (N, M, 4), where N is the number of environments, and M is the number of target frames.

source_pos_w: torch.Tensor = None#

Position of the source frame after offset (in world frame).

Shape is (N, 3), where N is the number of environments.

source_quat_w: torch.Tensor = None#

Orientation of the source frame after offset (in world frame) quaternion (w, x, y, z).

Shape is (N, 4), where N is the number of environments.

property target_rot_source: torch.Tensor#: Alias for target_quat_source.

Deprecated since version v0.2.1: Use target_quat_source instead. Will be removed in v0.3.0.

property target_rot_w: torch.Tensor#: Alias for target_quat_w.

Deprecated since version v0.2.1: Use target_quat_w instead. Will be removed in v0.3.0.

property source_rot_w: torch.Tensor#: Alias for source_quat_w.

Deprecated since version v0.2.1: Use source_quat_w instead. Will be removed in v0.3.0.

class omni.isaac.orbit.sensors.FrameTransformerCfg[source]#

Bases: SensorBaseCfg

Configuration for the frame transformer sensor.

Classes:

FrameCfg

Information specific to a coordinate frame.

Attributes:

`prim_path`	The prim path of the body to transform from (source frame).
`source_frame_offset`	The pose offset from the source prim frame.
`target_frames`	A list of the target frames.
`visualizer_cfg`	The configuration object for the visualization markers.

class FrameCfg[source]#

Bases: object

Information specific to a coordinate frame.

Attributes:

`prim_path`	The prim path corresponding to the parent rigid body.
`name`	User-defined name for the new coordinate frame.
`offset`	The pose offset from the parent prim frame.

prim_path: str#

The prim path corresponding to the parent rigid body.

This prim should be part of the same articulation as FrameTransformerCfg.prim_path.

name: str | None#

User-defined name for the new coordinate frame. Defaults to None.

If None, then the name is extracted from the leaf of the prim path.

offset: OffsetCfg#: The pose offset from the parent prim frame.

update_period: float#: Update period of the sensor buffers (in seconds). Defaults to 0.0 (update every step).

history_length: int#: Number of past frames to store in the sensor buffers. Defaults to 0, which means that only the current data is stored (no history).

debug_vis: bool#: Whether to visualize the sensor. Defaults to False.

prim_path: str#: The prim path of the body to transform from (source frame).

source_frame_offset: OffsetCfg#: The pose offset from the source prim frame.

target_frames: list[omni.isaac.orbit.sensors.frame_transformer.frame_transformer_cfg.FrameTransformerCfg.FrameCfg]#

A list of the target frames.

This allows a single FrameTransformer to handle multiple target prims. For example, in a quadruped, we can use a single FrameTransformer to track each foot’s position and orientation in the body frame using four frame offsets.

visualizer_cfg: VisualizationMarkersCfg#: The configuration object for the visualization markers. Defaults to FRAME_MARKER_CFG.

Note

This attribute is only used when debug visualization is enabled.

class omni.isaac.orbit.sensors.OffsetCfg[source]#

The offset pose of one frame relative to another frame.

Attributes:

`pos`	Translation w.r.t.
`rot`	Quaternion rotation (w, x, y, z) w.r.t.

pos: tuple[float, float, float]#: Translation w.r.t. the parent frame. Defaults to (0.0, 0.0, 0.0).

rot: tuple[float, float, float, float]#: Quaternion rotation (w, x, y, z) w.r.t. the parent frame. Defaults to (1.0, 0.0, 0.0, 0.0).

Ray-Cast Sensor#

class omni.isaac.orbit.sensors.RayCaster[source]#

Bases: SensorBase

A ray-casting sensor.

The ray-caster uses a set of rays to detect collisions with meshes in the scene. The rays are defined in the sensor’s local coordinate frame. The sensor can be configured to ray-cast against a set of meshes with a given ray pattern.

The meshes are parsed from the list of primitive paths provided in the configuration. These are then converted to warp meshes and stored in the warp_meshes list. The ray-caster then ray-casts against these warp meshes using the ray pattern provided in the configuration.

Note

Currently, only static meshes are supported. Extending the warp mesh to support dynamic meshes is a work in progress.

Attributes:

`cfg`	The configuration parameters.
`meshes`	The warp meshes available for raycasting.
`num_instances`	Number of instances of the sensor.
`data`	Data from the sensor.
`device`	Memory device for computation.
`has_debug_vis_implementation`	Whether the sensor has a debug visualization implemented.

Methods:

`__init__`(cfg)	Initializes the ray-caster object.
`reset`([env_ids])	Resets the sensor internals.
`set_debug_vis`(debug_vis)	Sets whether to visualize the sensor data.

cfg: RayCasterCfg#: The configuration parameters.

meshes: ClassVar[dict[str, wp.Mesh]] = {}#

The warp meshes available for raycasting.

The keys correspond to the prim path for the meshes, and values are the corresponding warp Mesh objects.

Note

We store a global dictionary of all warp meshes to prevent re-loading the mesh for different ray-cast sensor instances.

__init__(cfg: RayCasterCfg)[source]#

Initializes the ray-caster object.

Parameters:: cfg – The configuration parameters.

property num_instances: int#

Number of instances of the sensor.

This is equal to the number of sensors per environment multiplied by the number of environments.

property data: RayCasterData#

Data from the sensor.

This property is only updated when the user tries to access the data. This is done to avoid unnecessary computation when the sensor data is not used.

For updating the sensor when this property is accessed, you can use the following code snippet in your sensor implementation:

# update sensors if needed
self._update_outdated_buffers()
# return the data (where `_data` is the data for the sensor)
return self._data

reset(env_ids: Sequence[int] | None = None)[source]#

Resets the sensor internals.

Parameters:: env_ids – The sensor ids to reset. Defaults to None.

property device: str#: Memory device for computation.

property has_debug_vis_implementation: bool#: Whether the sensor has a debug visualization implemented.

set_debug_vis(debug_vis: bool) → bool#

Sets whether to visualize the sensor data.

Parameters:: debug_vis – Whether to visualize the sensor data.
Returns:: Whether the debug visualization was successfully set. False if the sensor does not support debug visualization.

class omni.isaac.orbit.sensors.RayCasterData[source]#

Data container for the ray-cast sensor.

Attributes:

`pos_w`	Position of the sensor origin in world frame.
`quat_w`	Orientation of the sensor origin in quaternion (w, x, y, z) in world frame.
`ray_hits_w`	The ray hit positions in the world frame.

pos_w: torch.Tensor = None#

Position of the sensor origin in world frame.

Shape is (N, 3), where N is the number of sensors.

quat_w: torch.Tensor = None#

Orientation of the sensor origin in quaternion (w, x, y, z) in world frame.

Shape is (N, 4), where N is the number of sensors.

ray_hits_w: torch.Tensor = None#

The ray hit positions in the world frame.

Shape is (N, B, 3), where N is the number of sensors, B is the number of rays in the scan pattern per sensor.

class omni.isaac.orbit.sensors.RayCasterCfg[source]#

Bases: SensorBaseCfg

Configuration for the ray-cast sensor.

Classes:

OffsetCfg

The offset pose of the sensor's frame from the sensor's parent frame.

Attributes:

`mesh_prim_paths`	The list of mesh primitive paths to ray cast against.
`offset`	The offset pose of the sensor's frame from the sensor's parent frame.
`attach_yaw_only`	Whether the rays' starting positions and directions only track the yaw orientation.
`pattern_cfg`	The pattern that defines the local ray starting positions and directions.
`max_distance`	Maximum distance (in meters) from the sensor to ray cast to.
`drift_range`	The range of drift (in meters) to add to the ray starting positions (xyz).
`visualizer_cfg`	The configuration object for the visualization markers.

class OffsetCfg[source]#

Bases: object

The offset pose of the sensor’s frame from the sensor’s parent frame.

Attributes:

`pos`	Translation w.r.t.
`rot`	Quaternion rotation (w, x, y, z) w.r.t.

pos: tuple[float, float, float]#: Translation w.r.t. the parent frame. Defaults to (0.0, 0.0, 0.0).

rot: tuple[float, float, float, float]#: Quaternion rotation (w, x, y, z) w.r.t. the parent frame. Defaults to (1.0, 0.0, 0.0, 0.0).

mesh_prim_paths: list[str]#: The list of mesh primitive paths to ray cast against.

Note

Currently, only a single static mesh is supported. We are working on supporting multiple static meshes and dynamic meshes.

offset: OffsetCfg#: The offset pose of the sensor’s frame from the sensor’s parent frame. Defaults to identity.

prim_path: str#: Prim path (or expression) to the sensor.

Note

The expression can contain the environment namespace regex {ENV_REGEX_NS} which will be replaced with the environment namespace.

Example: {ENV_REGEX_NS}/Robot/sensor will be replaced with /World/envs/env_.*/Robot/sensor.

update_period: float#: Update period of the sensor buffers (in seconds). Defaults to 0.0 (update every step).

history_length: int#: Number of past frames to store in the sensor buffers. Defaults to 0, which means that only the current data is stored (no history).

debug_vis: bool#: Whether to visualize the sensor. Defaults to False.

attach_yaw_only: bool#

Whether the rays’ starting positions and directions only track the yaw orientation.

This is useful for ray-casting height maps, where only yaw rotation is needed.

pattern_cfg: PatternBaseCfg#: The pattern that defines the local ray starting positions and directions.

max_distance: float#: Maximum distance (in meters) from the sensor to ray cast to. Defaults to 1e6.

drift_range: tuple[float, float]#

The range of drift (in meters) to add to the ray starting positions (xyz). Defaults to (0.0, 0.0).

For floating base robots, this is useful for simulating drift in the robot’s pose estimation.

visualizer_cfg: VisualizationMarkersCfg#: The configuration object for the visualization markers. Defaults to RAY_CASTER_MARKER_CFG.

Note

This attribute is only used when debug visualization is enabled.

Ray-Cast Camera#

class omni.isaac.orbit.sensors.RayCasterCamera[source]#

Bases: RayCaster

A ray-casting camera sensor.

The ray-caster camera uses a set of rays to get the distances to meshes in the scene. The rays are defined in the sensor’s local coordinate frame. The sensor has the same interface as the omni.isaac.orbit.sensors.Camera that implements the camera class through USD camera prims. However, this class provides a faster image generation. The sensor converts meshes from the list of primitive paths provided in the configuration to Warp meshes. The camera then ray-casts against these Warp meshes only.

Currently, only the following annotators are supported:

"distance_to_camera": An image containing the distance to camera optical center.
"distance_to_image_plane": An image containing distances of 3D points from camera plane along camera’s z-axis.
"normals": An image containing the local surface normal vectors at each pixel.

Note

Currently, only static meshes are supported. Extending the warp mesh to support dynamic meshes is a work in progress.

Attributes:

`cfg`	The configuration parameters.
`UNSUPPORTED_TYPES`	A set of sensor types that are not supported by the ray-caster camera.
`data`	Data from the sensor.
`image_shape`	A tuple containing (height, width) of the camera sensor.
`frame`	Frame number when the measurement took place.
`device`	Memory device for computation.
`has_debug_vis_implementation`	Whether the sensor has a debug visualization implemented.
`meshes`	The warp meshes available for raycasting.
`num_instances`	Number of instances of the sensor.

Methods:

`__init__`(cfg)	Initializes the camera object.
`set_intrinsic_matrices`(matrices[, ...])	Set the intrinsic matrix of the camera.
`reset`([env_ids])	Resets the sensor internals.
`set_world_poses`([positions, orientations, ...])	Set the pose of the camera w.r.t.
`set_world_poses_from_view`(eyes, targets[, ...])	Set the poses of the camera from the eye position and look-at target position.
`set_debug_vis`(debug_vis)	Sets whether to visualize the sensor data.

cfg: RayCasterCameraCfg#: The configuration parameters.

UNSUPPORTED_TYPES: ClassVar[set[str]] = {'bounding_box_2d_loose', 'bounding_box_2d_loose_fast', 'bounding_box_2d_tight', 'bounding_box_2d_tight_fast', 'bounding_box_3d', 'bounding_box_3d_fast', 'instance_id_segmentation', 'instance_id_segmentation_fast', 'instance_segmentation', 'instance_segmentation_fast', 'motion_vectors', 'rgb', 'semantic_segmentation', 'skeleton_data'}#: A set of sensor types that are not supported by the ray-caster camera.

__init__(cfg: RayCasterCameraCfg)[source]#

Initializes the camera object.

Parameters:: cfg – The configuration parameters.
Raises:: ValueError – If the provided data types are not supported by the ray-caster camera.

property data: CameraData#

Data from the sensor.

This property is only updated when the user tries to access the data. This is done to avoid unnecessary computation when the sensor data is not used.

For updating the sensor when this property is accessed, you can use the following code snippet in your sensor implementation:

# update sensors if needed
self._update_outdated_buffers()
# return the data (where `_data` is the data for the sensor)
return self._data

property image_shape: tuple[int, int]#: A tuple containing (height, width) of the camera sensor.

property frame: torch.tensor#: Frame number when the measurement took place.

set_intrinsic_matrices(matrices: torch.Tensor, focal_length: float = 1.0, env_ids: Sequence[int] | None = None)[source]#

Set the intrinsic matrix of the camera.

Parameters:

matrices – The intrinsic matrices for the camera. Shape is (N, 3, 3).
focal_length – Focal length to use when computing aperture values. Defaults to 1.0.
env_ids – A sensor ids to manipulate. Defaults to None, which means all sensor indices.

reset(env_ids: Sequence[int] | None = None)[source]#

Resets the sensor internals.

Parameters:: env_ids – The sensor ids to reset. Defaults to None.

Set the pose of the camera w.r.t. the world frame using specified convention.

Since different fields use different conventions for camera orientations, the method allows users to set the camera poses in the specified convention. Possible conventions are:

"opengl" - forward axis: -Z - up axis +Y - Offset is applied in the OpenGL (Usd.Camera) convention
"ros" - forward axis: +Z - up axis -Y - Offset is applied in the ROS convention
"world" - forward axis: +X - up axis +Z - Offset is applied in the World Frame convention

See omni.isaac.orbit.sensors.camera.utils.convert_orientation_convention() for more details on the conventions.

Parameters:

positions – The cartesian coordinates (in meters). Shape is (N, 3). Defaults to None, in which case the camera position in not changed.
orientations – The quaternion orientation in (w, x, y, z). Shape is (N, 4). Defaults to None, in which case the camera orientation in not changed.
env_ids – A sensor ids to manipulate. Defaults to None, which means all sensor indices.
convention – The convention in which the poses are fed. Defaults to “ros”.

Raises:

RuntimeError – If the camera prim is not set. Need to call initialize() method first.

set_world_poses_from_view(eyes: torch.Tensor, targets: torch.Tensor, env_ids: Sequence[int] | None = None)[source]#

Set the poses of the camera from the eye position and look-at target position.

Parameters:

eyes – The positions of the camera’s eye. Shape is N, 3).
targets – The target locations to look at. Shape is (N, 3).
env_ids – A sensor ids to manipulate. Defaults to None, which means all sensor indices.

Raises:

RuntimeError – If the camera prim is not set. Need to call initialize() method first.
NotImplementedError – If the stage up-axis is not “Y” or “Z”.

property device: str#: Memory device for computation.

property has_debug_vis_implementation: bool#: Whether the sensor has a debug visualization implemented.

meshes: ClassVar[dict[str, wp.Mesh]] = {}#

The warp meshes available for raycasting.

The keys correspond to the prim path for the meshes, and values are the corresponding warp Mesh objects.

Note

We store a global dictionary of all warp meshes to prevent re-loading the mesh for different ray-cast sensor instances.

property num_instances: int#

Number of instances of the sensor.

This is equal to the number of sensors per environment multiplied by the number of environments.

set_debug_vis(debug_vis: bool) → bool#

Sets whether to visualize the sensor data.

Parameters:: debug_vis – Whether to visualize the sensor data.
Returns:: Whether the debug visualization was successfully set. False if the sensor does not support debug visualization.

class omni.isaac.orbit.sensors.RayCasterCameraCfg[source]#

Bases: RayCasterCfg

Configuration for the ray-cast sensor.

Classes:

OffsetCfg

The offset pose of the sensor's frame from the sensor's parent frame.

Attributes:

`offset`	The offset pose of the sensor's frame from the sensor's parent frame.
`data_types`	List of sensor names/types to enable for the camera.

class OffsetCfg[source]#

Bases: object

The offset pose of the sensor’s frame from the sensor’s parent frame.

Attributes:

`pos`	Translation w.r.t.
`rot`	Quaternion rotation (w, x, y, z) w.r.t.
`convention`	The convention in which the frame offset is applied.

pos: tuple[float, float, float]#: Translation w.r.t. the parent frame. Defaults to (0.0, 0.0, 0.0).

rot: tuple[float, float, float, float]#: Quaternion rotation (w, x, y, z) w.r.t. the parent frame. Defaults to (1.0, 0.0, 0.0, 0.0).

convention: Literal['opengl', 'ros', 'world']#

The convention in which the frame offset is applied. Defaults to “ros”.

"opengl" - forward axis: -Z - up axis: +Y - Offset is applied in the OpenGL (Usd.Camera) convention.
"ros" - forward axis: +Z - up axis: -Y - Offset is applied in the ROS convention.
"world" - forward axis: +X - up axis: +Z - Offset is applied in the World Frame convention.

prim_path: str#: Prim path (or expression) to the sensor.

Note

The expression can contain the environment namespace regex {ENV_REGEX_NS} which will be replaced with the environment namespace.

Example: {ENV_REGEX_NS}/Robot/sensor will be replaced with /World/envs/env_.*/Robot/sensor.

update_period: float#: Update period of the sensor buffers (in seconds). Defaults to 0.0 (update every step).

history_length: int#: Number of past frames to store in the sensor buffers. Defaults to 0, which means that only the current data is stored (no history).

debug_vis: bool#: Whether to visualize the sensor. Defaults to False.

mesh_prim_paths: list[str]#: The list of mesh primitive paths to ray cast against.

Note

Currently, only a single static mesh is supported. We are working on supporting multiple static meshes and dynamic meshes.

attach_yaw_only: bool#

Whether the rays’ starting positions and directions only track the yaw orientation.

This is useful for ray-casting height maps, where only yaw rotation is needed.

pattern_cfg: PatternBaseCfg#: The pattern that defines the local ray starting positions and directions.

max_distance: float#: Maximum distance (in meters) from the sensor to ray cast to. Defaults to 1e6.

drift_range: tuple[float, float]#

The range of drift (in meters) to add to the ray starting positions (xyz). Defaults to (0.0, 0.0).

For floating base robots, this is useful for simulating drift in the robot’s pose estimation.

visualizer_cfg: VisualizationMarkersCfg#: The configuration object for the visualization markers. Defaults to RAY_CASTER_MARKER_CFG.

Note

This attribute is only used when debug visualization is enabled.

offset: OffsetCfg#: The offset pose of the sensor’s frame from the sensor’s parent frame. Defaults to identity.

data_types: list[str]#: List of sensor names/types to enable for the camera. Defaults to [“distance_to_image_plane”].

orbit.sensors

Contents

orbit.sensors#

Sensor Base#

USD Camera#

Contact Sensor#

Frame Transformer#

Ray-Cast Sensor#

Ray-Cast Camera#