embodichain.lab.sim.sensors

embodichain.lab.sim.sensors#

Sensors attached to the simulation scene.

Camera, StereoCamera, and ContactSensor all derive from BaseSensor and expose batched TensorDict data buffers.

Overview#

Sensors attached to the simulation scene. Every sensor derives from BaseSensor (itself a BatchEntity), is configured through a SensorCfg subclass, and maintains a batched TensorDict data buffer of shape [num_envs]. The built-ins are Camera (single RGB-D camera with configurable intrinsics/extrinsics), StereoCamera (a camera pair with a baseline transform and optional disparity), and ContactSensor (collision detection between rigid bodies and articulation links via Warp kernels).

Classes

SensorCfg

Configuration class for sensors.

BaseSensor

Base class for sensor abstraction in the simulation engine.

CameraCfg

Configuration class for Camera.

Camera

Base class for sensor abstraction in the simulation engine.

StereoCameraCfg

Configuration class for StereoCamera.

StereoCamera

Base class for sensor abstraction in the simulation engine.

ContactSensorCfg

Configuration class for contact sensors.

ArticulationContactFilterCfg

Configuration for filtering contacts from an articulation's links.

ContactSensor

Sensor to get contacts from rigid bodies and articulation links.

Sensor#

class embodichain.lab.sim.sensors.BaseSensor[source]#

Bases: BatchEntity

Base class for sensor abstraction in the simulation engine.

Sensors should inherit from this class and implement the update and get_data methods.

Methods:

`__init__`(config[, device, num_instances])
`begin_control_step`()	Begin interval accumulation for a sensor that samples physics substeps.
`destroy`()	Destroy all entities managed by this batch entity.
`get_arena_pose`([to_matrix])	Get the pose of the sensor in the arena frame.
`get_data`()	Retrieve data from the sensor.
`reset`([env_ids])	Reset the entity to its initial state.
`update`(**kwargs)	Update the sensor state based on the current simulation state.
`update_physics_step`(dt)	Sample a sensor that opts into physics substeps.

Attributes:

requires_substep_update

Whether the environment must sample this sensor each physics substep.

__init__(config, device=device(type='cpu'), *, num_instances=None)[source]#

begin_control_step()[source]#

Begin interval accumulation for a sensor that samples physics substeps.

Return type:: None

destroy()#

Destroy all entities managed by this batch entity.

Return type:: None

abstract get_arena_pose(to_matrix=False)[source]#

Get the pose of the sensor in the arena frame.

Parameters:: to_matrix (bool) – If True, return the pose as a 4x4 transformation matrix; otherwise return (x, y, z, qx, qy, qz, qw).
Return type:: Tensor
Returns:: A tensor representing the pose of the sensor in the arena frame.

get_data()[source]#

Retrieve data from the sensor.

Parameters:: copy – If True, return a copy of the data buffer. Defaults to True.
Return type:: TensorDict
Returns:: The data collected by the sensor.

property requires_substep_update: bool#: Whether the environment must sample this sensor each physics substep.

reset(env_ids=None)[source]#

Reset the entity to its initial state.

Parameters:: env_ids (Sequence[int] | None) – The environment IDs to reset. If None, reset all environments.
Return type:: None

abstract update(**kwargs)[source]#

Update the sensor state based on the current simulation state.

This method is called periodically to ensure the sensor data is up-to-date.

Parameters:: **kwargs – Additional keyword arguments for sensor update.
Return type:: None

update_physics_step(dt)[source]#

Sample a sensor that opts into physics substeps.

Parameters:: dt (float) – Elapsed physics time in seconds.
Return type:: None

class embodichain.lab.sim.sensors.SensorCfg[source]#

Configuration class for sensors.

This class can be extended to include specific sensor configurations.

Classes:

OffsetCfg

Configuration of the sensor offset relative to the parent frame.

Methods:

`from_dict`(init_dict)	Initialize the configuration from a dictionary.
`get_data_types`()	Get the data types supported by this sensor configuration.

Attributes:

`init_local_pose`	4x4 transformation matrix of the root in local frame.
`init_pos`	Position of the root in simulation world frame.
`init_rot`	Euler angles (in degree) of the root in simulation world frame.

class OffsetCfg[source]#

Configuration of the sensor offset relative to the parent frame.

Attributes:

`parent`	Name of the parent frame.
`pos`	Position of the sensor in the parent frame.
`quat`	Orientation in the parent frame as `(x, y, z, w)`.

parent: str | None#

Name of the parent frame. If not specified, the sensor will be placed in the arena frame.

This is usually the case when the sensor is not attached to any specific object, eg, link of a robot arm.

pos: Tuple[float, float, float]#: Position of the sensor in the parent frame. Defaults to (0.0, 0.0, 0.0).

quat: Tuple[float, float, float, float]#: Orientation in the parent frame as (x, y, z, w). Defaults to identity.

classmethod from_dict(init_dict)[source]#

Initialize the configuration from a dictionary.

Return type:: SensorCfg

abstract get_data_types()[source]#

Get the data types supported by this sensor configuration.

Return type:: List[str]
Returns:: A list of data types that this sensor configuration supports.

init_local_pose: np.ndarray | None#: 4x4 transformation matrix of the root in local frame. If specified, it will override init_pos and init_rot.

init_pos: tuple[float, float, float]#: Position of the root in simulation world frame. Defaults to (0.0, 0.0, 0.0).

init_rot: tuple[float, float, float]#: Euler angles (in degree) of the root in simulation world frame. Defaults to (0.0, 0.0, 0.0).

Camera#

class embodichain.lab.sim.sensors.Camera[source]#

Bases: BaseSensor

Base class for sensor abstraction in the simulation engine.

Sensors should inherit from this class and implement the update and get_data methods.

Methods:

`__init__`(config[, device])
`attach_to_parent_nodes`(parent_nodes)	Attach camera views to one resolved parent node per environment.
`begin_control_step`()	Begin interval accumulation for a sensor that samples physics substeps.
`destroy`()	Remove render cameras before releasing their World-owned group.
`get_arena_pose`([to_matrix])	Get the pose of the sensor in the arena frame.
`get_data`()	Retrieve data from the sensor.
`get_intrinsics`()	Get the camera intrinsics for both left and right cameras.
`get_local_pose`([to_matrix])	Get the local pose of the camera.
`look_at`(eye, target[, up, env_ids])	Set the camera to look at a target point.
`reset`([env_ids])	Reset the entity to its initial state.
`set_intrinsics`(intrinsics[, env_ids])	Set the camera intrinsics for both left and right cameras.
`set_local_pose`(pose[, env_ids])	Set the local pose of the camera.
`update`(**kwargs)	Update the sensor data.
`update_physics_step`(dt)	Sample a sensor that opts into physics substeps.

Attributes:

`group_id`	Get the camera group ID in the dexsim world.
`is_attached`	Return whether all camera views are attached to parent nodes.
`requires_substep_update`	Whether the environment must sample this sensor each physics substep.

__init__(config, device=device(type='cpu'), *, owner)[source]#

attach_to_parent_nodes(parent_nodes)[source]#

Attach camera views to one resolved parent node per environment.

Parameters:

parent_nodes (Sequence[object]) – Parent render nodes ordered by environment index.

Raises:

RuntimeError – If the number of parent nodes does not match the number of camera instances.
ValueError – If any parent node is missing.

Return type:

None

begin_control_step()#

Begin interval accumulation for a sensor that samples physics substeps.

Return type:: None

destroy()[source]#

Remove render cameras before releasing their World-owned group.

Return type:: None

get_arena_pose(to_matrix=False)[source]#

Get the pose of the sensor in the arena frame.

Parameters:: to_matrix (bool) – If True, return the pose as a 4x4 transformation matrix. If False, return (x, y, z, qx, qy, qz, qw).
Return type:: Tensor
Returns:: A tensor representing the pose of the sensor in the arena frame.

get_data()#

Retrieve data from the sensor.

Parameters:: copy – If True, return a copy of the data buffer. Defaults to True.
Return type:: TensorDict
Returns:: The data collected by the sensor.

get_intrinsics()[source]#

Get the camera intrinsics for both left and right cameras.

Returns:: The intrinsics for the left camera with shape (N, 3, 3).
Return type:: torch.Tensor

get_local_pose(to_matrix=False)[source]#

Get the local pose of the camera.

Parameters:: to_matrix (bool) – If True, return the pose as a 4x4 matrix. If False, return (x, y, z, qx, qy, qz, qw).
Returns:: The local pose of the camera.
Return type:: torch.Tensor

property group_id: int#

Get the camera group ID in the dexsim world.

Returns:: The camera group ID.
Return type:: int

property is_attached: bool#

Return whether all camera views are attached to parent nodes.

Returns:: True after parent attachment and extrinsics application succeed.

look_at(eye, target, up=None, env_ids=None)[source]#

Set the camera to look at a target point.

Parameters:

eye (torch.Tensor) – The position of the camera (eye) with shape (N, 3).
target (torch.Tensor) – The point the camera should look at (target) with shape (N, 3).
up (torch.Tensor | None) – The up direction vector. If None, defaults to [0, 0, 1].
env_ids (Sequence[int] | None) – The environment IDs to set the look at for. If None, set for all environments.

Return type:

None

property requires_substep_update: bool#: Whether the environment must sample this sensor each physics substep.

reset(env_ids=None)[source]#

Reset the entity to its initial state.

Parameters:: env_ids (Sequence[int] | None) – The environment IDs to reset. If None, reset all environments.
Return type:: None

set_intrinsics(intrinsics, env_ids=None)[source]#

Set the camera intrinsics for both left and right cameras.

Parameters:

intrinsics (torch.Tensor) – The intrinsics for the left camera with shape (4,) / (3, 3) or (N, 4) / (N, 3, 3).
env_ids (Sequence[int] | None) – The environment ids to set the intrinsics. If None, set for all environments.

Return type:

None

set_local_pose(pose, env_ids=None)[source]#

Set the local pose of the camera.

Note

The pose uses OpenGL camera axes: +Y is up and -Z is forward.

Parameters:

pose (torch.Tensor) – The local pose as (N, 4, 4) matrices or (N, 7) vectors in (x, y, z, qx, qy, qz, qw) order.
env_ids (Sequence[int] | None) – The environment IDs to set the pose for. If None, set for all environments.

Return type:

None

update(**kwargs)[source]#

Update the sensor data.

The supported data types are:

color: RGB images with shape (B, H, W, 4) and dtype torch.uint8
depth: Depth images with shape (B, H, W) and dtype torch.float32
mask: Instance segmentation masks with shape (B, H, W) and dtype torch.int32
normal: Normal images with shape (B, H, W, 3) and dtype torch.float32
position: Position images with shape (B, H, W, 3) and dtype torch.float32

Parameters:: **kwargs – Additional keyword arguments for sensor update.
Return type:: None

update_physics_step(dt)#

Sample a sensor that opts into physics substeps.

Parameters:: dt (float) – Elapsed physics time in seconds.
Return type:: None

class embodichain.lab.sim.sensors.CameraCfg[source]#

Bases: SensorCfg

Configuration class for Camera.

Classes:

`ExtrinsicsCfg`	Configuration class for camera extrinsics.
`OffsetCfg`	Configuration of the sensor offset relative to the parent frame.

Methods:

`from_dict`(init_dict)	Initialize the configuration from a dictionary.
`get_data_types`()	Get the data types supported by this sensor configuration.
`get_view_attrib`()	Get the view attributes for the camera.

Attributes:

`init_local_pose`	4x4 transformation matrix of the root in local frame.
`init_pos`	Position of the root in simulation world frame.
`init_rot`	Euler angles (in degree) of the root in simulation world frame.
`visualization_role`	Role used to group this camera in live visualization previews.

class ExtrinsicsCfg[source]#

Bases: OffsetCfg

Configuration class for camera extrinsics.

The extrinsics define the position and orientation of the camera in the 3D world. If eye, target, and up are provided, they will be used to compute the extrinsics. Otherwise, the position and orientation will be set to the defaults.

SimulationManager resolves parent as a Robot or Articulation link name. Use "<asset_uid>/<link_name>" when the link name is ambiguous. Explicit mounted poses use OpenGL axes (-Z forward, +Y up). Unparented explicit poses use optical axes (+Z forward, +Y down) and are converted in Camera.reset(). Look-at inputs specify viewing direction directly in either case.

Attributes:

`parent`	Name of the parent frame.
`pos`	Position of the sensor in the parent frame.
`quat`	Orientation in the parent frame as `(x, y, z, w)`.
`up`	Alternative way to specify the camera extrinsics using eye, target, and up vectors.

parent: str | None#

Name of the parent frame. If not specified, the sensor will be placed in the arena frame.

This is usually the case when the sensor is not attached to any specific object, eg, link of a robot arm.

pos: Tuple[float, float, float]#: Position of the sensor in the parent frame. Defaults to (0.0, 0.0, 0.0).

quat: Tuple[float, float, float, float]#: Orientation in the parent frame as (x, y, z, w). Defaults to identity.

up: Optional[Tuple[float, float, float]]#: Alternative way to specify the camera extrinsics using eye, target, and up vectors.

class OffsetCfg#

Bases: object

Configuration of the sensor offset relative to the parent frame.

Attributes:

`parent`	Name of the parent frame.
`pos`	Position of the sensor in the parent frame.
`quat`	Orientation in the parent frame as `(x, y, z, w)`.

parent: str | None#

Name of the parent frame. If not specified, the sensor will be placed in the arena frame.

This is usually the case when the sensor is not attached to any specific object, eg, link of a robot arm.

pos: Tuple[float, float, float]#: Position of the sensor in the parent frame. Defaults to (0.0, 0.0, 0.0).

quat: Tuple[float, float, float, float]#: Orientation in the parent frame as (x, y, z, w). Defaults to identity.

classmethod from_dict(init_dict)#

Initialize the configuration from a dictionary.

Return type:: SensorCfg

get_data_types()[source]#

Get the data types supported by this sensor configuration.

Return type:: List[str]
Returns:: A list of data types that this sensor configuration supports.

get_view_attrib()[source]#

Get the view attributes for the camera.

The camera view whcich is used to render the scene Default view attributes for the camera are: [COLOR, DEPTH, MASK] The supported view attributes are:

COLOR: RGBA images

DEPTH: Depth images

MASK: Instance segmentation masks

NORMAL: Normal images

POSITION: Position images with 3D coordinates.

Return type:: ViewFlags
Returns:: The view attributes for the camera.

init_local_pose: np.ndarray | None#: 4x4 transformation matrix of the root in local frame. If specified, it will override init_pos and init_rot.

init_pos: tuple[float, float, float]#: Position of the root in simulation world frame. Defaults to (0.0, 0.0, 0.0).

init_rot: tuple[float, float, float]#: Euler angles (in degree) of the root in simulation world frame. Defaults to (0.0, 0.0, 0.0).

visualization_role: Literal['sensor', 'record']#: Role used to group this camera in live visualization previews.

Attachment resolution#

SimulationManager.add_sensor() delegates parent-name resolution to the function below before creating camera views. It accepts a canonical link name or "<asset_uid>/<link_name>" to distinguish links shared by several assets. The resolver receives the scene asset mapping explicitly and queries Articulation.get_link_render_nodes(); it does not look up a global manager.

Camera.attach_to_parent_nodes() then attaches the resolved nodes, reapplies parent-relative extrinsics, and updates is_attached. Stereo cameras share this path. Directly constructed cameras require an explicit attachment call.

resolve_parent_nodes(parent, assets, num_envs)

Resolve a canonical Robot or Articulation link in each environment.

embodichain.lab.sim.sensors.attachment.resolve_parent_nodes(parent, assets, num_envs)[source]#

Resolve a canonical Robot or Articulation link in each environment.

Parameters:

parent (str) – A canonical link name, or "<asset_uid>/<link_name>" to disambiguate links shared by multiple assets.
assets (Mapping[str, Articulation]) – Scene asset UIDs mapped to Articulation or Robot instances.
num_envs (int) – Expected number of camera and asset instances.

Return type:

list[Node]

Returns:

Parent render nodes ordered by environment index.

Raises:

ValueError – If the parent link is missing or ambiguous.
RuntimeError – If the asset count differs from num_envs, or an environment is missing the link or its render node.

Stereo Camera#

class embodichain.lab.sim.sensors.StereoCamera[source]#

Bases: Camera

Base class for sensor abstraction in the simulation engine.

Sensors should inherit from this class and implement the update and get_data methods.

Methods:

`__init__`(config[, device])
`attach_to_parent_nodes`(parent_nodes)	Attach camera views to one resolved parent node per environment.
`begin_control_step`()	Begin interval accumulation for a sensor that samples physics substeps.
`destroy`()	Remove render cameras before releasing their World-owned group.
`get_arena_pose`([to_matrix])	Get the pose of the sensor in the arena frame.
`get_data`()	Retrieve data from the sensor.
`get_intrinsics`()	Get the camera intrinsics for both left and right cameras.
`get_left_right_arena_pose`()	Get the local pose of the left and right cameras.
`get_local_pose`([to_matrix])	Get the local pose of the camera.
`look_at`(eye, target[, up, env_ids])	Set the camera to look at a target point.
`reset`([env_ids])	Reset the entity to its initial state.
`set_intrinsics`(intrinsics[, ...])	Set the camera intrinsics for both left and right cameras.
`set_local_pose`(pose[, env_ids])	Set the local pose of the camera.
`update`(**kwargs)	Update the sensor data.
`update_physics_step`(dt)	Sample a sensor that opts into physics substeps.

Attributes:

`group_id`	Get the camera group ID in the dexsim world.
`is_attached`	Return whether all camera views are attached to parent nodes.
`requires_substep_update`	Whether the environment must sample this sensor each physics substep.

__init__(config, device=device(type='cpu'), *, owner)[source]#

attach_to_parent_nodes(parent_nodes)#

Attach camera views to one resolved parent node per environment.

Parameters:

parent_nodes (Sequence[object]) – Parent render nodes ordered by environment index.

Raises:

RuntimeError – If the number of parent nodes does not match the number of camera instances.
ValueError – If any parent node is missing.

Return type:

None

begin_control_step()#

Begin interval accumulation for a sensor that samples physics substeps.

Return type:: None

destroy()#

Remove render cameras before releasing their World-owned group.

Return type:: None

get_arena_pose(to_matrix=False)#

Get the pose of the sensor in the arena frame.

Parameters:: to_matrix (bool) – If True, return the pose as a 4x4 transformation matrix. If False, return (x, y, z, qx, qy, qz, qw).
Return type:: Tensor
Returns:: A tensor representing the pose of the sensor in the arena frame.

get_data()#

Retrieve data from the sensor.

Parameters:: copy – If True, return a copy of the data buffer. Defaults to True.
Return type:: TensorDict
Returns:: The data collected by the sensor.

get_intrinsics()[source]#

Get the camera intrinsics for both left and right cameras.

Returns:: The intrinsics for the left and right cameras with shape (B, 3, 3).
Return type:: Tuple[torch.Tensor, torch.Tensor]

get_left_right_arena_pose()[source]#

Get the local pose of the left and right cameras.

Returns:: The local pose of the left camera with shape (num_envs, 4, 4).
Return type:: torch.Tensor

get_local_pose(to_matrix=False)#

Get the local pose of the camera.

Parameters:: to_matrix (bool) – If True, return the pose as a 4x4 matrix. If False, return (x, y, z, qx, qy, qz, qw).
Returns:: The local pose of the camera.
Return type:: torch.Tensor

property group_id: int#

Get the camera group ID in the dexsim world.

Returns:: The camera group ID.
Return type:: int

property is_attached: bool#

Return whether all camera views are attached to parent nodes.

Returns:: True after parent attachment and extrinsics application succeed.

look_at(eye, target, up=None, env_ids=None)#

Set the camera to look at a target point.

Parameters:

eye (torch.Tensor) – The position of the camera (eye) with shape (N, 3).
target (torch.Tensor) – The point the camera should look at (target) with shape (N, 3).
up (torch.Tensor | None) – The up direction vector. If None, defaults to [0, 0, 1].
env_ids (Sequence[int] | None) – The environment IDs to set the look at for. If None, set for all environments.

Return type:

None

property requires_substep_update: bool#: Whether the environment must sample this sensor each physics substep.

reset(env_ids=None)#

Reset the entity to its initial state.

Parameters:: env_ids (Sequence[int] | None) – The environment IDs to reset. If None, reset all environments.
Return type:: None

set_intrinsics(intrinsics, right_intrinsics=None, env_ids=None)[source]#

Set the camera intrinsics for both left and right cameras.

Parameters:

intrinsics (torch.Tensor) – The intrinsics for the left camera with shape (4,) / (3, 3) or (B, 4) / (B, 3, 3).
right_intrinsics (torch.Tensor | None) – The intrinsics for the right camera with shape (4,) / (3, 3) or (B, 4) / (B, 3, 3). If None, use the same intrinsics as the left camera.
env_ids (Sequence[int] | None) – The environment ids to set the intrinsics. If None, set for all environments.

Return type:

None

set_local_pose(pose, env_ids=None)#

Set the local pose of the camera.

Note

The pose uses OpenGL camera axes: +Y is up and -Z is forward.

Parameters:

pose (torch.Tensor) – The local pose as (N, 4, 4) matrices or (N, 7) vectors in (x, y, z, qx, qy, qz, qw) order.
env_ids (Sequence[int] | None) – The environment IDs to set the pose for. If None, set for all environments.

Return type:

None

update(**kwargs)[source]#

Update the sensor data.

The supported data types are:

color: RGB images with shape (B, H, W, 4) and dtype torch.uint8
depth: Depth images with shape (B, H, W, 1) and dtype torch.float32
mask: Instance segmentation masks with shape (B, H, W, 1) and dtype torch.int32
normal: Normal images with shape (B, H, W, 3) and dtype torch.float32
position: Position images with shape (B, H, W, 3) and dtype torch.float32
disparity: Disparity images with shape (B, H, W, 1) and dtype torch.float32

Parameters:: **kwargs – Additional keyword arguments for sensor update.
Return type:: None

update_physics_step(dt)#

Sample a sensor that opts into physics substeps.

Parameters:: dt (float) – Elapsed physics time in seconds.
Return type:: None

class embodichain.lab.sim.sensors.StereoCameraCfg[source]#

Bases: CameraCfg

Configuration class for StereoCamera.

Classes:

`ExtrinsicsCfg`	Configuration class for camera extrinsics.
`OffsetCfg`	Configuration of the sensor offset relative to the parent frame.

Methods:

`from_dict`(init_dict)	Initialize the configuration from a dictionary.
`get_data_types`()	Get the data types supported by this sensor configuration.
`get_view_attrib`()	Get the view attributes for the camera.

Attributes:

`init_local_pose`	4x4 transformation matrix of the root in local frame.
`init_pos`	Position of the root in simulation world frame.
`init_rot`	Euler angles (in degree) of the root in simulation world frame.
`left_to_right`	Get the transformation matrix from left camera to right camera.
`right_to_left`	Get the transformation matrix from right camera to left camera.
`visualization_role`	Role used to group this camera in live visualization previews.

class ExtrinsicsCfg#

Bases: OffsetCfg

Configuration class for camera extrinsics.

The extrinsics define the position and orientation of the camera in the 3D world. If eye, target, and up are provided, they will be used to compute the extrinsics. Otherwise, the position and orientation will be set to the defaults.

SimulationManager resolves parent as a Robot or Articulation link name. Use "<asset_uid>/<link_name>" when the link name is ambiguous. Explicit mounted poses use OpenGL axes (-Z forward, +Y up). Unparented explicit poses use optical axes (+Z forward, +Y down) and are converted in Camera.reset(). Look-at inputs specify viewing direction directly in either case.

Attributes:

`parent`	Name of the parent frame.
`pos`	Position of the sensor in the parent frame.
`quat`	Orientation in the parent frame as `(x, y, z, w)`.
`up`	Alternative way to specify the camera extrinsics using eye, target, and up vectors.

parent: str | None#

Name of the parent frame. If not specified, the sensor will be placed in the arena frame.

This is usually the case when the sensor is not attached to any specific object, eg, link of a robot arm.

pos: Tuple[float, float, float]#: Position of the sensor in the parent frame. Defaults to (0.0, 0.0, 0.0).

quat: Tuple[float, float, float, float]#: Orientation in the parent frame as (x, y, z, w). Defaults to identity.

up: Optional[Tuple[float, float, float]]#: Alternative way to specify the camera extrinsics using eye, target, and up vectors.

class OffsetCfg#

Bases: object

Configuration of the sensor offset relative to the parent frame.

Attributes:

`parent`	Name of the parent frame.
`pos`	Position of the sensor in the parent frame.
`quat`	Orientation in the parent frame as `(x, y, z, w)`.

parent: str | None#

Name of the parent frame. If not specified, the sensor will be placed in the arena frame.

This is usually the case when the sensor is not attached to any specific object, eg, link of a robot arm.

pos: Tuple[float, float, float]#: Position of the sensor in the parent frame. Defaults to (0.0, 0.0, 0.0).

quat: Tuple[float, float, float, float]#: Orientation in the parent frame as (x, y, z, w). Defaults to identity.

classmethod from_dict(init_dict)#

Initialize the configuration from a dictionary.

Return type:: SensorCfg

get_data_types()[source]#

Get the data types supported by this sensor configuration.

Return type:: List[str]
Returns:: A list of data types that this sensor configuration supports.

get_view_attrib()#

Get the view attributes for the camera.

The camera view whcich is used to render the scene Default view attributes for the camera are: [COLOR, DEPTH, MASK] The supported view attributes are:

COLOR: RGBA images

DEPTH: Depth images

MASK: Instance segmentation masks

NORMAL: Normal images

POSITION: Position images with 3D coordinates.

Return type:: ViewFlags
Returns:: The view attributes for the camera.

init_local_pose: np.ndarray | None#: 4x4 transformation matrix of the root in local frame. If specified, it will override init_pos and init_rot.

init_pos: tuple[float, float, float]#: Position of the root in simulation world frame. Defaults to (0.0, 0.0, 0.0).

init_rot: tuple[float, float, float]#: Euler angles (in degree) of the root in simulation world frame. Defaults to (0.0, 0.0, 0.0).

property left_to_right: Tensor#: Get the transformation matrix from left camera to right camera.

property right_to_left: Tensor#: Get the transformation matrix from right camera to left camera.

visualization_role: Literal['sensor', 'record']#: Role used to group this camera in live visualization previews.

Contact Sensor#

class embodichain.lab.sim.sensors.ContactSensor[source]#

Bases: BaseSensor

Sensor to get contacts from rigid bodies and articulation links.

The sensor preserves every backend-emitted contact row that passes the backend’s positive-impulse filter. Geometry-only Newton solvers retain all candidate contact rows.

Methods:

`__init__`(config[, device, owner])
`begin_control_step`()	Start a new interval without discarding persistent contact timing.
`create_history`(name, actor_ids, *[, ...])	Register a sensor-owned reduction with explicit counterpart filtering.
`destroy`()	Destroy all entities managed by this batch entity.
`filter_by_user_ids`(item_user_ids[, env_ids])	Filter contact report by backend-neutral contact actor IDs.
`get_actor_ids`(uid[, link_names])	Resolve a scene entity's ordered contact IDs for every environment.
`get_actor_info`(actor_id)	Resolve an ID from the `user_ids` field to its Spawn identity.
`get_arena_pose`([to_matrix])	Not used.
`get_data`()	Retrieve data from the sensor.
`get_local_pose`([to_matrix])	Get the local pose of the camera.
`reset`([env_ids])	Clear contact samples and histories only for the selected rows.
`set_local_pose`(pose[, env_ids])	Set the local pose of the camera.
`update`(**kwargs)	Update the sensor state based on the current simulation state.
`update_physics_step`(dt)	Sample contacts and advance registered histories after physics.

Attributes:

`contact_capabilities`	Capabilities reported by the active DexSim contact binding.
`dropped_contacts`	Rows lost across the current control interval, or latest standalone update.
`item_env_ids`	Environment IDs of the selected contact actors.
`item_user_env_ids_map`	Compatibility map from contact actor ID to environment ID.
`item_user_ids`	Backend-neutral actor IDs selected by the contact query.
`max_total_contacts`	Get the maximum total number of contacts across all environments.
`requires_substep_update`	Whether this sensor participates in the physics-substep lifecycle.
`total_current_contacts`	Get the current total number of contacts across all environments.

__init__(config, device=device(type='cpu'), *, owner=None)[source]#

begin_control_step()[source]#

Start a new interval without discarding persistent contact timing.

Return type:: None

property contact_capabilities: ContactQueryCapabilities#: Capabilities reported by the active DexSim contact binding.

create_history(name, actor_ids, *, counterpart_ids=None, include_unknown_counterpart=False, force_threshold=0.0)[source]#

Register a sensor-owned reduction with explicit counterpart filtering.

Parameters:

name (str) – Unique history name within the sensor.
actor_ids (Tensor) – Body IDs to monitor, in per-environment order.
counterpart_ids (Tensor | None) – Allowed counterparts; None accepts all contacts.
include_unknown_counterpart (bool) – Also accept unidentified query counterparts.
force_threshold (float) – Minimum contact force in newtons for timing events.

Return type:

ContactHistory

Returns:

History that automatically enables substep sampling and selected-row reset.

destroy()#

Destroy all entities managed by this batch entity.

Return type:: None

property dropped_contacts: int#

Rows lost across the current control interval, or latest standalone update.

This diagnostic reads device counters only when requested. Query overflow and per-environment scatter overflow are counted separately and summed.

filter_by_user_ids(item_user_ids, env_ids=None)[source]#

Filter contact report by backend-neutral contact actor IDs.

Parameters:

item_user_ids (Tensor) – Actor IDs from this sensor’s user_ids field.
env_ids (Optional[Sequence[int]]) – Environment IDs to filter. If None, filter all environments.

Returns:

A TensorDict containing only the filtered contacts for the specified environments.

Return type:

data

get_actor_ids(uid, link_names=None)[source]#

Resolve a scene entity’s ordered contact IDs for every environment.

Parameters:

uid (str) – Robot, articulation or rigid-object UID in the Spawn scene.
link_names (Optional[Sequence[str]]) – Ordered articulation links; None selects the rigid actor.

Return type:

Tensor

Returns:

IDs shaped (environments, selected bodies). Global actors are broadcast.

get_actor_info(actor_id)[source]#

Resolve an ID from the user_ids field to its Spawn identity.

Return type:: ContactActorInfo

get_arena_pose(to_matrix=False)[source]#

Not used.

Parameters:: to_matrix (bool) – If True, return the pose as a 4x4 transformation matrix.
Return type:: Tensor | None
Returns:: A tensor representing the pose of the sensor in the arena frame.

get_data()[source]#

Retrieve data from the sensor.

Return type:: TensorDict
Returns:: Batched contact data. position is in the Arena frame; normal points from actor 0 toward actor 1; friction and impulse are impulses; distance is signed separation; user_ids contains backend-neutral contact actor IDs; and is_valid marks rows populated by the latest update. Values in invalid rows are unspecified and may come from an earlier update.

get_local_pose(to_matrix=False)[source]#

Get the local pose of the camera.

Parameters:: to_matrix (bool) – If True, return the pose as a 4x4 matrix. If False, return as a quaternion.
Returns:: The local pose of the camera.
Return type:: torch.Tensor

item_env_ids: torch.Tensor | None#: Environment IDs of the selected contact actors.

item_user_env_ids_map: torch.Tensor | None#: Compatibility map from contact actor ID to environment ID.

item_user_ids: torch.Tensor | None#: Backend-neutral actor IDs selected by the contact query.

property max_total_contacts: int#

Get the maximum total number of contacts across all environments.

Returns:: Maximum total number of contacts.
Return type:: int

property requires_substep_update: bool#: Whether this sensor participates in the physics-substep lifecycle.

reset(env_ids=None)[source]#

Clear contact samples and histories only for the selected rows.

Parameters:: env_ids (Optional[Sequence[int]]) – Rows to reset. None selects every row.
Return type:: None

set_local_pose(pose, env_ids=None)[source]#

Set the local pose of the camera.

Note: The pose should be in the OpenGL coordinate system, which means the Y is up and Z is forward.

Parameters:

pose (torch.Tensor) – The local pose to set, should be a 4x4 transformation matrix.
env_ids (Sequence[int] | None) – The environment IDs to set the pose for. If None, set for all environments.

Return type:

None

property total_current_contacts: int#

Get the current total number of contacts across all environments.

Note

This method returns the total number of contacts detected in the most recent update.

Returns:: Total number of contacts.
Return type:: int

update(**kwargs)[source]#

Update the sensor state based on the current simulation state.

This method is called periodically to ensure the sensor data is up-to-date.

Parameters:: **kwargs – Additional keyword arguments for sensor update.
Return type:: None

update_physics_step(dt)[source]#

Sample contacts and advance registered histories after physics.

Parameters:: dt (float) – Elapsed physics time in seconds.
Return type:: None

class embodichain.lab.sim.sensors.ContactSensorCfg[source]#

Bases: SensorCfg

Configuration class for contact sensors.

This class defines the configuration for contact sensors that detect collisions between rigid bodies and articulation links.

Classes:

OffsetCfg

Configuration of the sensor offset relative to the parent frame.

Attributes:

`articulation_cfg_list`	articulation link contact filter configs
`filter_need_both_actor`	Whether to filter contact only when both actors are in the filter list.
`init_local_pose`	4x4 transformation matrix of the root in local frame.
`init_pos`	Position of the root in simulation world frame.
`init_rot`	Euler angles (in degree) of the root in simulation world frame.
`max_contacts_per_env`	Maximum number of contacts per environment the sensor can handle.
`rigid_uid_list`	rigid body contact filter configs

Methods:

`from_dict`(init_dict)	Initialize the configuration from a dictionary.
`get_data_types`()	Get the data types supported by this sensor configuration.

class OffsetCfg#

Bases: object

Configuration of the sensor offset relative to the parent frame.

Attributes:

`parent`	Name of the parent frame.
`pos`	Position of the sensor in the parent frame.
`quat`	Orientation in the parent frame as `(x, y, z, w)`.

parent: str | None#

Name of the parent frame. If not specified, the sensor will be placed in the arena frame.

This is usually the case when the sensor is not attached to any specific object, eg, link of a robot arm.

pos: Tuple[float, float, float]#: Position of the sensor in the parent frame. Defaults to (0.0, 0.0, 0.0).

quat: Tuple[float, float, float, float]#: Orientation in the parent frame as (x, y, z, w). Defaults to identity.

articulation_cfg_list: list[ArticulationContactFilterCfg]#: articulation link contact filter configs

filter_need_both_actor: bool#: Whether to filter contact only when both actors are in the filter list.

classmethod from_dict(init_dict)#

Initialize the configuration from a dictionary.

Return type:: SensorCfg

abstract get_data_types()#

Get the data types supported by this sensor configuration.

Return type:: List[str]
Returns:: A list of data types that this sensor configuration supports.

init_local_pose: np.ndarray | None#: 4x4 transformation matrix of the root in local frame. If specified, it will override init_pos and init_rot.

init_pos: tuple[float, float, float]#: Position of the root in simulation world frame. Defaults to (0.0, 0.0, 0.0).

init_rot: tuple[float, float, float]#: Euler angles (in degree) of the root in simulation world frame. Defaults to (0.0, 0.0, 0.0).

max_contacts_per_env: int#: Maximum number of contacts per environment the sensor can handle.

rigid_uid_list: list[str]#: rigid body contact filter configs

class embodichain.lab.sim.sensors.ArticulationContactFilterCfg[source]#

Bases: object

Configuration for filtering contacts from an articulation’s links.

This class defines which articulation and which links to monitor for contact events.

Methods:

`__init__`([articulation_uid, link_name_list])
`copy`(**kwargs)	Return a new object replacing specified fields with new values.
`from_dict`(init_dict)	Initialize the configuration from a dictionary.
`replace`(**kwargs)	Return a new object replacing specified fields with new values.
`to_dict`()	Convert an object into dictionary recursively.
`validate`([prefix])	Check the validity of configclass object.

Attributes:

`articulation_uid`	Articulation unique identifier.
`link_name_list`	link names in the articulation whose contacts need to be filtered.

__init__(articulation_uid=<factory>, link_name_list=<factory>)#

articulation_uid: str#: Articulation unique identifier.

copy(**kwargs)#

Return a new object replacing specified fields with new values.

This is especially useful for frozen classes. Example usage:

@configclass(frozen=True)
class C:
    x: int
    y: int

c = C(1, 2)
c1 = c.replace(x=3)
assert c1.x == 3 and c1.y == 2

Parameters:

obj (object) – The object to replace.
**kwargs – The fields to replace and their new values.

Return type:

object

Returns:

The new object.

classmethod from_dict(init_dict)[source]#

Initialize the configuration from a dictionary.

Parameters:: init_dict (dict[str, str | list[str]]) – Dictionary containing configuration parameters.
Returns:: The initialized configuration.
Return type:: ArticulationContactFilterCfg

link_name_list: list[str]#: link names in the articulation whose contacts need to be filtered.

replace(**kwargs)#

Return a new object replacing specified fields with new values.

This is especially useful for frozen classes. Example usage:

@configclass(frozen=True)
class C:
    x: int
    y: int

c = C(1, 2)
c1 = c.replace(x=3)
assert c1.x == 3 and c1.y == 2

Parameters:

obj (object) – The object to replace.
**kwargs – The fields to replace and their new values.

Return type:

object

Returns:

The new object.

to_dict()#

Convert an object into dictionary recursively.

Note

Ignores all names starting with “__” (i.e. built-in methods).

Parameters:: obj (object) – An instance of a class to convert.
Raises:: ValueError – When input argument is not an object.
Return type:: dict[str, Any]
Returns:: Converted dictionary mapping.

validate(prefix='')#

Check the validity of configclass object.

This function checks if the object is a valid configclass object. A valid configclass object contains no MISSING entries.

Parameters:

obj (object) – The object to check.
prefix (str) – The prefix to add to the missing fields. Defaults to ‘’.

Return type:

list[str]

Returns:

A list of missing fields.

Raises:

TypeError – When the object is not a valid configuration object.

Contact history#

Contact filtering and substep timing are owned by the sensor layer. Tasks select body and counterpart IDs, then consume interval events and completed flight durations without advancing the sensor state.

Sensor-owned contact reduction and timing for a selected set of actors.

Classes:

ContactHistory

Accumulate selected contacts across one control interval.

class embodichain.lab.sim.sensors.contact_history.ContactHistory[source]#

Accumulate selected contacts across one control interval.

contact and force describe the latest physics sample; found, peak_force and first_contact retain events from every substep. current_air_time measures an unfinished flight; last_air_time captures its duration on landing, including the landing sampling interval. Reading these tensors never advances time or clears an event.

Parameters:

actor_ids (Tensor) – Selected body IDs, shaped (environments, bodies).
counterpart_ids (Tensor | None) – Allowed counterpart IDs per environment; None accepts all.
include_unknown_counterpart (bool) – Accept the query’s unknown counterpart (-1) in addition to explicitly selected IDs. This does not identify ground.
force_threshold (float) – Minimum individual contact-force magnitude for a timing event.

Methods:

`__init__`(actor_ids, *[, counterpart_ids, ...])
`begin_control_step`()	Clear interval aggregates while retaining contact timing.
`reset`([env_ids])	Clear only selected environment rows, including unfinished timing.
`update`(data, dt)	Reduce a new physics sample and advance timing exactly once.

__init__(actor_ids, *, counterpart_ids=None, include_unknown_counterpart=False, force_threshold=0.0)[source]#

begin_control_step()[source]#

Clear interval aggregates while retaining contact timing.

Return type:: None

reset(env_ids=None)[source]#

Clear only selected environment rows, including unfinished timing.

Parameters:: env_ids (Sequence[int] | Tensor | None) – Rows to clear. None selects every environment.
Return type:: None

update(data, dt)[source]#

Reduce a new physics sample and advance timing exactly once.

Parameters:

data (Mapping[str, Tensor]) – ContactSensor data in its documented actor/impulse convention.
dt (float) – Duration of the physics sample in seconds.

Return type:

None

Creating a contact history automatically enables physics-substep sampling. CUDA history reduction consumes compact contact rows and device counts through sparse Warp kernels. The dropped_contacts diagnostic sums query loss and scatter overflow across the current control interval; standalone updates without a history report the most recent sample.

`SensorCfg`	Configuration class for sensors.
`BaseSensor`	Base class for sensor abstraction in the simulation engine.
`CameraCfg`	Configuration class for Camera.
`Camera`	Base class for sensor abstraction in the simulation engine.
`StereoCameraCfg`	Configuration class for StereoCamera.
`StereoCamera`	Base class for sensor abstraction in the simulation engine.
`ContactSensorCfg`	Configuration class for contact sensors.
`ArticulationContactFilterCfg`	Configuration for filtering contacts from an articulation's links.
`ContactSensor`	Sensor to get contacts from rigid bodies and articulation links.

embodichain.lab.sim.sensors

Contents

embodichain.lab.sim.sensors#

Overview#

Sensor#

Camera#

Attachment resolution#

Stereo Camera#

Contact Sensor#

Contact history#