Parallel-Gripper Grasp Generation

Parallel-Gripper Grasp Generation#

The GraspKit toolkit generates feasible grasp poses for parallel-jaw grippers from a target object’s triangle mesh. It supports programmatic antipodal sampling, browser-based grasp-region annotation, collision filtering, candidate ranking, and on-disk caching of sampled contact pairs.

This page also demonstrates how to execute a generated pose with a robot arm. It covers scene initialization, robot and object creation, grasp pose computation, and trajectory execution in the simulation loop.

Processing Pipeline#

Grasp generation has three stages:

Sample surface points and find antipodal contact pairs on the full mesh or an annotated region.
Construct 6-DoF grasp frames that align the gripper opening axis with each contact pair and respect the requested approach direction.
Remove candidates that collide with the object or ground, rank the remaining poses, and return the best candidates with their required opening lengths.

Tutorial Source#

The tutorial corresponds to the grasp_generator.py script in the scripts/tutorials/grasp directory.

Code for grasp_generator.py

# ----------------------------------------------------------------------------
# Copyright (c) 2021-2026 DexForce Technology Co., Ltd.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ----------------------------------------------------------------------------

"""
This script demonstrates the creation and simulation of a robot that grasps a rigid mug
in a simulated environment using the SimulationManager and grasp planning utilities.
"""

from __future__ import annotations

import argparse
import time
from embodichain.cli.sim import add_sim_args_to_parser


def build_parser() -> argparse.ArgumentParser:
    """Build CLI options without initializing simulation resources."""
    parser = argparse.ArgumentParser(
        description="Create and simulate a robot in SimulationManager"
    )
    add_sim_args_to_parser(parser)
    return parser


def parse_arguments(argv: list[str] | None = None) -> argparse.Namespace:
    """
    Parse command-line arguments to configure the simulation.

    Returns:
        argparse.Namespace: Parsed arguments including number of environments and rendering options.
    """
    parser = build_parser()
    return parser.parse_args() if argv is None else parser.parse_args(argv)


if __name__ == "__main__":
    # Parse before importing optional simulation/planning dependencies.
    _cli_args = parse_arguments()


import numpy as np
import torch

from embodichain.lab.sim import SimulationManager, SimulationManagerCfg
from embodichain.lab.visualization import visualization_cfg_from_args
from embodichain.lab.sim.objects import Robot, RigidObject
from embodichain.compute.trajectory import interpolate_with_distance
from embodichain.lab.sim.shapes import MeshCfg, MeshCollisionCfg
from embodichain.lab.sim.motion.solvers import URSolverCfg
from embodichain.data import get_data_path
from embodichain.toolkits.graspkit import ParallelJawGripperModelCfg
from embodichain.utils import logger
from embodichain.lab.sim.cfg import (
    RenderCfg,
    physics_cfg_for_backend,
    JointDrivePropertiesCfg,
    RobotCfg,
    LightCfg,
    RigidBodyPhysicsCfg,
    RigidObjectCfg,
    URDFCfg,
)
from embodichain.toolkits.graspkit.pg_grasp import (
    AntipodalGraspPoseGenerator,
    AntipodalGraspPoseGeneratorCfg,
    GraspAnnotationCfg,
    ParallelJawGraspCollisionCfg,
)


def initialize_simulation(args) -> SimulationManager:
    """
    Initialize the simulation environment based on the provided arguments.

    Args:
        args (argparse.Namespace): Parsed command-line arguments.

    Returns:
        SimulationManager: Configured simulation manager instance.
    """
    config = SimulationManagerCfg(
        headless=True,
        device=args.device,
        render_cfg=RenderCfg(renderer=args.renderer),
        physics_cfg=physics_cfg_for_backend(args.physics),
        physics_dt=1.0 / 100.0,
        arena_space=2.5,
        visualization=visualization_cfg_from_args(args),
    )
    sim = SimulationManager(config)

    light = sim.add_light(
        cfg=LightCfg(
            uid="main_light",
            color=(0.6, 0.6, 0.6),
            intensity=30.0,
            init_pos=(1.0, 0, 3.0),
        )
    )

    return sim


def create_robot(sim: SimulationManager, position=[0.0, 0.0, 0.0]) -> Robot:
    """
    Create and configure a robot with an arm and a dexterous hand in the simulation.

    Args:
        sim (SimulationManager): The simulation manager instance.

    Returns:
        Robot: The configured robot instance added to the simulation.
    """
    # Retrieve URDF paths for the robot arm and hand
    ur10_urdf_path = get_data_path("UniversalRobots/UR10/UR10.urdf")
    gripper_urdf_path = get_data_path("DH_PGC_140_50_M/DH_PGC_140_50_M.urdf")
    # Configure the robot with its components and control properties
    cfg = RobotCfg(
        uid="UR10",
        urdf_cfg=URDFCfg(
            components=[
                {"component_type": "arm", "urdf_path": ur10_urdf_path},
                {"component_type": "hand", "urdf_path": gripper_urdf_path},
            ]
        ),
        joint_drive_props=JointDrivePropertiesCfg(
            stiffness={"Joint[0-9]": 1e4, "FINGER[1-2]": 1e3},
            damping={"Joint[0-9]": 1e3, "FINGER[1-2]": 1e2},
            max_effort={"Joint[0-9]": 1e5, "FINGER[1-2]": 1e4},
            drive_type="force",
        ),
        control_parts={
            "arm": ["Joint[0-9]"],
            "eef": ["FINGER[1-2]"],
        },
        solver_cfg={
            "arm": URSolverCfg(
                ur_type="ur10",
                tcp=[
                    [0.0, 1.0, 0.0, 0.0],
                    [-1.0, 0.0, 0.0, 0.0],
                    [0.0, 0.0, 1.0, 0.14],
                    [0.0, 0.0, 0.0, 1.0],
                ],
            )
        },
        init_qpos=[0.0, -np.pi / 2, -np.pi / 2, np.pi / 2, -np.pi / 2, 0.0, 0.0, 0.0],
        init_pos=position,
    )
    return sim.add_robot(cfg=cfg)


def create_obj(sim: SimulationManager):
    mug_cfg = RigidObjectCfg(
        uid="table",
        shape=MeshCfg(
            fpath=get_data_path("BakeTextureObj/hdr_color_mesh.ply"),
            collision=MeshCollisionCfg(
                approximation="convex_decomposition",
                max_hulls=16,
                acd_method="coacd",
            ),
        ),
        attrs=RigidBodyPhysicsCfg.from_dict(
            {
                "mass_props": {"mass": 0.01},
                "material_props": {"dynamic_friction": 0.97, "static_friction": 0.99},
            }
        ),
        init_pos=[0.55, 0.0, 0.08],
        init_rot=[0.0, 0.0, 0.0],
    )
    mug = sim.add_rigid_object(cfg=mug_cfg)
    return mug


def get_grasp_traj(sim: SimulationManager, robot: Robot, grasp_xpos: torch.Tensor):
    num_envs = sim.num_envs
    rest_arm_qpos = robot.get_qpos("arm")

    approach_xpos = grasp_xpos.clone()
    approach_xpos[:, 2, 3] += 0.1

    _, qpos_approach = robot.compute_ik(
        pose=approach_xpos, joint_seed=rest_arm_qpos, name="arm"
    )
    _, qpos_grasp = robot.compute_ik(
        pose=grasp_xpos, joint_seed=qpos_approach, name="arm"
    )
    hand_open_qpos = torch.tensor([0.00, 0.00], dtype=torch.float32, device=sim.device)
    hand_close_qpos = torch.tensor(
        [0.025, 0.025], dtype=torch.float32, device=sim.device
    )

    arm_trajectory = torch.cat(
        [
            rest_arm_qpos[:, None, :],
            qpos_approach[:, None, :],
            qpos_grasp[:, None, :],
            qpos_grasp[:, None, :],
            qpos_approach[:, None, :],
            rest_arm_qpos[:, None, :],
        ],
        dim=1,
    )
    hand_trajectory = torch.cat(
        [
            hand_open_qpos[None, None, :].repeat(num_envs, 1, 1),
            hand_open_qpos[None, None, :].repeat(num_envs, 1, 1),
            hand_open_qpos[None, None, :].repeat(num_envs, 1, 1),
            hand_close_qpos[None, None, :].repeat(num_envs, 1, 1),
            hand_close_qpos[None, None, :].repeat(num_envs, 1, 1),
            hand_close_qpos[None, None, :].repeat(num_envs, 1, 1),
        ],
        dim=1,
    )
    all_trajectory = torch.cat([arm_trajectory, hand_trajectory], dim=-1)
    interp_trajectory = interpolate_with_distance(
        trajectory=all_trajectory, interp_num=200, device=sim.device
    )
    return interp_trajectory


if __name__ == "__main__":
    import time

    args = _cli_args
    sim = initialize_simulation(args)
    robot = create_robot(sim, position=[0.0, 0.0, 0.0])
    obj = create_obj(sim)
    sim.prepare()

    # get mug grasp pose
    if not args.headless:
        sim.open_window()

    # Annotate part of the mug to be grasped by following the instructions in the visualization window:
    # 1. View grasp object in browser (e.g http://localhost:11801)
    # 2. press 'Rect Select Region', select grasp region
    # 3. press 'Confirm Selection' to finish grasp region selection.

    start_time = time.time()

    # Construct one standalone generator. The same instance can also be passed
    # to AtomicActionEngine(grasp_pose_generators={"hand": grasp_generator}).
    grasp_generator = AntipodalGraspPoseGenerator(
        ParallelJawGripperModelCfg(
            model_id="tutorial_parallel_jaw",
            min_opening_width=0.003,
            max_opening_width=0.088,
            finger_length=0.078,
        ),
        algorithm_cfg=AntipodalGraspPoseGeneratorCfg(
            sample_count=10_000,
            max_candidates=30,
        ),
        collision_cfg=ParallelJawGraspCollisionCfg(
            point_sample_density=0.012,
            filter_ground_collision=True,
        ),
        annotation_cfg=GraspAnnotationCfg(
            selection_mode="whole_mesh",
            viser_port=11801,
        ),
    )

    # Extract target-local geometry. The generator, rather than the target,
    # owns all algorithm and end-effector parameters.
    vertices = obj.get_vertices(env_ids=[0], scale=True)[0]
    triangles = obj.get_triangles(env_ids=[0])[0]

    # Compute grasp poses per environment
    approach_direction = torch.tensor(
        [0, 0, -1], dtype=torch.float32, device=sim.device
    )
    obj_poses = obj.get_local_pose(to_matrix=True)
    rest_xpos = robot.compute_fk(
        qpos=robot.get_qpos("arm"), name="arm", to_matrix=True
    )[0]
    grasp_success, grasp_xpos, _ = grasp_generator.get_best_grasp_poses(
        mesh_vertices=vertices,
        mesh_triangles=triangles,
        obj_poses=obj_poses,
        approach_direction=approach_direction,
    )
    if not bool(grasp_success.all().item()):
        logger.log_warning("At least one environment has no valid grasp pose.")
        grasp_xpos = torch.where(
            grasp_success[:, None, None],
            grasp_xpos,
            rest_xpos.expand_as(grasp_xpos),
        )
    cost_time = time.time() - start_time
    logger.log_info(f"Get grasp pose cost time: {cost_time:.2f} seconds")

    grab_traj = get_grasp_traj(sim, robot, grasp_xpos)
    input("Press Enter to start the grab mug demo...")
    n_waypoint = grab_traj.shape[1]
    for i in range(n_waypoint):
        robot.set_qpos(grab_traj[:, i, :])
        sim.update(step=4)
        time.sleep(1e-2)
    input("Press Enter to exit the simulation...")

Tutorial Walkthrough#

Configuring the simulation#

Command-line arguments are parsed with argparse to select the number of parallel environments, the compute device, and optional rendering features such as renderer backend and headless mode.

The parsed arguments are passed to initialize_simulation, which builds a SimulationManagerCfg and creates the SimulationManager instance. When ray tracing is enabled a directional cfg.LightCfg is also added to the scene.

def initialize_simulation(args) -> SimulationManager:
    """
    Initialize the simulation environment based on the provided arguments.

    Args:
        args (argparse.Namespace): Parsed command-line arguments.

    Returns:
        SimulationManager: Configured simulation manager instance.
    """
    config = SimulationManagerCfg(
        headless=True,
        device=args.device,
        render_cfg=RenderCfg(renderer=args.renderer),
        physics_cfg=physics_cfg_for_backend(args.physics),
        physics_dt=1.0 / 100.0,
        arena_space=2.5,
        visualization=visualization_cfg_from_args(args),
    )
    sim = SimulationManager(config)

    light = sim.add_light(
        cfg=LightCfg(
            uid="main_light",
            color=(0.6, 0.6, 0.6),
            intensity=30.0,
            init_pos=(1.0, 0, 3.0),
        )
    )

    return sim

Creating a robot and a target object#

A UR10 arm with a parallel-jaw gripper is created via SimulationManager.add_robot(). The gripper URDF and drive properties are configured so that the arm joints and finger joints can be controlled independently.

def create_robot(sim: SimulationManager, position=[0.0, 0.0, 0.0]) -> Robot:
    """
    Create and configure a robot with an arm and a dexterous hand in the simulation.

    Args:
        sim (SimulationManager): The simulation manager instance.

    Returns:
        Robot: The configured robot instance added to the simulation.
    """
    # Retrieve URDF paths for the robot arm and hand
    ur10_urdf_path = get_data_path("UniversalRobots/UR10/UR10.urdf")
    gripper_urdf_path = get_data_path("DH_PGC_140_50_M/DH_PGC_140_50_M.urdf")
    # Configure the robot with its components and control properties
    cfg = RobotCfg(
        uid="UR10",
        urdf_cfg=URDFCfg(
            components=[
                {"component_type": "arm", "urdf_path": ur10_urdf_path},
                {"component_type": "hand", "urdf_path": gripper_urdf_path},
            ]
        ),
        joint_drive_props=JointDrivePropertiesCfg(
            stiffness={"Joint[0-9]": 1e4, "FINGER[1-2]": 1e3},
            damping={"Joint[0-9]": 1e3, "FINGER[1-2]": 1e2},
            max_effort={"Joint[0-9]": 1e5, "FINGER[1-2]": 1e4},
            drive_type="force",
        ),
        control_parts={
            "arm": ["Joint[0-9]"],
            "eef": ["FINGER[1-2]"],
        },
        solver_cfg={
            "arm": URSolverCfg(
                ur_type="ur10",
                tcp=[
                    [0.0, 1.0, 0.0, 0.0],
                    [-1.0, 0.0, 0.0, 0.0],
                    [0.0, 0.0, 1.0, 0.14],
                    [0.0, 0.0, 0.0, 1.0],
                ],
            )
        },
        init_qpos=[0.0, -np.pi / 2, -np.pi / 2, np.pi / 2, -np.pi / 2, 0.0, 0.0, 0.0],
        init_pos=position,
    )
    return sim.add_robot(cfg=cfg)

The target object (a mug) is loaded as a objects.RigidObject from a PLY mesh file:

def create_obj(sim: SimulationManager):
    mug_cfg = RigidObjectCfg(
        uid="table",
        shape=MeshCfg(
            fpath=get_data_path("BakeTextureObj/hdr_color_mesh.ply"),
            collision=MeshCollisionCfg(
                approximation="convex_decomposition",
                max_hulls=16,
                acd_method="coacd",
            ),
        ),
        attrs=RigidBodyPhysicsCfg.from_dict(
            {
                "mass_props": {"mass": 0.01},
                "material_props": {"dynamic_friction": 0.97, "static_friction": 0.99},
            }
        ),
        init_pos=[0.55, 0.0, 0.08],
        init_rot=[0.0, 0.0, 0.0],
    )
    mug = sim.add_rigid_object(cfg=mug_cfg)
    return mug

Annotating and computing grasp poses#

Grasp generation is performed by AntipodalGraspPoseGenerator. It implements the robot-independent GraspPoseGenerator contract and the shared ParallelJawGraspPoseGenerator base for two-finger parallel-jaw grippers.

The generator owns the gripper model, algorithm, collision, and annotation configuration. Target-local mesh vertices and triangles are supplied to each call. This separation lets a handwritten environment call the service directly and lets an AtomicActionEngine install the same instance under grasp_pose_generators={"hand": generator}. Scene affordances do not own a live generator or robot-specific parameters.

For each environment, get_best_grasp_poses() returns a success flag, a (4, 4) world-frame grasp pose, and the required opening width. Antipodal contact pairs are cached and reused automatically. Set GraspAnnotationCfg.selection_mode="interactive" to select a partial region through Viser, or use "whole_mesh" for unattended generation.

The approach direction is the unit vector along which the gripper approaches the object. In this tutorial, we use a fixed approach direction (straight down in world frame) for simplicity, but it can be customized based on the task or object geometry.

    # Construct one standalone generator. The same instance can also be passed
    # to AtomicActionEngine(grasp_pose_generators={"hand": grasp_generator}).
    grasp_generator = AntipodalGraspPoseGenerator(
        ParallelJawGripperModelCfg(
            model_id="tutorial_parallel_jaw",
            min_opening_width=0.003,
            max_opening_width=0.088,
            finger_length=0.078,
        ),
        algorithm_cfg=AntipodalGraspPoseGeneratorCfg(
            sample_count=10_000,
            max_candidates=30,
        ),
        collision_cfg=ParallelJawGraspCollisionCfg(
            point_sample_density=0.012,
            filter_ground_collision=True,
        ),
        annotation_cfg=GraspAnnotationCfg(
            selection_mode="whole_mesh",
            viser_port=11801,
        ),
    )

    # Extract target-local geometry. The generator, rather than the target,
    # owns all algorithm and end-effector parameters.
    vertices = obj.get_vertices(env_ids=[0], scale=True)[0]
    triangles = obj.get_triangles(env_ids=[0])[0]

    # Compute grasp poses per environment
    approach_direction = torch.tensor(
        [0, 0, -1], dtype=torch.float32, device=sim.device
    )
    obj_poses = obj.get_local_pose(to_matrix=True)
    rest_xpos = robot.compute_fk(
        qpos=robot.get_qpos("arm"), name="arm", to_matrix=True
    )[0]
    grasp_success, grasp_xpos, _ = grasp_generator.get_best_grasp_poses(
        mesh_vertices=vertices,
        mesh_triangles=triangles,
        obj_poses=obj_poses,
        approach_direction=approach_direction,
    )
    if not bool(grasp_success.all().item()):
        logger.log_warning("At least one environment has no valid grasp pose.")
        grasp_xpos = torch.where(
            grasp_success[:, None, None],
            grasp_xpos,
            rest_xpos.expand_as(grasp_xpos),
        )
    cost_time = time.time() - start_time
    logger.log_info(f"Get grasp pose cost time: {cost_time:.2f} seconds")

Building and executing the grasp trajectory#

Once a grasp pose is obtained, a waypoint trajectory is built that moves the arm from its rest configuration to an approach pose (offset above the grasp), down to the grasp pose, closes the fingers, lifts, and returns. The trajectory is interpolated for smooth motion and executed step-by-step in the simulation loop.

def get_grasp_traj(sim: SimulationManager, robot: Robot, grasp_xpos: torch.Tensor):
    num_envs = sim.num_envs
    rest_arm_qpos = robot.get_qpos("arm")

    approach_xpos = grasp_xpos.clone()
    approach_xpos[:, 2, 3] += 0.1

    _, qpos_approach = robot.compute_ik(
        pose=approach_xpos, joint_seed=rest_arm_qpos, name="arm"
    )
    _, qpos_grasp = robot.compute_ik(
        pose=grasp_xpos, joint_seed=qpos_approach, name="arm"
    )
    hand_open_qpos = torch.tensor([0.00, 0.00], dtype=torch.float32, device=sim.device)
    hand_close_qpos = torch.tensor(
        [0.025, 0.025], dtype=torch.float32, device=sim.device
    )

    arm_trajectory = torch.cat(
        [
            rest_arm_qpos[:, None, :],
            qpos_approach[:, None, :],
            qpos_grasp[:, None, :],
            qpos_grasp[:, None, :],
            qpos_approach[:, None, :],
            rest_arm_qpos[:, None, :],
        ],
        dim=1,
    )
    hand_trajectory = torch.cat(
        [
            hand_open_qpos[None, None, :].repeat(num_envs, 1, 1),
            hand_open_qpos[None, None, :].repeat(num_envs, 1, 1),
            hand_open_qpos[None, None, :].repeat(num_envs, 1, 1),
            hand_close_qpos[None, None, :].repeat(num_envs, 1, 1),
            hand_close_qpos[None, None, :].repeat(num_envs, 1, 1),
            hand_close_qpos[None, None, :].repeat(num_envs, 1, 1),
        ],
        dim=1,
    )
    all_trajectory = torch.cat([arm_trajectory, hand_trajectory], dim=-1)
    interp_trajectory = interpolate_with_distance(
        trajectory=all_trajectory, interp_num=200, device=sim.device
    )
    return interp_trajectory

Configuration#

Configuration ownership is split by meaning:

ParallelJawGripperModelCfg describes physical opening limits, finger dimensions, and palm depth. A concrete EEF name belongs only in its model_id.
AntipodalGraspPoseGeneratorCfg controls sample count, angular deviations, approach variants, and result count.
ParallelJawGraspCollisionCfg controls collision margin, point density, decomposition cost, and ground filtering.
GraspAnnotationCfg controls whole-mesh versus interactive selection, the Viser port, and explicit cache refresh.

Running the Tutorial#

To run the script, execute the following command from the project root:

python scripts/tutorials/grasp/grasp_generator.py

A simulation window will open showing the robot and the mug. The tutorial uses whole-mesh annotation by default and therefore does not require a browser.

You can customize the run with additional arguments:

python scripts/tutorials/grasp/grasp_generator.py --num_envs <n> --device <cuda/cpu> --renderer <auto|hybrid|fast-rt|rt> --headless

The script computes a grasp pose, prints the elapsed time, and then waits for you to press Enter before executing the full grasp trajectory. Press Enter again to exit once the motion is complete.

Grasp Annotation CLI#

EmbodiChain provides a dedicated CLI for interactively annotating grasp regions on a mesh and caching the resulting antipodal point pairs, without requiring a full simulation environment. The CLI constructs the same AntipodalGraspPoseGenerator used by tasks and calls its prepare_mesh() method; it does not use a separate annotation-time generator.

Basic usage:

embodichain annotate-grasp --mesh_path /path/to/object.ply

This will:

Load the mesh file via trimesh.
Launch a browser-based annotator (default port 15531).
Open http://localhost:15531 in your browser, use Rect Select Region to highlight the graspable area, then click Confirm Selection.
Compute antipodal point pairs on the selected region and cache them to disk.

Common options:

embodichain annotate-grasp \
    --mesh_path /path/to/object.ply \
    --viser_port 15531 \
    --n_sample 20000 \
    --max_length 0.1 \
    --min_length 0.001

CLI options#
Option	Default	Description
`--mesh_path`	(required)	Path to the mesh file (`.ply`, `.obj`, `.stl`, etc.).
`--viser_port`	`15531`	Port for the browser-based annotation UI.
`--n_sample`	`20000`	Number of surface points to sample for antipodal pair detection.
`--max_length`	`0.1`	Maximum distance (metres) between antipodal pairs; should match the gripper’s maximum opening width.
`--min_length`	`0.001`	Minimum distance (metres) between antipodal pairs; filters out degenerate pairs.
`--device`	`cpu`	Compute device (`cpu` or `cuda`).