Configure and Run an Embodied Task Program

Configure and Run an Embodied Task Program#

This tutorial deploys Task Program’s built-in Semantic Calls through explicit environment, integration, embodiment, and execution-policy configuration. The calls are declarative language instructions, not an independent execution API.

This page is for integration authors who bind scenes, robot profiles and execution policies. To customize an existing task by editing only its program, start with Customize your first expert trajectory.

The optional Authoring a Task Program in Python introduces the typed language without starting simulation; it is not a prerequisite for the Quickstart.

Use Atomic Skills instead when Python code should construct and execute typed ActionInvocation values directly. The architecture and ownership rules are described in Task Program architecture.

What you will configure#

A supported configuration-defined Task Program has six core owners. A task that also collects augmented trajectories may add a separate expansion declaration:

  1. program.yaml owns the embodiment-independent task flow and targets;

  2. integration.yaml owns task-specific semantic requirements, its nested scene_binding, resource defaults, action options, effect monitors, and runtime services;

  3. a reusable environment component (for example envs/default.yaml and envs/newton.yaml) owns the physical scene and ordinary Gym environment values;

  4. an embodiment component owns one robot, its sensors, and optional semantic skill profile;

  5. an execution-policy component owns motion, recovery, runner, and effect assurance settings; and

  6. task.<embodiment>.yaml selects those reusable owners for one runnable Gym ID.

There is no separate physical scene.yaml or task component. In particular, the reusable environment component contains no id or task_program field, so the same physical environment can also be selected by a handwritten-trajectory deployment.

The shared EmbodiedEnv and Task Program adapter then lower Semantic Calls to Atomic Skills while retaining the normal stepping, recording, and reset lifecycle.

In a source checkout, the official repeated Pick/Place example is a complete reference. Its physical backends live under envs/; the UR5 deployment additionally references expansion/repeated_pick_place.yaml through expansion.config:

  • extensions/embodichain_tasks/configs/tasks/manipulation/repeated_pick_place/envs/default.yaml and envs/newton.yaml;

  • extensions/embodichain_tasks/configs/tasks/manipulation/repeated_pick_place/task_program/;

  • extensions/embodichain_tasks/configs/tasks/manipulation/repeated_pick_place/task.ur5.yaml;

  • extensions/embodichain_tasks/configs/tasks/manipulation/repeated_pick_place/task.franka.yaml;

  • extensions/embodichain_tasks/configs/tasks/manipulation/repeated_pick_place/expansion/repeated_pick_place.yaml;

  • extensions/embodichain_tasks/configs/components/embodiments/ur5_dh_pgi_140_80.yaml;

  • extensions/embodichain_tasks/configs/components/execution_policies/trajectory_open_loop.yaml.

These files have explicit ownership rather than generic deep-merge semantics. The deployment must select components that satisfy the task’s scene and embodiment contracts.

Declare object-centric intent#

The program names semantic scene entities and robot resources. It contains no joint names, planners, controller commands, simulator objects, or Python callables:

program_id: repeated_cube_pick_place
targets:
  drop_pose:
    kind: cyclic_pose
    values:
      - position: [-0.40, 0.48, 0.10]
        quaternion_xyzw: [0.0, 0.0, 0.0, 1.0]
      - position: [-0.42, -0.08, 0.10]
        quaternion_xyzw: [0.0, 0.0, 0.0, 1.0]
program:
  kind: repeat
  count: 3
  body:
    kind: segment
    name: move_cube
    steps:
      kind: sequence
      items:
        - kind: invoke
          call:
            kind: pick
            object: cube
        - kind: invoke
          call:
            kind: place
            object: cube
            at: {kind: target_ref, target: drop_pose}

cube is a canonical semantic object rather than a simulator UID. The program deliberately contains no integration selection or robot-specific resource ID. The task integration supplies the logical Pick and Place resource defaults, and the selected embodiment supplies compatible concrete endpoints.

Compose the deployment#

The reusable environment owns physical simulation entities and ordinary environment values, including its explicit physics backend. Here is the complete Default environment used by the Quickstart:

environment_id: repeated_pick_place
physics: default
max_episodes: 1
max_episode_steps: 1200
num_envs: 1
arena_space: 2.5

simulation:
  light:
    direct:
      - uid: main_light
        light_type: sun
        color: [0.6, 0.6, 0.6]
        intensity: 5.0
        direction: [0.0, 0.0, -1.0]
      - uid: rect_light
        light_type: rect
        color: [1.0, 1.0, 1.0]
        intensity: 8.0
        init_pos: [-0.42, 0.08, 0.8]
        direction: [0.0, 0.0, -1.0]
        rect_width: 0.5
        rect_height: 0.5
  background: []
  rigid_object:
    - uid: cube
      shape:
        shape_type: Cube
        size: [0.05, 0.05, 0.05]
      body_type: dynamic
      init_pos: [-0.42, -0.08, 0.025]
      attrs:
        mass_props:
          mass: 0.05
        rigid_props:
          linear_damping: 0.2
          angular_damping: 0.2
        material_props:
          dynamic_friction: 0.97
          static_friction: 0.99
  rigid_object_group: []
  articulation: []

env:
  sim_steps_per_control: 4
  events: {}
  dataset:
    lerobot:
      func: LeRobotRecorder
      mode: save
      params:
        # Structured dataset output; robot_type comes from the selected embodiment.
        save_path: /tmp/repeated-pick-place/datasets
        extra:
          scene_type: tabletop
          task_description: repeated_pick_place_combined
          data_type: sim
        use_videos: true

The complete Franka deployment selects environment variants, Task Program components and an embodiment. It does not enable expansion:

id: TaskProgramRepeatedPickPlace-Franka-v1

environment:
  default: envs/default.yaml
  newton: envs/newton.yaml

task_program:
  program: task_program/program.yaml
  integration: task_program/integration.yaml
  execution_policy: ../../../components/execution_policies/trajectory_open_loop.yaml

embodiment:
  component: ../../../components/embodiments/franka_panda.yaml
  overrides:
    init_rot: [0.0, 0.0, 180.0]

During config_to_cfg(), the component resolver expands these selections, checks that every semantic simulation_uid exists in the physical scene, validates the scene and embodiment contracts, binds the exact scene-registry, robot-profile, and policy-preset IDs into the otherwise unbound program, and registers the common EmbodiedEnv. Component references resolve relative to the runnable deployment file.

Select effect assurance explicitly#

Every execution policy must declare how semantic state is authorized to advance. The trajectory-only component selects projected assurance:

policy_id: trajectory_open_loop_v1
preset_id: trajectory
effect_assurance: projected

projected applies the action plan’s expected symbolic effect after command completion. The matching task integration must declare an empty effect_monitors mapping, and successful completion is not proof that the physical grasp or release occurred.

For a physically verified curated call, select verified in the execution policy and map every used Pick, Place, or HandOver call to an explicit monitor in the task integration:

profile:
  effect_monitors:
    hand_over:
      monitor_id: builtin.composite_effect
      revision: "1"
      params: {consecutive_samples: 10}

The integration must also install the evidence providers required by that monitor. Command acknowledgement alone is not physical evidence.

Load and validate without simulation#

From the source checkout root, the provider-independent API performs strict file decoding before any live environment is constructed. Unlike the CLI, load_task_program() takes a filesystem path, not a package-resource reference:

from embodichain.lab.task_program import (
    TaskProgramIntegrationCfg,
    load_task_program,
)

program = load_task_program(
    "extensions/embodichain_tasks/configs/tasks/manipulation/"
    "repeated_pick_place/task_program/program.yaml",
    integration=TaskProgramIntegrationCfg(
        robot_profile="ur5_dh_pgi_140_80",
        scene_registry="task_program_repeated_pick_place",
        runtime_preset="trajectory",
    ),
)
print(program.program_id)

The explicit integration argument represents the trusted deployment selection that config_to_cfg() normally derives from the selected embodiment, scene binding, and execution policy. A configured program.yaml must not declare those IDs itself.

Unknown fields, duplicate YAML keys, invalid references, executable payloads, and bounded-expansion violations raise a Task Program config error. Static compilation additionally checks the program against a provider-free SceneManifest; it does not observe simulation state or generate controller actions.

Run through the environment entry point#

Run the complete configuration with the normal environment CLI:

embodichain run-task \
  --gym_config \
  embodichain_tasks/configs/tasks/manipulation/repeated_pick_place/task.ur5.yaml \
  --headless

To select another compatible program while keeping the trusted integration:

embodichain run-task \
  --gym_config path/to/task.ur5.yaml \
  --task-program path/to/program.yaml \
  --headless

The adapter produces lazy DemoSegment actions. The normal demo executor, not the semantic package, calls env.step() and owns recording and reset.

Add settling and task validation#

A segment can distinguish physical-call verification from task acceptance:

program:
  kind: segment
  name: deliver_can
  steps:
    kind: invoke
    call:
      kind: hand_over
      object: can
      final_target: {kind: target_ref, target: delivery_pose}
  post:
    - kind: wait_stable
      entity: can
      preset: rigid_object
  validators:
    - kind: object_near_target
      object: can
      target: delivery_pose
      position_tolerance: 0.12

The effect monitor verifies the semantic HandOver transition. wait_stable advances environment behavior after motion. object_near_target decides whether the final task segment is acceptable. These are intentionally separate boundaries.

Use registered semantic calls carefully#

Use RegisteredSemanticCallCfg only when a task needs an allowlisted shared lowering that is not one of Pick, Place, or HandOver. Its arguments remain declarative. The trusted simulation registration owns the matching lowerer factory and fingerprints its call ID, revision, and target atomic descriptor.

Do not place a callable, dotted import, simulator object, or task-local motion generator in the program. If a reusable behavior is fundamentally a new motion primitive, add it to Atomic Actions first and then expose a declarative registered call through Task Program.

Next steps#

Contents