Running Tasks with run-task

Running Tasks with run-task#

embodichain run-task is the common entry point for loading a configured Gymnasium environment. It supports three primary workflows: inspecting an environment interactively, running an expert rollout while recording data, and replaying a previously recorded trajectory.

Execution modes#

Mode

Command switch

What it does

Typical use

Rollout

No mode switch

Generates task actions, steps the environment, and lets configured dataset or video recorders save each episode.

Expert demonstration expansion and task smoke tests.

Preview

--preview

Resets the environment and opens an interactive IPython session on request. Dataset saving is disabled automatically.

Inspecting scene state, stepping manually, and checking observations.

Replay

--replay --replay_trajectory <file>

Restores recorded states, re-applies recorded actions, or provides an interactive trajectory scrubber, depending on --replay_mode.

Reproducing a rollout and diagnosing dynamics or task behavior.

--preview and --replay are mutually exclusive. Browser visualization with --viser is a display backend rather than a fourth execution mode, so it can be used while running or replaying an environment.

Inputs and startup behavior#

Every invocation requires a gym config:

embodichain run-task --gym_config path/to/gym_config.yaml

JSON, YAML, and YML files are supported. The input must be a runnable config with a non-empty id. It may declare simulation, robot, sensors, managers, and recorders inline, or select reusable components. A pure physical env.yaml component uses environment_id instead of id and is not runnable by itself.

A configuration-defined Task Program deployment conventionally uses task.<embodiment>.yaml. It selects either one environment.component or a environment.default/newton mapping, all three task_program paths (program, integration, and execution_policy), and embodiment.component:

embodichain run-task \
    --gym_config embodichain_tasks/configs/tasks/manipulation/tableware/pour_water/task.cobotmagic.yaml

At startup, run-task:

  1. discovers installed task packages through the embodichain.tasks entry point and executes their initialization hooks;

  2. loads the runnable config, expands its selected physical components, and composes any declared Task Program components;

  3. merges launcher runtime values; optional overrides such as --device, --renderer, and --max_episodes apply only when supplied;

  4. creates the environment selected by the gym config’s id; and

  5. enters rollout, preview, or replay mode.

Task-facing trajectory expansion can be kept in a sibling configuration and referenced from the runnable task:

expansion:
  config: expansion/<task-profile>.yaml

The referenced file may contain runtime, policy, overrides, and collection. The runner resolves it relative to the task, applies its runtime overlay after expanding the selected environment variant, and merges task-local expansion fields on top. run-task then uses that declaration for Task Program expansion. --expansion-profile remains a per-run profile override; --expansion-recipe-indices selects explicit logical recipes. The legacy --expansion-candidate-indices spelling is still accepted as an alias.

Episode targets and batches#

All offline collection paths use one target count:

collection:
  target_episodes: 64
  max_attempts: 3
  selection:
    mode: sequential
    start_recipe_index: 0

target_episodes is the number of successfully committed environment rows. num_envs only sets the maximum width of one parallel batch, so target_episodes: 64 with num_envs: 16 runs four batches. A final partial batch uses only the rows needed to reach the target. max_attempts applies to one selected batch attempt and failed attempts are discarded before retrying.

The legacy --max_episodes CLI option remains supported and maps to collection.target_episodes. An environment component’s max_episodes is the fallback when no collection target is declared. A task-level collection target takes precedence over that component default; conflicting explicit target values are rejected during parsing.

Expansion selection has two modes:

# Select recipe 0 through recipe 63 and split them into batches automatically.
collection:
  target_episodes: 64
  selection:
    mode: sequential
    start_recipe_index: 0

# Select exactly four logical recipes and collect four rows.
collection:
  target_episodes: 4
  selection:
    mode: explicit
    recipe_indices: [0, 16, 32, 48]

Recipe indices identify logical expansion candidates. They do not identify batch starts. Affordance branches, scene families, trajectory variants, and visual profiles change the selected episode’s content; they do not increase the target count.

Collection manifests record target_episodes, planned_episodes, committed_episodes, rejected_episodes, attempts, batch_count, and the prepare, commit, and discard reset counts. A prepare reset happens before a batch, a commit reset finalizes selected rows, and a discard reset clears a failed attempt. None of these reset boundaries adds an episode.

The runnable config, or its selected environment component, must declare physics: default or physics: newton. With an environment.default/newton mapping, the requested variant is expanded before this check. See Configuration Guide for paired backend configuration fragments. That backend is file-owned: --physics selects a named variant when the task declares an environment.default/newton mapping and confirms it for a single component. It never rewrites the selected file. Omitting --device preserves an authored device or the selected backend’s default; supplying --device overrides both environment tensors and backend execution, including an explicit CPU selection for Newton.

Use embodichain run-task --help for the complete option list. The CLI Reference also lists defaults and visualization arguments.

Preview an environment#

Preview mode is intended for inspection before an expensive data-expansion run:

embodichain run-task \
    --gym_config path/to/gym_config.yaml \
    --preview

After constructing and resetting the environment, the terminal accepts:

  • p: enter an IPython session with env in scope;

  • q: close the preview.

IPython is required only when entering the embedded session. Install it with pip install ipython if the p command reports that it is unavailable.

Inside the session, use the regular Gymnasium interface and EmbodiChain helpers to inspect state or advance the simulation. For example:

# Inspect the current robot configuration.
env.unwrapped.robot.get_qpos()

# Advance with an action that is valid for this task.
obs, reward, terminated, truncated, info = env.step(action)

# Display an RGB sensor observation.
env.unwrapped.preview_sensor_data("camera")

Preview mode does not execute the task’s expert policy. It also sets filter_dataset_saving=True, so configured structured dataset recorders do not write debugging episodes. Visual randomization remains enabled unless --filter_visual_rand is supplied.

Native window and Viser#

--preview controls the interactive terminal workflow. --viser instead selects the browser visualization backend and makes the simulation headless:

embodichain run-task \
    --gym_config path/to/gym_config.yaml \
    --viser \
    --viser-host 127.0.0.1 \
    --viser-port 8080

Use Viser to inspect selected environments, camera frustums, and all camera RGB previews locally or remotely. The expanded preview panel separates cameras created by record_camera_data under Record cameras from configured observation sensors under Sensor cameras. Limit the published batch with --viser-env-ids, and tune scene, image, or soft-body publication rates with the corresponding --viser-*-fps options. See Browser visualization with Viser for browser controls and remote-access details.

Run and record#

Without --preview or --replay, run-task enters offline rollout mode. For each vector batch, it asks the task for its demonstration segments, applies every action through env.step(), and commits selected environment rows with an explicit reset. collection.target_episodes is the exact number of persisted environment rows, not the number of vector batches. For example, target_episodes=10 with num_envs=4 runs three batches and commits only two rows from the final batch. The legacy --max_episodes option maps to the same target:

embodichain run-task \
    --gym_config path/to/gym_config.yaml \
    --action_config path/to/action_config.yaml \
    --headless \
    --device cuda \
    --max_episodes 10

Headless execution is normally preferred for throughput. Use --filter_dataset_saving for a rollout smoke test that should not create a structured dataset.

Failed attempts are discarded and retried by default, up to collection.max_attempts (with demo_max_attempts retained as a legacy fallback). Set save_failed_episodes: true on a dataset functor to keep a failed or truncated attempt that contains recorded frames. Such a commit counts toward collection.target_episodes and is not retried. Empty plans and exceptions have no complete dataset transaction and are still discarded.

Multi-segment episodes#

An episode is the complete task; a segment is one semantic subtask inside it. For example, moving three objects is one episode containing three pick/place segments, even if each segment has its own motion trajectory. The task owns the number, order, and targets of those segments. The runner only manages the episode lifecycle, termination checks, retry, and commit/discard boundary.

Existing tasks implementing create_demo_action_list() remain compatible and are recorded as a single legacy segment. A multi-object task can instead implement a lazy segment planner:

from embodichain.lab.gym.envs import DemoSegment


def create_demo_segments(self):
    for object_uid in self.object_order:
        # This runs after the preceding segment, so planning sees the latest
        # scene state.
        actions = self.plan_pick_and_place(object_uid)
        yield DemoSegment(
            actions=actions,
            name="pick_and_place",
            target_uid=object_uid,
            instruction=f"Place {object_uid} in its target bin",
            # Segment validation is separate from Gym episode termination.
            validator=lambda uid=object_uid: self.is_object_placed(uid),
        )

Direct callers of generate_function() must no longer use num_traj as a sub-trajectory count. Only None and 1 are accepted; larger values raise ValueError. Move the repeated subtasks into create_demo_segments() so the task, rather than the runner, owns their order and validation.

Gym terminated and truncated always describe the whole episode, never an individual segment. A segment normally ends when its action iterable is exhausted; its optional zero-argument validator then returns one boolean per parallel environment. A failed validator aborts the batch. Episode-level success termination stops the remaining lazy plan without requesting another segment.

The executor checks terminal signals after every action and temporarily disables Gym auto-reset. Dataset recording is transactional: an explicit reset commits selected rows, while reset(options={"save_data": False}) discards them. Successful final validation always commits. A failed result commits only when a configured dataset functor enables save_failed_episodes; exceptions, interrupts, empty plans, and closing an environment with a live rollout abort pending structured data, videos, and trajectories.

Segment actions pass through the same action-dimension normalization used by legacy create_demo_action_list() tasks. A time-limit truncation is always an unsuccessful expert result, including when it occurs on the planner’s final action. It is discarded by default or retained as failed data when configured, so a task’s max_episode_steps must be greater than the longest valid expert plan when collecting successful demonstrations.

In a vectorized environment, segments and actions remain on one shared planner clock, but completion is tracked independently. When one environment reports success, its terminal result and recording cursor become sticky; subsequent shared actions use a safe hold/no-op command for that row while unfinished rows continue. Consequently, rollout and trajectory lengths may differ by row. The executor’s result remains batch-atomic: without failed-data saving every row must eventually succeed, while any failure or truncation invalidates the batch. With save_failed_episodes, selected failed rows are committed with their per-row failure metadata. Rows not needed to reach collection.target_episodes are explicitly discarded, so parallel collection never overshoots the requested episode count.

save_failed_episodes belongs beside func and mode, not inside params:

{
  "func": "LeRobotRecorder",
  "mode": "save",
  "save_failed_episodes": true,
  "params": {
    "robot_meta": {"robot_type": "UR5"}
  }
}

Declare the overall goal as env.task_instruction for a handwritten task or root instruction in program.yaml for a Task Program. Declare finer instructions on DemoSegment values or program segment nodes. Recorder instruction remains a legacy fallback. See Task and segment language for expert data for complete examples, resolution rules, and reading both levels in training.

Run the built-in three-cycle example#

The shipped embodichain_tasks/configs/tasks/manipulation/repeated_pick_place/task.ur5.yaml deployment combines a reusable physical environment with a UR5 parallel-gripper embodiment to pick up and place the same cube three times. Each cycle has a grasp segment and a placement segment, giving six lazy segments. Later calls are grounded against the scene state left by earlier calls.

embodichain run-task \
    --gym_config embodichain_tasks/configs/tasks/manipulation/repeated_pick_place/task.ur5.yaml \
    --headless \
    --device cuda \
    --max_episodes 1

The Default physical component (envs/default.yaml) already enables LeRobotRecorder under /tmp/repeated-pick-place/datasets. The command records one overall task with six semantic segment occurrences and two subtask descriptions. Add --filter_dataset_saving for an execution-only smoke test. See Expert Data Expansion for recorder setup and Inspect Recorded LeRobot Data for validation and preview.

Choose the recording output you need#

EmbodiChain uses “recording” for three related but distinct outputs:

Recording outputs#

Output

How it is enabled

Contents

Intended consumer

Structured dataset

A dataset manager in the gym config.

Observations, actions, episode/segment annotations, task metadata, and optionally sensor videos.

Imitation-learning or data-processing pipelines.

Debug or demo video

record_camera_data or record_camera_data_async as an interval event functor in the gym config.

Human-viewable RGB video from a configured camera pose.

Visual debugging, reports, and demonstrations.

Replay trajectory

--record_trajectory.

Per-step robot and object kinematic state, raw task action, and replay metadata in a PyTorch .pt file.

run-task --replay and ReplayWrapper.

Dataset video is still structured training data; it is not interchangeable with a replay trajectory. Conversely, --record_trajectory does not configure a LeRobot dataset or export an MP4.

For structured datasets, see Expert Data Expansion and Dataset Functors. For human-viewable video, see Event Functors.

After finalization, use the exact auto-numbered dataset directory with the validation, grouped split, and recovery commands in Validate, split, and recover expert datasets. For official LeRobot 0.6.1 language and depth features, follow Export expert data for LeRobot 0.6.1 in its independent Python environment.

Record replayable trajectories#

Add --record_trajectory to a normal rollout:

embodichain run-task \
    --gym_config path/to/gym_config.yaml \
    --action_config path/to/action_config.yaml \
    --record_trajectory \
    --trajectory_save_dir outputs/trajectories

The recorder stores the robot root pose and complete joint position, the raw action before ActionManager preprocessing, and the pose or joint state of scene rigid objects and articulations. Each environment in a vectorized rollout is tracked independently. A trajectory is saved only at an explicit commit reset; this is normally a successful episode, or a recorded failure when save_failed_episodes is enabled on a dataset functor. close() is a durability barrier for already committed writes, not an implicit commit, so an unfinished trajectory is discarded.

Files are named like traj_env0_000000.pt. When --trajectory_save_dir is omitted, they are written below:

${EMBODICHAIN_DATA_ROOT:-~/.cache/embodichain_data}/trajectories/<run_id>/

Each file contains states, actions, and meta. Metadata includes the actual per-environment lengths, segment ranges and targets, timestep, robot identity and DOF, active joint IDs, recorded object IDs, and original environment IDs. LeRobot exports also contain per-frame annotation.segment_* fields and a meta/embodichain_episodes.jsonl sidecar with the complete segment records. The overall episode instruction stays in LeRobot’s task field; each semantic segment is exposed through a per-frame subtask_index resolved by meta/subtasks.parquet. The task/program owns these instructions before recording begins; configuration, fallback, and readback examples are in Task and segment language for expert data. The final directory is also printed when the run finishes.

Replay a trajectory#

Replay requires the trajectory file and the gym config used to record it:

embodichain run-task \
    --gym_config path/to/gym_config.yaml \
    --replay \
    --replay_trajectory outputs/trajectories/traj_env0_000000.pt

Keep the robot, active joint selection, scene object IDs, action processing, and episode horizon compatible with the recording run. Robot DOF or active joint mismatches are rejected. Missing or additional scene objects produce warnings. A single-environment trajectory can be broadcast with --num_envs; a multi-environment trajectory requires the replay environment count to match. Recorded lengths that exceed the current episode horizon are clamped.

If the task needs an action config to construct its environment or ActionManager, pass the same --action_config during replay as during recording.

If the gym config contains dataset recorders, add --filter_dataset_saving when the replay is for inspection only. Control replay applies this filter automatically.

Kinematic replay#

Kinematic replay is the default and the best first check:

embodichain run-task \
    --gym_config path/to/gym_config.yaml \
    --replay \
    --replay_trajectory path/to/trajectory.pt \
    --replay_mode kinematic

Physics is disabled and recorded poses and joint positions are written into the scene at every step. This gives an exact visual/state reproduction and renders new observations, but it does not recompute meaningful task rewards or successes from the recorded actions. At completion, run-task reports the maximum robot joint-state error against the recording.

Dynamic replay#

Dynamic replay sends each recorded raw action back through env.step():

embodichain run-task \
    --gym_config path/to/gym_config.yaml \
    --replay \
    --replay_trajectory path/to/trajectory.pt \
    --replay_mode dynamic

Physics, ActionManager preprocessing, observations, rewards, and termination logic all run normally. Use this mode to test whether the behavior can be re-simulated rather than merely restored. Results may diverge if the replay configuration changes physics, control, randomization, assets, or timestep settings that are not stored in the trajectory file.

Interactive control replay#

Control mode uses kinematic state restoration and lets you scrub the trajectory from the terminal or Viser:

embodichain run-task \
    --gym_config path/to/gym_config.yaml \
    --replay \
    --replay_trajectory path/to/trajectory.pt \
    --replay_mode control

The commands are:

  • n: next step;

  • p or b: previous step;

  • <N> followed by Enter: jump to step N;

  • a: auto-play, then press any key to pause;

  • r: return to step 0;

  • q: quit.

Run control mode with a native render window. --headless leaves no window in which to see the scrubbed state. Alternatively, add --viser to open an expanded Replay control panel in the browser. Its integer Frame slider jumps directly to any recorded frame, stays synchronized with terminal and auto-play commands, and pauses auto-play when dragged.

Contents