Robot world model#
embodichain_models.dynamics.rwm predicts action-conditioned robot states
using frame-local increments and FP32 integration. It owns the model, state
representation, autoregressive objective, prediction state, replay and offline
recipe. Shared optimizer updates and recovery use
Supervised learning.
Public entry points#
The named family lazily exports the model and its supervised integration.
build_rwm_trainer retains its public symbol name at this canonical import
path and at embodichain_models.dynamics.rwm.sl_recipe. The family does not
initialize simulation or import example helpers.
Configure a fixed-control-period robot state increment model. |
|
Predict body-frame increments and integrate a canonical robot state. |
|
|
Freeze parameters and statistics while retaining state/action autograd. |
Fit a complete autoregressive window using normalized state errors. |
|
|
Expose an owned transition window as initial state and future supervision. |
Retain a batched prediction state without native simulation buffers. |
|
Keep graph-free physical transitions in a fixed-size device ring. |
|
|
Build the standalone offline RWM recipe for |
- class embodichain_models.dynamics.rwm.RobotWorldModelCfg[source]#
Configure a fixed-control-period robot state increment model.
- Parameters:
num_joints (
int) – Number of ordered joint position/velocity entries.action_dim (
int) – Width of the explicitly defined control vector.context_dim (
int) – Width of recorded physical context, such as friction.hidden_dims (
tuple[int,...]) – Hidden MLP widths.step_dt (
float) – Fixed control period represented by each prediction, seconds.joint_names (
tuple[str,...]) – Optional ordered identifiers included in checkpoint identity.action_semantics (
str) – Meaning of controls, retained in checkpoint identity.amp_bfloat16 (
bool) – Use BF16 network computation on CUDA; state math is FP32.normalization_epsilon (
float) – Input standard-deviation stabilizer.
Methods:
- __init__(num_joints=<factory>, action_dim=<factory>, context_dim=<factory>, hidden_dims=<factory>, step_dt=<factory>, joint_names=<factory>, action_semantics=<factory>, amp_bfloat16=<factory>, normalization_epsilon=<factory>)#
- __new__(**kwargs)#
- class embodichain_models.dynamics.rwm.RobotWorldModel[source]#
Predict body-frame increments and integrate a canonical robot state.
- Parameters:
cfg (
RobotWorldModelCfg|None) – Model dimensions and physical/control identity. Canonical states have width18 + 2*num_joints; seeembodichain_models.dynamics.rwm.robot_state.
Statistics update only through
update_normalization(), never duringforward(). The model starts as an identity transition because its final layer is zero-initialized. This does not imply a physics prior.Methods:
__init__([cfg])Initialize internal Module state, shared by both nn.Module and ScriptModule.
__new__(**kwargs)- __init__(cfg=None)[source]#
Initialize internal Module state, shared by both nn.Module and ScriptModule.
- __new__(**kwargs)#
- embodichain_models.dynamics.rwm.frozen_world_model(model)[source]#
Freeze parameters and statistics while retaining state/action autograd.
- Parameters:
model (
RobotWorldModel) – Model to use for a dependent rollout and backward pass.- Yields:
None. Complete dependent backward work before leaving the context and before changing model parameters; this context does not use no_grad.
- Return type:
Iterator[None]
- class embodichain_models.dynamics.rwm.RobotWorldModelObjective[source]#
Fit a complete autoregressive window using normalized state errors.
- Parameters:
error_floor (
float) – Minimum physical increment scale for near-constant fields.
Loss is the mean squared normalized error over valid transitions and state increment features. Invalid suffixes are excluded before physical-error arithmetic and may contain nonfinite padding. This supervised objective does not create rewards, continuation probabilities or policy gradients. The last-step diagnostic weights each window by its valid transition count.
Methods:
- __new__(**kwargs)#
- embodichain_models.dynamics.rwm.replay_supervised_batch(windows, *, source_id='transition-replay')[source]#
Expose an owned transition window as initial state and future supervision.
- Parameters:
windows (
Mapping[str,Tensor]) – Output ofTransitionReplay.sample_windows.state,action,next_state,contextandvalidare required.source_id (
str) – Stable identity supplied by the replay’s owning workflow.
- Return type:
- Returns:
Batch with the measured starting state and actions as inputs. Future measured states enter targets only; their values never teacher-force the autoregressive rollout.
- class embodichain_models.dynamics.rwm.RWMPredictor[source]#
Retain a batched prediction state without native simulation buffers.
- Parameters:
model (
RobotWorldModel) – Caller-owned world model.initial_state (
Tensor) – Initial canonical FP32 state batch.
Calls allocate new states, preserving outstanding results and unselected environment graphs. A caller performing policy optimization should use
frozen_world_modelaround the rollout and dependent backward work.Methods:
- __new__(**kwargs)#
- class embodichain_models.dynamics.rwm.TransitionReplay[source]#
Keep graph-free physical transitions in a fixed-size device ring.
- Parameters:
capacity (
int) – Maximum vector steps retained, per environment.num_envs (
int) – Number of transitions supplied on each append.state_dim (
int) – Flat state width, independently of a particular robot model.action_dim (
int) – Flat control width.context_dim (
int) – Optional recorded physical-context width.device (
str|device) – Storage and sampling device.max_bytes (
int|None) – Optional allocation ceiling, checked before allocating.seed (
int) – Replay-owned sampling RNG seed.
Float payloads are FP32. Terminal and truncated transitions are retained; their following episode is excluded from a sampled window using a validity mask and repeated final valid transition. Caller-owned simulator views are never retained, and previously returned samples survive ring replacement.
Methods:
- __init__(capacity, num_envs, state_dim, action_dim, *, context_dim=0, device='cpu', max_bytes=None, seed=0)[source]#
- __new__(**kwargs)#
- embodichain_models.dynamics.rwm.build_rwm_trainer(config)[source]#
Build the standalone offline RWM recipe for
train-sl.- Parameters:
config (
Mapping[str,Any]) – Standardmodel/data/objective/optimizer/trainer/evaluationmapping.data.pathis a tensor-only PyTorch file containinginputs,targets,masksandmetadata. Metadata must declaresplit,action_semanticsandstep_dt. Named model joints require the same orderedjoint_namesin metadata. Optionaldata.validation_pathmust identify a validation split. Inputs followRobotWorldModelObjective.compute_loss().- Return type:
- Returns:
Shared supervised trainer. Initial normalization uses only valid measured transitions in the training corpus. This factory does not collect data, initialize simulation or construct a policy learner.
State representation and integration#
Canonical states contain world position, a row-major body-to-world rotation matrix, world linear/angular velocity and ordered joint positions/velocities. Increments use the current body frame and right-multiplied exponential rotation coordinates. The logarithm selects the shortest rotation; first-order derivatives away from its pi branch cut are supported.
|
Map rotation vectors to matrices with finite derivatives at zero. |
|
Map proper rotation matrices to shortest rotation vectors. |
|
Encode height, body-frame gravity and velocities, and joint state. |
|
Encode the transition to |
|
Apply frame-local increments while preserving proper rotations. |
|
Measure prediction error in the target state's body frame. |
- embodichain_models.dynamics.rwm.robot_state.rotation_exp(vector)[source]#
Map rotation vectors to matrices with finite derivatives at zero.
- Parameters:
vector (
Tensor) – Floating tensor with final dimension three, in radians.- Return type:
Tensor- Returns:
Rotation matrices with shape
vector.shape[:-1] + (3, 3).
- embodichain_models.dynamics.rwm.robot_state.rotation_log(matrix)[source]#
Map proper rotation matrices to shortest rotation vectors.
- Parameters:
matrix (
Tensor) – Floating tensor ending in(3, 3). Inputs must be proper rotations; this function does not project arbitrary matrices.- Return type:
Tensor- Returns:
Rotation vectors. At the pi branch the selected axis is deterministic but its derivative is not a supported contract.
- embodichain_models.dynamics.rwm.robot_state.robot_state_features(state)[source]#
Encode height, body-frame gravity and velocities, and joint state.
- Parameters:
state (
Tensor) – Robot states in the module’s matrix-based layout. The world frame is Z-up and the gravity direction is(0, 0, -1).- Return type:
Tensor- Returns:
Features of width
10 + 2 * num_joints. Global XY position and yaw do not enter these features.
- embodichain_models.dynamics.rwm.robot_state.robot_state_increment(state, next_state)[source]#
Encode the transition to
next_statein the current body frame.- Parameters:
state (
Tensor) – Current states in the module’s canonical layout.next_state (
Tensor) – Aligned next states with exactly the same shape.
- Return type:
Tensor- Returns:
Increments of width
12 + 2 * num_joints.
- embodichain_models.dynamics.rwm.robot_state.integrate_robot_state(state, increment)[source]#
Apply frame-local increments while preserving proper rotations.
- Parameters:
state (
Tensor) – Current states in the module’s canonical layout.increment (
Tensor) – Aligned increments with final widthstate.shape[-1]-6.
- Return type:
Tensor- Returns:
Newly allocated next states with gradients to both arguments.
- embodichain_models.dynamics.rwm.robot_state.robot_state_error(predicted, target)[source]#
Measure prediction error in the target state’s body frame.
- Parameters:
predicted (
Tensor) – Predicted canonical robot states.target (
Tensor) – Aligned target states.
- Return type:
Tensor- Returns:
Position, angular, velocity and joint errors in increment layout.
Model and normalization#
The network predicts normalized state increments. Input/target statistics change
only through an explicit update from measured training transitions. Checkpoint
identity retains the fixed control period, action semantics and ordered joints.
frozen_world_model freezes parameters and statistics while preserving
state/action gradients through dependent rollout and backward work.
Configure a fixed-control-period robot state increment model. |
|
Predict body-frame increments and integrate a canonical robot state. |
|
|
Freeze parameters and statistics while retaining state/action autograd. |
- class embodichain_models.dynamics.rwm.model.RobotWorldModelCfg[source]#
Configure a fixed-control-period robot state increment model.
- Parameters:
num_joints (
int) – Number of ordered joint position/velocity entries.action_dim (
int) – Width of the explicitly defined control vector.context_dim (
int) – Width of recorded physical context, such as friction.hidden_dims (
tuple[int,...]) – Hidden MLP widths.step_dt (
float) – Fixed control period represented by each prediction, seconds.joint_names (
tuple[str,...]) – Optional ordered identifiers included in checkpoint identity.action_semantics (
str) – Meaning of controls, retained in checkpoint identity.amp_bfloat16 (
bool) – Use BF16 network computation on CUDA; state math is FP32.normalization_epsilon (
float) – Input standard-deviation stabilizer.
Methods:
__init__([num_joints, action_dim, ...])copy(**kwargs)Return a new object replacing specified fields with new values.
replace(**kwargs)Return a new object replacing specified fields with new values.
to_dict()Convert an object into dictionary recursively.
validate([prefix])Check the validity of configclass object.
- __init__(num_joints=<factory>, action_dim=<factory>, context_dim=<factory>, hidden_dims=<factory>, step_dt=<factory>, joint_names=<factory>, action_semantics=<factory>, amp_bfloat16=<factory>, normalization_epsilon=<factory>)#
- copy(**kwargs)#
Return a new object replacing specified fields with new values.
This is especially useful for frozen classes. Example usage:
@configclass(frozen=True) class C: x: int y: int c = C(1, 2) c1 = c.replace(x=3) assert c1.x == 3 and c1.y == 2
- Parameters:
obj (
object) – The object to replace.**kwargs – The fields to replace and their new values.
- Return type:
object- Returns:
The new object.
- replace(**kwargs)#
Return a new object replacing specified fields with new values.
This is especially useful for frozen classes. Example usage:
@configclass(frozen=True) class C: x: int y: int c = C(1, 2) c1 = c.replace(x=3) assert c1.x == 3 and c1.y == 2
- Parameters:
obj (
object) – The object to replace.**kwargs – The fields to replace and their new values.
- Return type:
object- Returns:
The new object.
- to_dict()#
Convert an object into dictionary recursively.
Note
Ignores all names starting with “__” (i.e. built-in methods).
- Parameters:
obj (
object) – An instance of a class to convert.- Raises:
ValueError – When input argument is not an object.
- Return type:
dict[str,Any]- Returns:
Converted dictionary mapping.
- validate(prefix='')#
Check the validity of configclass object.
This function checks if the object is a valid configclass object. A valid configclass object contains no MISSING entries.
- Parameters:
obj (
object) – The object to check.prefix (
str) – The prefix to add to the missing fields. Defaults to ‘’.
- Return type:
list[str]- Returns:
A list of missing fields.
- Raises:
TypeError – When the object is not a valid configuration object.
- class embodichain_models.dynamics.rwm.model.RobotWorldModel[source]#
Predict body-frame increments and integrate a canonical robot state.
- Parameters:
cfg (
RobotWorldModelCfg|None) – Model dimensions and physical/control identity. Canonical states have width18 + 2*num_joints; seeembodichain_models.dynamics.rwm.robot_state.
Statistics update only through
update_normalization(), never duringforward(). The model starts as an identity transition because its final layer is zero-initialized. This does not imply a physics prior.Methods:
__init__([cfg])Initialize internal Module state, shared by both nn.Module and ScriptModule.
forward(state, action[, context])Advance one fixed-period transition without detaching inputs.
Return model layout and action identity saved beside tensor weights.
set_extra_state(state)Reject weights belonging to a different state or action contract.
update_normalization(state, action, next_state)Merge an explicitly selected batch of real training transitions.
- __init__(cfg=None)[source]#
Initialize internal Module state, shared by both nn.Module and ScriptModule.
- forward(state, action, context=None)[source]#
Advance one fixed-period transition without detaching inputs.
- Parameters:
state (
Tensor) – Canonical FP32 robot states.action (
Tensor) – Aligned FP32 controls with configured semantics.context (
Tensor|None) – Aligned physical context, required when its width is nonzero.
- Return type:
Tensor- Returns:
FP32 next states retaining state/action gradients even when every model parameter has
requires_grad=False.
- get_extra_state()[source]#
Return model layout and action identity saved beside tensor weights.
- Return type:
dict[str,Any]- Returns:
Versioned configuration including state, joint and action semantics.
- set_extra_state(state)[source]#
Reject weights belonging to a different state or action contract.
- Parameters:
state (
dict[str,Any]) – Identity returned byget_extra_state().- Return type:
None
- update_normalization(state, action, next_state, context=None)[source]#
Merge an explicitly selected batch of real training transitions.
- Parameters:
state (
Tensor) – Current states from the training distribution.action (
Tensor) – Controls that producednext_state.next_state (
Tensor) – Physical next states captured before reset.context (
Tensor|None) – Recorded physical context.
- Return type:
None
- embodichain_models.dynamics.rwm.model.frozen_world_model(model)[source]#
Freeze parameters and statistics while retaining state/action autograd.
- Parameters:
model (
RobotWorldModel) – Model to use for a dependent rollout and backward pass.- Yields:
None. Complete dependent backward work before leaving the context and before changing model parameters; this context does not use no_grad.
- Return type:
Iterator[None]
Supervision and prediction lifecycle#
The objective builds an untruncated autoregressive graph. Invalid prefix-mask suffixes are excluded before differentiable arithmetic, including nonfinite padding. The predictor returns newly allocated states; selective reset preserves unselected rows and outstanding graphs. Callers explicitly detach prediction state when beginning a new graph.
Fit a complete autoregressive window using normalized state errors. |
|
|
Expose an owned transition window as initial state and future supervision. |
- class embodichain_models.dynamics.rwm.supervised.RobotWorldModelObjective[source]#
Fit a complete autoregressive window using normalized state errors.
- Parameters:
error_floor (
float) – Minimum physical increment scale for near-constant fields.
Loss is the mean squared normalized error over valid transitions and state increment features. Invalid suffixes are excluded before physical-error arithmetic and may contain nonfinite padding. This supervised objective does not create rewards, continuation probabilities or policy gradients. The last-step diagnostic weights each window by its valid transition count.
Methods:
__init__([error_floor])__new__(**kwargs)compute_loss(model, batch)Build an untruncated multi-step prediction graph.
- __new__(**kwargs)#
- compute_loss(model, batch)[source]#
Build an untruncated multi-step prediction graph.
- Parameters:
model (
RobotWorldModel) – Configured robot world model.batch (
SupervisedBatch) –initial_state[B,D],actions[B,T,A]and optionalcontexts[B,T,C]inputs; canonicaltargets[B,T,D]and a Boolean prefix maskvalid[B,T].
- Return type:
- Returns:
Connected normalized loss and detached aggregation-ready diagnostics.
- embodichain_models.dynamics.rwm.supervised.replay_supervised_batch(windows, *, source_id='transition-replay')[source]#
Expose an owned transition window as initial state and future supervision.
- Parameters:
windows (
Mapping[str,Tensor]) – Output ofTransitionReplay.sample_windows.state,action,next_state,contextandvalidare required.source_id (
str) – Stable identity supplied by the replay’s owning workflow.
- Return type:
- Returns:
Batch with the measured starting state and actions as inputs. Future measured states enter targets only; their values never teacher-force the autoregressive rollout.
Retain a batched prediction state without native simulation buffers. |
- class embodichain_models.dynamics.rwm.predictor.RWMPredictor[source]#
Retain a batched prediction state without native simulation buffers.
- Parameters:
model (
RobotWorldModel) – Caller-owned world model.initial_state (
Tensor) – Initial canonical FP32 state batch.
Calls allocate new states, preserving outstanding results and unselected environment graphs. A caller performing policy optimization should use
frozen_world_modelaround the rollout and dependent backward work.Methods:
__init__(model, initial_state)Cut the current prediction graph while preserving every state value.
load_state_dict(state)Restore prediction state under the same physical/action contract.
reset(initial_state[, env_ids])Replace all or selected states without overwriting old graph tensors.
Return an owned graph-free prediction-state snapshot.
step(action[, context])Advance one transition and retain the returned graph-connected state.
Attributes:
Current graph-connected state; callers must not mutate its storage.
- detach_state()[source]#
Cut the current prediction graph while preserving every state value.
- Return type:
Tensor- Returns:
The detached state. No reset, resampling or statistics update occurs.
- load_state_dict(state)[source]#
Restore prediction state under the same physical/action contract.
- Parameters:
state (
Mapping[str,Any]) – Snapshot fromstate_dict().- Return type:
None
- reset(initial_state, env_ids=None)[source]#
Replace all or selected states without overwriting old graph tensors.
- Parameters:
initial_state (
Tensor) – Full replacement batch, or one state per selected id.env_ids (
Tensor|None) – Optional unique one-dimensional environment indices.
- Return type:
Tensor- Returns:
Current state after replacement. Unselected rows retain their graph.
- property state: Tensor#
Current graph-connected state; callers must not mutate its storage.
Owned transition replay#
Append copies detached physical transitions before environment reset. Sampled windows and oldest-first histories retain episode boundaries and survive ring overwrite. The fixed allocation can be checked against a byte ceiling. Replay snapshots retain their sampling RNG; acquisition and refill remain caller-owned.
Keep graph-free physical transitions in a fixed-size device ring. |
- class embodichain_models.dynamics.rwm.replay.TransitionReplay[source]#
Keep graph-free physical transitions in a fixed-size device ring.
- Parameters:
capacity (
int) – Maximum vector steps retained, per environment.num_envs (
int) – Number of transitions supplied on each append.state_dim (
int) – Flat state width, independently of a particular robot model.action_dim (
int) – Flat control width.context_dim (
int) – Optional recorded physical-context width.device (
str|device) – Storage and sampling device.max_bytes (
int|None) – Optional allocation ceiling, checked before allocating.seed (
int) – Replay-owned sampling RNG seed.
Float payloads are FP32. Terminal and truncated transitions are retained; their following episode is excluded from a sampled window using a validity mask and repeated final valid transition. Caller-owned simulator views are never retained, and previously returned samples survive ring replacement.
Methods:
__init__(capacity, num_envs, state_dim, ...)append(state, action, next_state, *[, ...])Copy one vector step; flags end episodes after the stored transition.
load_state_dict(state)Restore a matching replay after validating the complete layout.
sample_starts(batch_size, history_length, *)Gather imagination starts and oldest-first boundary-padded history.
sample_windows(batch_size, horizon, *[, ...])Gather horizon transitions without carrying data across episodes.
Snapshot owned replay content, boundaries and the sampling RNG.
- __init__(capacity, num_envs, state_dim, action_dim, *, context_dim=0, device='cpu', max_bytes=None, seed=0)[source]#
- append(state, action, next_state, *, context=None, terminated=None, truncated=None)[source]#
Copy one vector step; flags end episodes after the stored transition.
- Parameters:
state (
Tensor) – Current FP32 states,[num_envs, state_dim].action (
Tensor) – Executed FP32 controls.next_state (
Tensor) – Physical next states captured before any reset.context (
Tensor|None) – Recorded FP32 physical context; required for nonzero width.terminated (
Tensor|None) – Hard terminal flags, default false.truncated (
Tensor|None) – Time-limit flags, default false, retained separately.
- Return type:
None
- load_state_dict(state)[source]#
Restore a matching replay after validating the complete layout.
- Parameters:
state (
Mapping[str,Any]) – Snapshot produced bystate_dict(). Payloads may reside on another device and are copied to this replay’s device.- Return type:
None
- sample_starts(batch_size, history_length, *, generator=None)[source]#
Gather imagination starts and oldest-first boundary-padded history.
- Parameters:
batch_size (
int) – Number of starts.history_length (
int) – Number of states including the selected current one.generator (
Generator|None) – Optional sampling RNG on the replay device.
- Return type:
dict[str,Any]- Returns:
startfields,historyfields and Booleanhistory_valid. Missing history repeats the earliest available state in that same episode. Requests never retry or wait for more data.
- sample_windows(batch_size, horizon, *, generator=None)[source]#
Gather horizon transitions without carrying data across episodes.
- Parameters:
batch_size (
int) – Number of sampled environment/window pairs.horizon (
int) – Requested transition count, no greater than retained steps.generator (
Generator|None) – Optional sampling RNG on the replay device.
- Return type:
dict[str,Tensor]- Returns:
Owned field tensors with
[batch, horizon]leading axes and a Booleanvalidmask. The terminal transition itself is valid; positions after it repeat its data withvalid=False.
- state_dict()[source]#
Snapshot owned replay content, boundaries and the sampling RNG.
- Return type:
dict[str,Any]- Returns:
Independent tensors suitable for optional exact replay restoration. Full snapshots cost another replay-sized allocation; routine RWM recovery may instead rebuild and refill replay explicitly.
Offline SL recipe#
Use embodichain_models.dynamics.rwm.sl_recipe:build_rwm_trainer as the factory
for embodichain train-sl. Tensor corpora declare their split, action semantics,
control period and ordered joints. Normalization uses valid measured training
transitions. The resulting common SL checkpoint preserves corpus identity,
normalization, optimizer, RNG and the sampling cursor. The controlled fitting
example runs through embodichain run-example learning/rwm.
|
Build the standalone offline RWM recipe for |
- embodichain_models.dynamics.rwm.sl_recipe.build_rwm_trainer(config)[source]#
Build the standalone offline RWM recipe for
train-sl.- Parameters:
config (
Mapping[str,Any]) – Standardmodel/data/objective/optimizer/trainer/evaluationmapping.data.pathis a tensor-only PyTorch file containinginputs,targets,masksandmetadata. Metadata must declaresplit,action_semanticsandstep_dt. Named model joints require the same orderedjoint_namesin metadata. Optionaldata.validation_pathmust identify a validation split. Inputs followRobotWorldModelObjective.compute_loss().- Return type:
- Returns:
Shared supervised trainer. Initial normalization uses only valid measured transitions in the training corpus. This factory does not collect data, initialize simulation or construct a policy learner.