Source-linked AI summary
Graph-Operator World Models for Morphology-Parameter Generalization in Continuous Control
Xu Yang, Yiqin Yang, Qianchuan Zhao
TL;DR
World models often leave unclear which transition components should remain reusable when known morphology parameters change. GraphOp-WM separates a shared local dynamics basis from a morphology-conditioned graph operator and evaluates this design on controlled MuJoCo splits for unseen parameter changes within related robot families.
Problem
The paper asks whether a compact world model can generalize to unseen physical morphology parameters by separating reusable local dynamics from morphology-induced coupling.
Method
GraphOp-WM represents articulated robots as attributed graphs and factorizes transitions into a morphology-independent local basis and a structured morphology-conditioned operator.
Results
The paper defines controlled MuJoCo interpolation, extrapolation, and held-out-composition splits across Hopper, Walker2d, and HalfCheetah for out-of-distribution prediction and planning.
Takeaways & Limitations
GraphOp-WM is intended as a compact structural prior for parameter generalization within related articulated robot families.
Takeaways & Limitations
Universal transfer to arbitrary robot families remains outside the formulation’s scope.
Abstract
from arXiv · showhide
World models for continuous control are commonly trained for a fixed physical system and can degrade when known morphology parameters such as link lengths, masses, damping, and actuation change. Existing approaches often provide these parameters as conditioning information, but leave unspecified which part of the learned transition should remain reusable and which part should change with morphology. We propose Graph-Operator World Models (GraphOp-WM), a structured world model for generalization across unseen morphology parameters within related articulated robot families. GraphOp-WM represents bodies and their kinematic relations as an attributed graph and factorizes each transition into a morphology-independent local dynamics basis and a morphology-conditioned structured operator. The operator combines node-local modulation, kinematic-tree coupling, and a low-rank global correction, while architectural information separation, basis normalization, and paired-morphology supervision encourage static morphology dependence to be carried by the operator pathway. Graph-level readout and edge-wise action representations provide a compatible interface for reward, value, and TD-MPC-style planning. We further define controlled MuJoCo parameter splits covering interpolation, extrapolation, and held-out compositions of link geometry, mass, damping, and actuation parameters in Hopper, Walker2d, and HalfCheetah.
1 Introduction
The paper frames morphology-parameter generalization as reusing local dynamics while adapting morphology-induced coupling in articulated robots. GraphOp-WM implements this separation with a structured graph operator and evaluates it on controlled unseen-parameter splits.
- Motivation: Morphology changes alter transition dynamics while preserving reusable local structure, which generic conditioning does not specify how to transform.The motivating parameters include link geometry, mass, inertia, damping, and actuator gear.
- Approach: GraphOp-WM represents bodies as nodes and joints and actuators as edges in a graph-structured world model.Its graph encoder converts morphology into a structured transition operator over local dynamic responses.
- Approach: The transition factorization separates a morphology-independent local dynamics basis from a morphology-conditioned operator combining local, graph-supported, and low-rank global coupling.The operator captures body-local response, kinematic-tree coupling, and residual global modes such as trunk–limb and contact-mediated coupling.
- Approach: Architectural information separation, basis normalization, and paired-morphology supervision assign morphology-dependent transition changes to the operator pathway.The shared basis is prevented from observing static morphology information.
- Evaluation: The benchmark measures interpolation, extrapolation, and held-out composition generalization across geometry, mass, damping, and actuation changes.The experiments use controlled MuJoCo splits for out-of-distribution prediction and planning.
2 Related Work
Related work spans general world models, contextual and morphology-conditioned models, morphology-aware control, and graph-based dynamics. GraphOp-WM targets known physical morphology changes by treating morphology as a structured operator over local responses.
- World models: World models support planning and imagined policy optimization, while prior generality relies mainly on capacity, task embeddings, and multitask data.GraphOp-WM retains a control-centric modeling and planning interface while adding explicit dynamics factorization.
- Contextual and morphology-conditioned models: Contextual world models infer latent task or dynamics information from interaction, whereas robot morphology is available before interaction from physical specifications.This distinguishes known embodiment parameters from properties that must be inferred online.
- Contextual and morphology-conditioned models: GraphOp-WM treats known morphology changes as a structured operator rather than only a pooled vector, token embedding, or expert-routing signal.The stated target is extrapolation to unseen parameter values and combinations within related articulated systems.
- Morphology-conditioned control: Morphology-conditioned control methods share computation across body parts or condition controllers on embodiment structure.Examples include graph policies, modular actuator policies, Transformer conditioning, restricted attention, and morphology-agnostic encoders.
- Graph dynamics and learned operators: Graph dynamics and learned operators provide object- and relation-centric inductive biases for physical prediction, including graph transitions and blockstructured linear dynamics.These works motivate structured representations for compositional physical systems.
3 Problem Setting
The problem considers control tasks indexed by morphology, with attributed graphs encoding body, joint, and actuator structure. The primary goal is zero-shot prediction and planning on unseen variants within related articulated families.
- Problem formulation: Each morphology is represented as an attributed graph whose body nodes contain geometry and inertial metadata.Joint edges encode kinematic relations, while actuator edges associate commands with controlled joints.
- Generalization objective: The training set contains trajectories from training morphologies, and evaluation targets unseen variants in related articulated families.The objective is zero-shot dynamics prediction and planning on held-out morphologies.
- Generalization objective: The benchmark separates interpolation, parameter extrapolation, held-out parameter combinations, and within-family topology extension.These splits define distinct forms of out-of-distribution generalization.
- Scope: Universal transfer to arbitrary robot families remains outside the formulation’s scope.The setup assumes fixed topology and body, joint, and actuator correspondence within each robot family.
4 Method
GraphOp-WM parses each articulated robot into a packed attributed graph and factorizes transitions into a shared local dynamics basis and morphology-conditioned structured operator. Information separation, normalization, paired-morphology supervision, and graph-based prediction heads preserve reusable dynamics while assigning morphology-dependent effects to structured coupling.
- Graph representation: Each robot is represented as a body–joint graph with dynamic features on bodies and joint position, velocity, and actuator commands on edges.Within a family, topology and body–joint–actuator correspondence remain fixed, while morphology varies through static node and edge attributes.
- State tokens: A shared body encoder produces dynamic node tokens without observing masses, lengths, inertias, morphology identifiers, or pooled morphology embeddings.This blocks direct access to static morphology through the state pathway.
- Structured operator: The transition factorizes shared local increments with an operator combining body-local response, kinematic-tree coupling, and low-rank global modes.The low-rank term captures residual nonlocal effects and is retained when transition-Jacobian diagnostics show a rapidly decaying residual spectrum.
- Information separation: Static morphology enters only the operator pathway, while basis normalization removes reciprocal scale ambiguity and paired-morphology supervision assigns transition differences to operators.State-dependent gates activate static structure according to pose, velocity, and contact state rather than trainable morphology identifiers.
- Prediction and control interfaces: Graph-level readouts support reward and value prediction, while actuator-edge representations support variable-action policy heads and action-conditioned value computation.Task embeddings enter reward, value, and policy heads but not the transition basis or morphology operator; head dimensions remain fixed across body and actuator counts.
- Training objective: Training combines dynamics, reward, TD, paired-morphology, and operator losses after graph packing, basis prediction, operator construction, and multi-step rollout.The complete objective is L = Ldyn + λrLr + λQLQ + λV LV + λpairLpair + λopLop.
5 Model-Based Control with Variable Graphs
GraphOp-WM adapts TD-MPC-style planning to morphology-dependent graph states and variable action tensors. It preserves the planner’s MPPI/CEM optimization logic and temporal-difference targets while replacing vector-based model components with packed graph modules.
- Variable-graph planning: TD-MPC-style planning rolls out candidate trajectories using the current morphology’s known node and actuator counts.Different morphologies use separate action tensors at separate planning calls, so no global maximum action dimension is required.
- TD-MPC integration: The adaptation replaces vector encoders, MLP transitions, fixed reward and value inputs, and fixed policy outputs with packed graph modules and variable-action heads.MPPI/CEM optimization logic and temporal-difference targets remain unchanged.
6 MuJoCo Benchmark Definition
The benchmark evaluates morphology-parameter generalization in Hopper, Walker2d, and HalfCheetah using deterministic MuJoCo edits. It isolates interpolation, extrapolation, and held-out compositional generalization while keeping core environment conditions fixed within each family.
- Robot families: The benchmark uses Hopper, Walker2d, and HalfCheetah, converting each canonical MJCF model into the attributed graph representation.Variants are generated by deterministic MJCF edits.
- Parameterization: Variants scale link geometry, body mass, torso mass, joint damping, and actuator gear relative to canonical models.Inertia is recomputed from scaled geometry and mass, while friction, gravity, timestep, termination rules, and rewards remain fixed within each family.
- Evaluation splits: Interpolation uses unseen values and combinations inside the training envelope, extrapolation uses values outside it, and the compositional split combines observed directions into jointly unseen morphologies.Training variants are generated with maximin Latin-hypercube sampling over training ranges.
7 Conclusion
GraphOp-WM is a structured world model that separates reusable local dynamics from morphology-conditioned operator effects in articulated robots. Its benchmark targets out-of-distribution prediction and planning across controlled parameter changes, within related robot families rather than across arbitrary topologies.
- Conclusion: GraphOp-WM represents articulated robots as graphs and separates a morphology-independent local dynamics basis from a morphology-conditioned structured operator.Information separation, basis normalization, and paired-morphology supervision give the two transition components distinct roles.
- Conclusion: The benchmark defines interpolation, extrapolation, and held-out composition splits for geometry, mass, damping, and actuation changes.These splits study out-of-distribution prediction and planning under controlled morphology variation.
- Scope: GraphOp-WM is intended as a compact structural prior for parameter generalization within related articulated robot families rather than a universal cross-topology controller.