AnyWorld: Factorized Egocentric World Models for Cross-Embodiment Generalization

• The materials identify Action, Scene, and Embodiment as named factors.Source: REFERENCE VIDEOS · p. 1
• From a human egocentric interaction, AnyWorld generates robot-native rollouts across embodiments, viewpoints, and scene configurations while preserving interaction dynamics and object interactions.Source: Figure 1 · p. 1
• Fine-tuning VLA models on generated real IRON-domain rollouts significantly improves real-robot manipulation robustness.Source: Figure 1 · p. 1

1 Introduction

• Scaling contact-rich robot experience is difficult, and generalizable manipulation requires repeated task-relevant interactions across bodies, viewpoints, object layouts, and scenes.Physical-robot data collection remains expensive, hardware-dependent, and difficult to scale.Source: §1 · p. 2
• Human egocentric videos offer abundant contact-rich interactions, but each recording is tied to one body, camera trajectory, and environment, making it a narrow experience slice.The paper frames each interaction as a reusable seed for recomposition. Human-video sources cited in the passage include Grauman et al. (2022; 2024) and Hoque et al. (2025).Source: §1 · p. 2
• AnyWorld factorizes interactions into action, camera, and embodiment controls, using the initial visual state to specify the target body, scene, object configuration, and interaction geometry.This enables a single human interaction to be recomposed into robot-native rollouts that preserve its underlying physical interaction while varying embodiment, viewpoint, or environment.Source: §1 · p. 2
• The model is pretrained on large-scale human interactions, fine-tuned across embodiments, and generates rollouts from an initial visual state plus action, camera, and embodiment conditions.It supports body, viewpoint, and first-frame-based scene recomposition, including combinations, to provide synthetic experience for downstream VLA training.Source: §1 · p. 2
• AnyWorld’s rollouts support controllable recomposition across embodiments, viewpoints, and scenes, and improve downstream VLA adaptation in RoboCasa GR1 and real-robot IRON experiments.The evaluation measures whether rollouts follow specified action, camera, and embodiment conditions and assesses video quality with standard generation metrics.Source: §1 · p. 3
• Controlled IRON interventions use recomposed experience to correct a spurious completion prior and transfer language-grounded spatial target selection; action-only intervention does not reliably learn the latter.The paper presents visual recomposition alongside action calibration as necessary for this transfer.Source: §1 · p. 3

Robotic world models

• Prior robotic world models predict action-conditioned visual or latent futures and use imagined rollouts, while DreamDojo pretrains on egocentric videos to transfer interaction priors across embodiments (Gao et al., 2026).Related methods include autoregressive, diffusion, and latent-action interfaces (Wu et al., 2024; Zhu et al., 2025a;b; Gao et al., 2025), with imagined rollouts used for policy learning or improvement (Jang et al., 2025; Guo et al., 2026; Zhang et al., 2026c; Chen et al., 2026b).Source: §2 · p. 3
• Unlike prior robotic world models that stay within a fixed embodiment, adapt to a target robot, or improve action prediction, AnyWorld recomposes one unpaired human interaction into many robot-domain experiences.Recomposition changes action, camera, embodiment, and initial visual context.Source: §2 · p. 3

Human egocentric videos and cross-embodiment transfer

• Prior work uses human egocentric videos for representations, rewards, visual-gap reduction, latent actions, physical tokens, VLA pretraining, and dexterous cross-embodiment transfer (Nair et al., 2023; Ma et al., 2023; Kareer et al., 2025; Ye et al., 2025; Zhang et al., 2026a; Dai et al., 2026; Chen et al., 2026a; Li et al., 2025; Hoque et al., 2025; Goswami et al., 2025; Xu et al., 2023; Zhang et al., 2026b).Complementary VLA research addresses deployment efficiency and spatiotemporal reasoning in compact policies (Ni et al., 2026).Source: §2 · p. 3
• Unlike methods that use human experience for representation learning, supervision, retargeting, or policy training, AnyWorld renders robot-native videos from unpaired interactions by factorizing trajectory, viewpoint, embodiment, and initial visual context.This enables recomposition across target bodies, cameras, and scene configurations.Source: §2 · p. 3

3.1 Problem Formulation

• The task is zero-shot conversion of an egocentric human video into a target-robot visual rollout using extracted action and camera controls plus a target embodiment/scene initial frame, without paired clips.Source: §3.1 · pp. 3–4

3.2 Action-Camera-Embodiment Factorization

• Rendered skeleton controls specify image-plane motion, Plücker-ray embeddings encode camera geometry, and an initial frame plus text tag specify embodiment and scene context.Source: §3.2 · p. 4

3.3 Cross-Embodiment World Model

• AnyWorld implements Gθ as latent diffusion, feeding action controls as input channels, camera geometry through an additive adapter, and embodiment/caption text through transformer blocks.Source: §3.3 · p. 4

3.4 Training

• Training first pretrains on EgoDex egocentric human interactions, then fine-tunes on unpaired multi-embodiment videos converted into shared action, camera, and embodiment conditions without clip-level human–robot correspondence.Source: §3.4 · p. 5

3.5 Zero-Shot Experience Recomposition

• At inference, editing the initial frame and embodiment tag can render the same human action-camera trajectory with a robot body, new scene, or both; changing camera controls changes viewpoint.Source: §3.5 · p. 5

3.6 Joint Visual–Action Experience Recomposition

• Downstream policy data transfers both re-embodied, optionally re-scened observations and morphology-calibrated robot actions: actions specify execution, while visuals provide the robot-native state for grounding.Source: §3.6 · p. 5

4.1 Experimental Setup

• Training uses 200K EgoDex clips for pretraining and unpaired human and robot clips for mixed-embodiment fine-tuning; controllability is assessed with ActionAlign, CameraAlign, and EmbodAcc.Pretraining runs for 30K steps and fine-tuning for 5K steps; no paired human–robot videos are used.Source: §4.1 · p. 6

4.2 Qualitative Robot Experience Recomposition

• With action controls fixed, AnyWorld recomposes the same interaction across robot bodies, viewpoints, and scenes while preserving its motion or physical interaction.Examples include human-to-RoboCasa GR1 and RoboCasa GR1-to-human recomposition, as well as changing viewpoint while keeping the action, body, and scene fixed.Source: §4.2 · pp. 6–7, Figure 3 · p. 6, Figure 4 · p. 7

4.3 Controllability and Video Quality

• AnyWorld achieves the best action, camera, and embodiment controllability, while remaining competitive with and slightly improving over baselines on average VBench video quality.The controllability evaluation covers 60 videos across IRON, RoboCasa GR1, and EgoDex; video quality includes subject consistency, background consistency, flickering, and smoothness.Source: §4.3 · p. 7, Table 1 · p. 8, Table 2 · p. 8

4.4 Training Ablations

• The 2:1 human-to-robot fine-tuning mixture achieves the best average controllability, balancing human interaction diversity with robot embodiment grounding.The 4:1 mixture has lower CameraAlign and EmbodAcc, while 1:1 improves CameraAlign but has lower EmbodAcc than 2:1.Source: §4.4 · p. 8, Table 3 · p. 9

4.5 Transferred Experience for VLA Adaptation

• 54.6% vs 49.8% on 18 RoboCasa GR1 pick-and-place tasks and 55.0% vs 20.0% on 20 IRON banana-grasp trials show gains from transferred EgoDex rollouts.The comparison is against UniT; rollouts are added during target-embodiment adaptation.Source: §4.5 · p. 8, Table 4 · p. 9

4.6 Targeted Capability Transfer through Controlled Recomposition

• Re-scened human observations with calibrated IRON actions repair a spurious completion prior; reliable spatial target switching requires jointly recomposed robot-native visuals and actions, not action-only counterfactuals.The state-conditional intervention restores task continuation, while the language-grounding intervention pairs a matched dual-target observation with opposite spatial instructions and corresponding actions.Source: §4.6 · p. 9, Figure 6 · p. 10

5 Conclusion

• Generated robot-domain experience improves UniT-based VLA adaptation in RoboCasa GR1 simulation and on a real IRON robot.Controlled capability interventions also repair a spurious completion prior and transfer language-grounded spatial target selection.Source: §5 · p. 10
• AnyWorld improves action-camera-embodiment controllability over strong video/world-model baselines while maintaining competitive video quality.The model supports recomposition across bodies, viewpoints, and scenes while preserving interaction structure.Source: §5 · p. 10
• AnyWorld decomposes interactions into action, camera, and embodiment, combining large-scale EgoDex pretraining with unpaired mixed-embodiment fine-tuning to generate robot-domain rollouts.This recomposition requires no paired human-robot demonstrations.Source: §5 · p. 10
• Action-only counterfactual pairing fails to transfer language-grounded spatial target selection reliably, indicating that robot-native visual recomposition is required to ground behavior.The controlled interventions also repair a spurious completion prior.Source: §5 · p. 10

6 Limitations

• Visual rollouts cannot fully capture tactile feedback or contact forces needed for precise contact-rich control.The paper proposes tactile supervision as a future direction.Source: §6 · p. 10
• Severe occlusion, fast motion, tracking errors, or strong camera shake can reduce controllability by undermining action-camera extraction.The method depends on reliable action-camera extraction.Source: §6 · p. 10
• Evaluation covers 3 embodiments, leaving generalization to more robot morphologies, objects, and long-horizon tasks for future work.Future directions include uncertainty-aware filtering and larger mixed-embodiment datasets.Source: §6 · p. 10

• The supplement describes reverse pseudo-pair construction for human-to-robot embodiment editing and the mixed-embodiment training recipe with structured conditioning.Source: §1 · p. 14
• It provides implementation details, qualitative results, and quantitative metric definitions for AnyWorld.Source: §1 · p. 14
• It reports policy-adaptation details and additional evaluations of interaction outcomes under motion variations.Source: §1 · p. 14

2 Implementation Details · 2.1 Generalizable Human-to-Robot Embodiment Editing via Reverse Pseudo-Pairs

• The editor replaces visible human body parts with a target robot morphology while preserving the scene, objects, and camera geometry; it supports IRON and RoboCasa GR1 and can extend to embodiments with available images.The edited first frame supplies the target embodiment’s initial visual condition for AnyWorld rollouts.Source: §2.1 · p. 14, §2.1.4 · p. 15

2.1.1 Reverse pseudo-pair construction

• Because paired human–robot images are unavailable, the method edits robot images into human-like observations, then trains the inverse editor using the resulting approximately aligned pseudo-pairs.This direction exploits open-domain editors’ stronger ability to generate plausible human hands and arms than embodiment-specific robot morphology.Source: §2.1 · p. 14, §2.1.1 · p. 15
• For each target embodiment, the pipeline samples approximately 800 robot frames and manually curates the edits, retaining approximately 300 high-quality pairs per embodiment.Curation removes examples with misaligned backgrounds, displaced objects, viewpoint drift, incorrect hand-object contact, inconsistent geometry, or severe artifacts.Source: §2.1.1 · pp. 14–15

2.1.2 Paired offline augmentation

• Synchronous flips, crops, and brightness perturbations augment each pair while keeping source and target aligned; fisheye-camera samples are removed from the final split.Each transformation’s parameters are shared by both images in a pair.Source: §2.1.2 · p. 15

2.1.3 Embodiment editor fine-tuning

• The editor is fine-tuned with supervised flow matching, full-tuning the diffusion transformer, and a separate editor is trained for each target robot.The embodiment-specific model learns each robot’s morphology, material appearance, and spatial placement while retaining the same pseudo-paired formulation.Source: §2.1.3 · p. 15

2.1.4 First-frame editing for world-model generation

• Figure 1 shows edits for IRON and RoboCasa GR1 that preserve hand position, interaction geometry, and scene details, including a plausible robot hand from a partially visible source hand.At inference, the editor converts a human demonstration’s first frame into the target-embodiment initial state for rollout generation.Source: §2.1.4 · pp. 15–16, Figure 1 · p. 16

Backbone and fine-tuning strategy

• Full fine-tuning of the Wan 2.1 14B backbone and new conditioning modules jointly adapts motion, camera-following, and embodiment-conditioned appearance.Both training stages use the factorized interface of action-control video, camera trajectory, and embodiment-aware text.Source: §2.2 · p. 16, Table 1 · p. 17

Human interaction pretraining

• Human pretraining uses aligned 115-frame EgoDex clips for 30K steps on 32 NVIDIA H200 GPUs to learn general egocentric interaction priors.Each sample includes action-control video, camera trajectory, a first-frame condition, and structured text.Source: §2.2 · pp. 16–17

Mixed-embodiment fine-tuning

• Mixed-embodiment fine-tuning uses unpaired EgoDex and IRON or RoboCasa GR1 videos for 5K steps on 8 NVIDIA H200 GPUs, maintaining a 1:1 human-to-robot data ratio.The human subset is resampled across epochs to retain diverse human interactions while repeatedly exposing the model to robot data; this stabilizes robot generation without catastrophically overwriting the human prior.Source: §2.2 · p. 17, Table 1 · p. 17

Structured embodiment tags

• Structured captions separate scene, action, and acting subject, allowing the same scene and action description to be reused across embodiments by changing only the subject phrase.The scene describes the environment and object layout, the action describes manipulation behavior, and the subject specifies the acting embodiment.Source: §2.2 · p. 17

Embodiment binding through cross-attention

• Cross-attention injects structured captions, and the target first frame plus embodiment tag suffices to bind visual state and local geometry to the acting body's identity.The design adds no explicit embodiment classifier, identity loss, or separate embodiment-token module.Source: §2.2 · p. 17

2.3 Policy Adaptation Details

• At each target domain, AnyWorld rollouts augment robot data for a 1K-step intermediate UniT adaptation, followed by 500 steps of real-robot-only fine-tuning under a matched baseline protocol.The baseline and AnyWorld method share the UniT initialization, target-robot data, and final fine-tuning; the added re-embodied videos are the sole difference.Source: §2.3 · pp. 17–18

2.4 Targeted Human-to-IRON Visual–Action Pair Construction

• Targeted IRON examples pair re-embodied human videos with calibrated actions, preserving task semantics and scene context while matching the visible robot morphology to its action supervision.Actions are resampled to 30 FPS and 50 steps, interpolated, mapped into the IRON-hand PCA subspace, calibrated and clipped, then re-anchored to the paired robot state.Source: §2.4 · p. 18
• To counter a false completion prior, re-embodied demonstrations show a banana already in the box alongside another valid target, without changing the policy architecture or frozen visual-language backbone.The robot-only baseline handles a standard single-banana grasp-and-place task but makes only small local motions when a banana is already inside the box.Source: §2.4 · p. 18
• For language-grounded target selection, approximately counterfactual pairs hold the symmetric dual-banana scene fixed while changing the left/right instruction and aligned action, with 26 left-target and 38 right-target samples.This construction discourages fixed-side bias and supervises binding among language, target location, and robot action.Source: §2.4 · p. 18

2.5 Metric Details

• The metrics are controllability proxies rather than direct physical task-success measures; videos are uniformly sampled to 16 frames, and method scores are averaged over valid evaluated videos.Standard error is reported when applicable, and invalid samples are excluded from the corresponding metric average.Source: §2.5 · p. 19
• ActionAlign measures action-following as the average positive cosine similarity between background-compensated generated flow and control-skeleton flow within the valid action region.Global background motion is estimated outside the action mask and subtracted to reduce the effect of egocentric camera motion.Source: §2.5 · p. 19
• CameraAlign compares source and generated camera-motion vectors using positive cosine similarity of median background flow, excluding the action region when a control mask is available.It reflects agreement in global pan, tilt, or translation direction.Source: §2.5 · p. 19
• EmbodAcc is binary frame-level embodiment-recognition accuracy, determined by whether cropped generated frames score higher against robot or human CLIP prompt sets as requested.Up to eight lower-image-region crops are sampled per video; this is not a continuous visual-fidelity score.Source: §2.5 · p. 19

2.6 Physically Consistent Outcome Simulation

• The controlled comparison varies root motion while keeping the initial scene, scene layout, object configuration, and embodiment fixed.This tests responses to embodied motion changes rather than visual plausibility alone.Source: §2.6 · p. 19
• AnyWorld predicts successful grasps for root motions that bring the robot within reach and misses or incomplete grasps for spatially misaligned motions.The outcomes indicate dependence between global motion, object reachability, and downstream contact outcome.Source: Figure 2 · p. 20, §2.6 · p. 20
• Modeling both positive and negative outcomes under controlled root-motion variations provides more informative rollouts for downstream policy learning and evaluation.The section frames motion-conditioned interaction success, not realistic video synthesis alone, as important for generated robot experience.Source: §2.6 · p. 20