PaperlayerView exact revision on arXiv

Source-linked AI summary

WristWorld: Generating Wrist-Views via 4D World Models for Robotic Manipulation

Zezhong Qian, Xiaowei Chi, Yuming Li, Shizun Wang, Zhiyuan Qin, Xiaozhu Ju, Sirui Han, Shanghang Zhang

arXiv:2510.07313v1cs.CVcs.RO

TL;DR

Wrist-view data is scarce in large robotic datasets, although it captures fine-grained interactions important to manipulation. WristWorld reconstructs wrist viewpoints from anchor views and generates temporally coherent videos; experiments report strong video-generation results and downstream VLA gains, including a 3.81% increase in Calvin average task completion length.

  • Problem

    Large-scale robotic datasets have limited wrist-view coverage, despite wrist observations capturing fine-grained hand–object interactions important to precise manipulation.

    Source: §1 · p. 2

  • Method

    WristWorld reconstructs wrist poses and 4D geometry from anchor views, then generates wrist-view videos conditioned on reconstructed projections and anchor-view semantics.

    Source: §1 · p. 2, §3 · p. 3, §3.3 · p. 6

  • Results

    WristWorld achieves state-of-the-art wrist-view video generation with superior spatial consistency and increases Calvin average task completion length by 3.81%.

    Source: §1 · p. 2

  • Takeaways & Limitations

    WristWorld can extend existing single-view world models with multi-view capability without requiring new wrist-view data collection.

    Source: §1 · p. 2

Abstract

from arXiv · show

Wrist-view observations are crucial for VLA models as they capture fine-grained hand-object interactions that directly enhance manipulation performance. Yet large-scale datasets rarely include such recordings, resulting in a substantial gap between abundant anchor views and scarce wrist views. Existing world models cannot bridge this gap, as they require a wrist-view first frame and thus fail to generate wrist-view videos from anchor views alone. Amid this gap, recent visual geometry models such as VGGT emerge with geometric and cross-view priors that make it possible to address extreme viewpoint shifts. Inspired by these insights, we propose WristWorld, the first 4D world model that generates wrist-view videos solely from anchor views. WristWorld operates in two stages: (i) Reconstruction, which extends VGGT and incorporates our Spatial Projection Consistency (SPC) Loss to estimate geometrically consistent wrist-view poses and 4D point clouds; (ii) Generation, which employs our video generation model to synthesize temporally coherent wrist-view videos from the reconstructed perspective. Experiments on Droid, Calvin, and Franka Panda demonstrate state-of-the-art video generation with superior spatial consistency, while also improving VLA performance, raising the average task completion length on Calvin by 3.81% and closing 42.4% of the anchor-wrist view gap.

1 INTRODUCTION

WristWorld targets the shortage of wrist-view data by generating wrist-view videos from anchor views alone. Its two-stage reconstruction and generation pipeline enforces geometric and temporal consistency, and experiments report downstream VLA gains.

1 INTRODUCTION

  • Wrist-view observations capture fine-grained hand–object interactions important to precise manipulation, but large-scale robotic datasets offer limited wrist-view coverage.Collecting wrist-view recordings requires extra sensors, careful calibration, and specialized setups.Source: §1 · p. 2
  • Anchor-to-wrist synthesis is difficult because articulated manipulators cause time-varying occlusions, target wrist perspectives may be unseen in training, and anchor-view reconstructions can be sparse and temporally inconsistent.Naïve view synthesis under these conditions can produce spatial or temporal artifacts.Source: §1 · p. 2
  • WristWorld reconstructs geometrically consistent 4D point clouds and wrist-view camera poses, then generates temporally coherent videos from reconstructed wrist projections and CLIP-encoded anchor-view features.Its reconstruction stage extends VGGT with a dedicated wrist head and uses Spatial Projection Consistency loss to align 2D correspondences with reconstructed geometry.Source: §1 · p. 2
  • WristWorld achieves state-of-the-art wrist-view video generation with superior spatial consistency, increases Calvin average task completion length by 3.81%, and closes 42.4% of the anchor–wrist performance gap.It can also extend existing single-view world models without new wrist-view data collection.Source: §1 · p. 2
  • The framework is presented as a two-stage approach that targets both temporal and geometric consistency in anchor-to-wrist video generation.Its contributions also identify wrist-head reconstruction, SPC loss, and anchor-view semantic features as components of consistent wrist-view synthesis.Source: §1 · p. 2

2 RELATED WORK

Prior work spans geometric reconstruction, video generation for manipulation, and VLA policies. The reviewed approaches leave moving wrist-camera pose and anchor-to-wrist generation as relevant gaps.

3D Reconstruction for Robotic Perception

  • Robotic reconstruction work combines scene geometry with policies or predicts multi-view geometry, but moving wrist-camera pose is rarely modeled and large datasets rely on manual calibration rather than online wrist-centric pose prediction.GNFactor jointly optimizes a NeRF scene model and manipulation policy; VGGT encodes multi-view observations for 3D prediction (Wang et al., 2025b).Source: §2 · pp. 2–3

Video Generation Models for Manipulation

  • Manipulation video-generation work addresses compositionality, controllability, hallucinations, planning, synthetic data, spatial consistency, and multitask policies through varied conditioning and generation strategies.Examples include RoboDreamer (Zhou et al., 2024), This&That (Wang et al., 2025a), VideoAgent, DreamGen, and EnerVerse (Huang et al., 2025; Li et al., 2025).Source: §2 · p. 3

Vision–Language–Action (VLA) Robotics Models

3 METHOD

WristWorld uses a two-stage pipeline to generate geometrically consistent wrist-view videos from third-person observations. It reconstructs wrist poses and point-cloud projections, then uses those projections and semantic guidance to synthesize temporally coherent sequences.

3 METHOD

  • The method reconstructs wrist poses and point-cloud projections from multi-view features, then generates temporally coherent wrist-view sequences conditioned on those maps and semantic guidance.The framework is illustrated in Figure 2.Source: §3 · p. 3, Figure 2 · p. 4

3.1 PRELIMINARY

  • The preliminary video-diffusion setup denoises corrupted video latents, while VGGT supplies fused multi-view features and predicts 3D quantities and cross-view correspondences.The DiT receives conditioning tokens to guide denoising.Source: §3.1 · pp. 3–4

3.2 RECONSTRUCTION STAGE

  • A wrist head uses aggregated VGGT features and learnable wrist queries to estimate wrist-camera pose, including when wrist-view data is unavailable.The wrist queries attend to aggregated features through a Transformer decoder.Source: §3.2 · p. 4
  • SPC enforces geometric consistency by matching anchor and wrist pixels, associating anchor pixels with reconstructed 3D points, and comparing their predicted wrist projections with matched wrist pixels.The projection loss combines reprojection consistency and depth feasibility.Source: §3.2 · pp. 4–5

3.3 GENERATION STAGE

  • The model projects reconstructed scenes under the estimated wrist poses into temporally aligned condition maps for video generation.The maps provide frame-consistent structural guidance.Source: §3.2 · p. 5, §3.3 · p. 5
  • The video generator concatenates wrist-projection latents with noisy wrist latents and supplements structural guidance with CLIP features from anchor views and text embeddings.The external semantic pathway addresses global semantics that projection maps may miss, such as small or blurred objects.Source: §3.3 · pp. 5–6
  • WristWorld achieves the best performance across all metrics on Droid and Franka Panda without first-frame guidance.The comparison includes methods without wrist-view input and methods requiring a wrist-view first frame.Source: Table 1 · p. 6

4 EXPERIMENT

Across the evaluated datasets, WristWorld generates wrist-view videos that outperform baselines and improve downstream VLA performance. The experiments also examine dataset and training setups, plug-in use with single-view world models, and the contributions of individual components.

4.1 IMPLEMENTATION DETAILS

  • Pretraining samples 10k Droid videos using two anchor views; Calvin uses 10% of its D-split data for training, and the team collects 1,700 Franka Panda demonstrations.The framework trains reconstruction and video generation in two stages, then performs cross-view fine-tuning with Franka demonstrations.Source: §4.1 · pp. 6–7

4.3 DATA-DRIVEN VLA ENHANCEMENT

  • On Calvin, wrist-view augmentation raises average task completion length by 3.81%, narrows the anchor–wrist gap by 42.4%, and improves full five-task completion by 5%.Franka Panda demonstrations also show consistent task-performance improvement with generated wrist views.Source: Table 2 · p. 7, Table 3 · p. 7, §4.3 · p. 8

4.5 ABLATION STUDY

  • Ablations identify wrist-view projection as critical to video quality, while SPC loss supports accurate condition-map guidance; the full combination yields the best performance across metrics.Removing projection sharply increases FVD and degrades perceptual metrics; the reported no-projection-embeddings result is only an estimate.Source: Table 4 · p. 8, §4.5 · p. 9

4.2 VIDEO GENERATION QUANTITATIVE EVALUATION

  • WristWorld surpasses prior work across all reported metrics on Droid and Franka Panda without first-frame guidance; visualizations show improved spatial and viewpoint consistency on Calvin.On Franka Panda, generated wrist views closely match ground truth.Source: §4.2 · p. 7, Figure 5 · p. 8

4.3 DATA-DRIVEN VLA ENHANCEMENT

  • The VLA evaluation augments training data with generated wrist-view sequences while keeping the VPP architecture, losses, and demonstrations unchanged.The method is evaluated as data augmentation rather than a change to the VLA model itself.Source: §4.3 · p. 8

4.4 PLUG-AND-PLAY EXTENSION TO SINGLE-VIEW WORLD MODELS

  • A plug-in module converts a single-view world model’s anchor rollout into temporally aligned wrist video without wrist-frame initialization, improving spatial and perceptual quality across baselines.The table caption reports high-quality results with fewer anchor views; Franka visualizations compare generated sequences with ground-truth wrist observations.Source: §4.4 · pp. 8–9, Table 5 · p. 9, Figure 6 · p. 9

5 CONCLUSION

WristWorld reconstructs wrist-view geometry from anchor views and uses it to generate temporally coherent videos. Experiments across three settings report strong video-generation results and downstream VLA gains.

5 CONCLUSION

  • WristWorld is a two-stage 4D framework that synthesizes geometrically and temporally consistent wrist-view videos from anchor-view inputs.A wrist head and SPC loss support pose estimation and condition-map generation; the maps are combined with CLIP semantics and text guidance for video synthesis.Source: §5 · pp. 9–10
  • The framework’s supported scope includes synthesizing wrist views from anchor-view inputs and augmenting datasets with those generated views.The conclusion describes this as a scalable, data-driven solution for robotic training.Source: §5 · pp. 9–10
  • Experiments on Calvin, Droid, and Franka Panda report strong video-generation quality and significant improvements in downstream VLA learning.The conclusion frames wrist-centric views as a way to bridge exocentric and egocentric perspectives for robotic training.Source: §5 · p. 10

A.1 DATASETS

The appendix describes three data sources used to evaluate wrist-view generation and its downstream manipulation applications: simulated Calvin data, large-scale real-world Droid data, and an in-house Franka Panda collection.

Calvin

  • Calvin is a simulated benchmark for long-horizon, language-conditioned manipulation, with five core tasks that can be composed into multi-step instructions.Its synchronized multi-view demonstrations and environment variations support controlled evaluation of generalization to new objects, layouts, and task compositions.Source: §A.1 · p. 14

Droid

  • Droid contains approximately 76k trajectories across 350 hours, 564 scenes, and 86 manipulation tasks, with synchronized multi-view RGB-D, robot-state, and control data.Its standardized Franka Panda setup includes two external anchor cameras and a wrist-mounted camera.Source: §A.1 · p. 14

Franka Panda demonstrations

  • The in-house Franka Panda dataset contains approximately 1.7k demonstrations of varied manipulation skills, recorded with synchronized anchor and wrist views, proprioception, and control commands.It captures calibration imperfections and actuation variability under real physical dynamics.Source: §A.1 · p. 14

A.2 TRAINING AND IMPLEMENTATION DETAILS

WristWorld combines a VGGT-based reconstruction stage with a diffusion-transformer generation stage to produce wrist-view videos from anchor views. Its implementation trains a wrist head for camera-pose prediction and conditions video generation on wrist projections and anchor-view features.

Reconstruction stage

  • The reconstruction stage freezes VGGT-1B’s point, depth, and camera heads while training a wrist head to predict wrist pose from two anchor views relative to ext1.The wrist head is a three-layer transformer decoder with eight attention heads per layer and 1024-dimensional embeddings; training images are resized to 518×518 with intrinsics scaled accordingly.Source: §A.2 · pp. 14–15
  • SPC supervises visible points with normalized reprojection error and penalizes back-facing points with negative predicted depth, splitting projections by wrist-frame depth.The projections use dataset-provided intrinsics.Source: §A.2 · pp. 14–15

Generation stage

  • Video generation builds on Wan 1.3B DiT, concatenating VAE-encoded wrist-projection condition maps with noisy wrist-view latents and adding projected anchor-view CLIP features as pseudo tokens.Temporal and view embeddings represent sequence alignment and camera identity; generation uses at most 512 conditioning tokens and CFG 5.0.Source: §A.2 · p. 15

A.3 VISUALIZATION

The Franka Panda visualizations compare generated middle frames across methods and manipulation scenes. WristWorld shows the strongest geometric consistency and wrist-following, though rare small, fast-moving details may still blur.

Evaluation setup

  • Compared methods use identical input trajectories and preprocessing, with middle frames chosen to emphasize long-horizon behavior and reduce initial-condition bias.SVD, Wan 14B, and Cosmos-Predict2 use first-frame guidance; the remaining methods use public default settings.Source: §A.3 · pp. 15–16
  • Higher classifier-free guidance yields sharper but less diverse outputs, reflecting its fidelity–diversity trade-off in diffusion sampling.Source: §A.3 · p. 15

Qualitative results

  • Across diverse scenes, WristWorld shows the strongest geometric consistency and wrist-following; Figure 7 rows denote methods and columns denote manipulation scenes.The top row shows ground-truth wrist-camera frames, and middle-frame visualizations probe long-horizon stability.Source: §A.3 · p. 16, Figure 7 · p. 16
  • First-frame-guided baselines tend to show background and pose drift over time, while unguided methods often show ghosting and fine-detail loss under fast motion or partial occlusions.WristWorld may still blur very small, fast-moving details in rare cases.Source: §A.3 · p. 16

A.4 FRANKA PANDA SETTING

The Franka Panda setup uses multiple RealSense cameras to provide wrist, overhead, and side views. These perspectives support reconstruction and visual feedback for manipulation.

Camera arrangement

  • The wrist camera provides close-up workspace views, while the top camera captures overhead information.Source: §A.4 · p. 16, Figure 8 · p. 17
  • Left and right side cameras provide complementary perspectives that facilitate 3D reconstruction and visual feedback for manipulation.Source: §A.4 · p. 17
  • The Franka Panda setup includes wrist-mounted, top, left, and right Intel RealSense cameras.Source: §A.4 · p. 16, Figure 8 · p. 17

Related papers

Loading 2510.07313v1…