WAM-Nav: Asymmetric Latent World-Action Modeling for Unified Visual Navigation

• Reactive policies lack predictive reasoning, while decoupled approaches incur computational latency and compounding errors from separately trained action and imagination modules.Source: §1 · p. 2
• WAM-Nav jointly models action generation and latent visual foresight within a unified framework, unlike purely reactive or decoupled modular pipelines.Source: §1 · pp. 1–2, Figure 1 · p. 2
• A shared Diffusion Transformer uses asymmetric joint diffusion to generate long-horizon actions and short-horizon visual foresight concurrently.Source: §1 · p. 2
• Dual-stream conditioning combines episode-level ego-motion history with sequential visual observations, while unified goal alignment supports Image-Goal, Point-Goal, and No-Goal exploration in one policy.Source: §1 · p. 2
• 15.7% and 3.3% success-rate improvements are reported for Image-Goal and Point-Goal navigation, respectively.Source: §1 · p. 3
• Real-world deployment achieves an average 85% task success rate across diverse indoor and outdoor environments, supporting zero-shot sim-to-real transfer.Source: §1 · p. 3

2.1 Learning-based Embodied Visual Navigation

• Reactive learning-based policies directly map local observations to actions but do not explicitly model future perceptual consequences, leaving them susceptible to collisions in cluttered environments [14] [28].Source: §2.1 · p. 3
• Recent methods extend navigation with CNN policies, Transformer sub-goal planning, diffusion action distributions, or actor-critic trajectory selection [9] [10].Source: §2.1 · p. 3

2.2 World Models and Visual Imagination for Navigation

• Visual-imagination systems such as Pathdreamer, NWM, and NavigateDiff decouple future visual prediction from trajectory planning or inverse-dynamics execution [31], [16], [14].Source: §2.2 · p. 3
• WAM-Nav jointly predicts latent visual dynamics and actions to promote temporal consistency and physical feasibility while minimizing inference latency.Source: §2.2 · p. 3

3.1 Unified Goal Alignment

• WAM-Nav projects Image-Goal, Point-Goal, and No-Goal inputs into visual-semantic and geometric queries, preserving modality-specific information within a unified interface.The resulting queries condition subsequent action and visual-foresight generation.Source: §3.1 · pp. 3–4

3.2 Dual-Stream Contextual Conditioning

• Dual-Stream Contextual Conditioning fuses goal-modulated visual memory with trajectory-aware kinematic history to guide collision-free and smooth action trajectories.Visual memory supplies spatial geometry, while motion history captures physical momentum; their fusion forms the conditioning context C.Source: §3.2 · pp. 4–5

3.3 Asymmetric Action-Foresight Generation

• The shared DiT jointly models long-horizon action trajectories and short-horizon latent visual foresight under the unified context C, with the visual horizon no longer than the action horizon.The design uses a shared representation space and asymmetric horizons to retain trajectory continuity while limiting visual prediction length.Source: §3.3 · p. 5
• Shared self-attention exchanges constraints between action and latent-visual tokens, while cross-attention injects the conditioning context and adaptive normalization incorporates the flow timestep.The shared DiT predicts joint action and visual velocity fields through stacked blocks.Source: §3.3 · p. 5

3.4 Model Training and Inference

• WAM-Nav trains end-to-end with flow-matching velocity regression and multimodal goal alignment, then performs receding-horizon control by sampling candidate trajectories and executing the first.The alignment term uses a symmetric contrastive InfoNCE loss to maintain consistency across goal modalities.Source: §3.4 · p. 6

4.1 Simulation Experiments

• 50.2% SR / 48.2% SPL on Image-Goal, 80.4% SR / 78.0% SPL on Point-Goal, and 171.1 m2 explored area on No-Goal represent WAM-Nav’s best average task performance.The gains are consistent across ClutterScenes and InternScenes, supporting unified multi-task navigation.Source: §4.1 · p. 7
• WAM-Nav produces smoother, more proactive obstacle avoidance than NavDP while its decoded latent foresights remain consistent with ground-truth observations.Short-horizon foresight anticipates near-future geometric constraints without long autoregressive visual rollouts.Source: §4.1 · p. 7, Figure 3 · p. 7
• 0.26 s inference latency and 0.7 TFLOPs per decision keep WAM-Nav efficient, compared with NWM’s 1.43 s and 8.3 TFLOPs.WAM-Nav has comparable trainable parameters to NavDP despite a larger total count from frozen visual priors.Source: §4.1 · p. 7
• 50.2% SR / 48.2% SPL is achieved when ego-motion history and latent foresight are combined, while either component alone is weaker or less consistent across scenes.Motion history improves smoothness, whereas latent foresight supplies geometric constraints for obstacle avoidance.Source: §4.1 · p. 7

4.2 Deployment on Real World

• 85% average success is achieved in zero-shot deployment across four indoor and outdoor environments on a Unitree G1 humanoid robot.The evaluation uses 10 trials per scene and compares WAM-Nav with NavDP in real-world scenarios.Source: §4.2 · p. 8, Figure 4 · p. 8

5 Conclusion

• WAM-Nav unifies long-horizon action generation and latent visual foresight, with dual-stream conditioning and unified goal alignment supporting Image-Goal, Point-Goal, and No-Goal navigation.The conclusion links these components to geometrically safe and kinematically smooth trajectories.Source: §5 · p. 8
• Limited camera height and field of view can impair near-field obstacle perception, while absent explicit embodiment modeling can mismatch camera-level avoidance with full-body traversability.The paper proposes adaptive viewpoint control and embodiment-aware training as future directions.Source: §5 · p. 8

A Overview

• The supplementary material covers the problem definition, method details, training configuration, baselines, experiments, deployment hardware, and a failure-case visualization.These materials span Sections B through H.Source: A Overview · p. 12

B Problem Definition

• The policy conditions asymmetric joint generation on visual history, ego-motion history, and the goal to produce long-horizon actions alongside short-horizon latent visual foresight.Long-horizon actions preserve continuity while short-horizon foresight supplies near-future geometric constraints.Source: B Problem Definition · p. 12

C.1 Trajectory-Aware Motion History

• The motion-history stream updates a sliding pose buffer online and re-expresses it in the current egocentric frame as relative translations and heading differences.The procedure recenters on the current pose, rotates translations, and wraps heading differences.Source: §C.1 · p. 13

C.2 Architecture of the Shared DiT Block

• The stacked shared DiT blocks project the coupled streams to action and visual flow-matching velocity fields, enabling consistent joint generation.The architecture uses stream-specific modulation with shared parameters for cross-modal coupling.Source: §C.2 · p. 13
• Separate adaLN-Zero branches modulate noised action and latent visual tokens before shared self-attention couples the two streams.Shared cross-attention and FFN layers then ground both streams on conditioning context C.Source: Figure 5 · p. 13, §C.2 · p. 13
• Table 2 summarizes WAM-Nav’s key hyperparameters for training and deployment.Source: Table 2 · p. 14

D Implementation Details of WAM-Nav

• The visual backbone and Stable Diffusion VAE remain frozen, while the goal encoder, fusion decoder, motion encoder, and shared DiT are trained from scratch.Training uses AdamW, a 1.5×10^-4 peak learning rate, cosine annealing with 5% warmup, and 10 Euler deployment steps.Source: D Implementation Details of WAM-Nav · p. 14

E Baseline Details

• The baseline suite covers Point-Goal reinforcement learning, reactive trajectory and semantic planners, navigation foundation models, diffusion policies, and generative world models.Source: E Baseline Details · p. 14
• NWM provides a generative world-model comparison by predicting future egocentric observations for foresight-driven planning.Source: E Baseline Details · p. 14
• NavDP serves as a diffusion-based baseline using privileged simulation priors for zero-shot sim-to-real transfer.Source: E Baseline Details · p. 14

• Table 3 compares performance across three navigation tasks.Source: Table 3 · p. 15

• Table 4 evaluates different WAM-Nav components on Image-Goal navigation.Source: Table 4 · p. 15

F.3 Additional Ablation Studies

• The combined goal queries achieve the strongest No-Goal exploration while retaining competitive Image-Goal and Point-Goal performance.Single visual or geometric queries favor different task types and produce less balanced results.Source: §F.3 · p. 15
• Both insufficient and excessive temporal context reduce navigation performance, while removing explicit goal injection causes a clear performance drop.A short memory window provides limited motion context, whereas an overly long window reduces responsiveness to sudden geometric changes.Source: §F.3 · pp. 15–16
• The fully shared DiT outperforms decoupled and partially shared action-foresight architectures.This supports directly coupling latent foresight with action generation through shared representations.Source: §F.3 · p. 16
• Short visual foresight with H_vis = 1 performs best, whereas longer visual horizons progressively degrade performance.The action horizon remains fixed at H_act = 24 to preserve long-horizon trajectory continuity.Source: §F.3 · p. 16, Table 7 · p. 16

F.4 Comparison with NavDP

• 42.9% SR on the G1 platform demonstrates zero-shot WAM-Nav performance without parameter retraining.WAM-Nav also outperforms NavDP across all evaluated G1 task settings and achieves higher performance across tasks on H2.Source: §F.4 · p. 17, Table 8 · p. 17
• In the Hard Image-Goal subset, WAM-Nav reaches 25.0% SR and 23.8% SPL, surpassing NavDP by 7.6% and 2.5%, respectively.The performance gap increases as task difficulty grows.Source: §F.4 · p. 17, Table 9 · p. 17

G.1 Hardware setup

• The Unitree G1 setup uses an RGB-D camera on a pan-tilt servo and an RTX 4060 onboard computer for real-time perception and control.The camera captures RGB-D observations from fixed viewpoints while the onboard computer performs computation.Source: §G.1 · pp. 17–18, Figure 7 · p. 18

G.2 Deployment Details

• Deployment runs inference at 1 Hz, tracks predicted trajectories with MPC, and executes motor commands through a 50 Hz control loop.The model occupies approximately 1.3 GB of GPU memory, and each command is applied for 0.1 s.Source: §G.2 · p. 18

• Limited field of view can weaken perception of nearby low obstacles, causing the robot to overlook and collide with them.Source: H Additional Failure Case Analysis · pp. 18–19
• Ignoring the robot’s body shape can produce trajectories that fit the camera but collide with side obstacles.Source: H Additional Failure Case Analysis · p. 19