PaperlayerView exact revision on arXiv

Source-linked AI summary

G$^2$WAM: World Action Models for Efficient and Geometry Grounded Image-Goal Navigation

Ke Gui, Chenghao Shi, Ziyue Wang, Longwen Chen, Xieyuanli Chen, Huimin Lu

arXiv:2610.16558v1cs.AIcs.RO

TL;DR

Image-goal navigation requires both long-horizon dynamics modeling and persistent spatial context, while existing spatial approaches can underuse priors or be too costly. G2WAM anchors goal and observation frames in a shared scene coordinate system and jointly predicts latent states and actions; it reports improved trajectory prediction and closed-loop navigation, with a previously seen scene as a scope boundary.

  • Problem

    Image-goal navigation needs spatial context for targets beyond the current view, but existing approaches may underuse spatial priors or require costly external 3D reconstruction.

    Source: §1 · pp. 1–2

  • Method

    G2WAM anchors current, goal, and history frames through learned localization, then jointly predicts future latent states and actions without CEM-style planning or pixel reconstruction.

    Source: §1 · p. 2, §3 · p. 3

  • Results

    G2WAM reports best overall trajectory prediction and navigation performance, with closed-loop success rising from 54% to 63%.

    Source: §1 · p. 2, §5 · p. 10

  • Takeaways & Limitations

    The reported approach provides stable spatial context and high-efficiency navigation without explicit 3D reconstruction.

    Source: §5 · p. 10

    Scene-coordinate anchoring assumes a previously seen scene and requires pre-collected trajectory data.

    Source: §3.1 · p. 4

Abstract

from arXiv · show

World models capture environment dynamics and support counterfactual reasoning for planning. In image-goal navigation, where the target often lies beyond the current field of view, the agent must not only model environmental dynamics but also maintain a persistent spatial memory of the scene. Existing approaches either introduce spatial context by fine-tuning pretrained models on the target task, which underexploits their spatial priors, or rely on external 3D reconstruction, which is computationally expensive for real-world deployment. In this paper, we propose the Geometry Grounded World Action Model (G$^2$WAM) for image-goal navigation. G$^2$WAM encodes spatial cues through a learned scene-coordinate localization module that anchors the current, goal, and history frames in a common reference frame, yielding a stable and consistent spatial context that enables efficient navigation without explicit 3D reconstruction. Following the world-action-model paradigm, G$^2$WAM jointly predicts future states and actions, eliminating the need for costly cross-entropy method (CEM) style planning. In addition, G$^2$WAM operates entirely in latent space, avoiding the computational burden of pixel-level reconstruction and substantially reducing inference cost. Across four trajectory prediction benchmarks, G$^2$WAM reduces average ATE and RPE by 40.1% and 35.2%, respectively. In closed-loop navigation experiments, it improves the success rate from 54.0% to 63.0% and the success weighted by path length (SPL) from 36.4% to 59.2%, while maintaining high efficiency.

1 INTRODUCTION

Image-goal navigation must support long-horizon reasoning despite rollout degradation, costly planning, and limited spatial context. G2WAM combines shared-frame localization with joint latent-state and action prediction, reporting stronger trajectory and navigation results.

1 INTRODUCTION

  • Long-horizon imagination degrades as rollout errors accumulate, while CEM requires repeated forward evaluations and open-loop plans need conversion into closed-loop waypoints.The waypoint conversion introduces additional error, according to the paper.Source: §1 · p. 1
  • Existing spatial-context approaches either underuse spatial priors or rely on methods too heavy for practical deployment.Prior approaches include injecting depth during training or adding external 3D reconstruction, Gaussian-splatting, or latent spatial memory.Source: §1 · pp. 1–2
  • G2WAM jointly predicts latent future states and executable action chunks, avoiding CEM-style planning and pixel-level future reconstruction.This world-action-model design is presented as a way to reduce planning and reconstruction costs.Source: §1 · p. 2
  • A learned localization network anchors observations and the goal in a shared frame, providing consistent spatial context from relative pose and navigation history without explicit geometry reconstruction.The context includes the egocentric relative pose to the goal and the history trajectory.Source: §1 · p. 2
  • Average ATE and RPE fall by 40.1% and 35.2%, while closed-loop success rises from 54% to 63% and SPL from 36.4% to 59.2%.The paper reports best overall trajectory prediction and navigation performance while maintaining high efficiency.Source: §1 · p. 2

2 RELATED WORK

Prior navigation work spans reactive policies, world models, and geometry-aware methods for spatial consistency. G2WAM combines latent state-action prediction with estimated-pose context rather than explicit scene reconstruction or future-image generation under geometric constraints.

2.1 VISUAL NAVIGATION

  • Reactive goal-conditioned policies map current and goal images to actions but do not explicitly predict future states; metric localization offers complementary geometric guidance.The related work contrasts these policies with world-model-based approaches that model environment dynamics.Source: §2.1 · pp. 2–3

2.2 WORLD MODELS AND WORLD ACTION MODELS FOR NAVIGATION

  • Navigation world models predict future images or latent states, while world action models jointly predict state and action; G2WAM adds relative-pose and causal-motion context and predicts metric waypoints without future-image reconstruction.The passage distinguishes prior examples including NWM, DINO-WM, LS-NWM, ReL-NWM, Spatial-perceiving WAM, NavWAM, and WAMNav.Source: §2.2 · p. 3
  • Geometry-aware world models either maintain geometry-grounded spatial memory or impose geometric constraints without a persistent scene representation.Examples include explicit 3D scene construction and reprojection, geometry-organized historical memory, feature alignment, and future-anchor guidance.Source: §2.2 · p. 3
  • Unlike those approaches, G2WAM derives spatial context from estimated poses and jointly rolls out latent states and executable actions without explicit 3D reconstruction.It uses a lightweight metric condition rather than generating future images under geometric constraints.Source: §2.2 · p. 3

3 METHOD

G2WAM uses lightweight relative geometry to condition a latent world-action rollout that predicts future states and waypoint increments. Its localization interface, action conditioning, and training objective are designed to provide spatially grounded navigation without explicit scene reconstruction or pixel decoding.

3 METHOD

  • G2WAM anchors goal and observed frames in a shared scene frame, then jointly predicts latent future states and canonical waypoint increments using relative geometry.The model rolls out in latent space and avoids CEM-style planning and pixel-level reconstruction.Source: §3 · p. 3, Figure 2 · p. 4

3.1 SPATIAL CONTEXT VIA SCENE-COORDINATE ANCHORING

  • Scene-coordinate anchoring estimates current, goal, and history-frame poses to form the current-to-goal transform and consecutive motion increments.Pre-collected scene data and camera-to-robot calibration are part of the localization interface; the world-action model can use an end-to-end or traditional localization pipeline.Source: §3.1 · p. 4
  • The anchoring approach assumes a previously seen scene and requires pre-collected trajectory data.The paper motivates this scope by its focus on out-of-field-of-view navigation.Source: §3.1 · p. 4

3.2 IMPLICIT WORLD ACTION MODEL

  • A frozen image encoder supplies patch latents, and a goal-conditioned transition context feeds a joint decoder that predicts the next latent and canonical waypoint increment in an autoregressive rollout without image decoding.The relative pose and motion profile condition the model alongside latent history and goal information.Source: §3.2 · p. 5

3.3 SPATIAL CONTEXT CONDITIONING AND ACTION ADAPTION

  • Relative pose supplies target geometry, while recent motion records trajectory history and adapts canonical predictions into platform-specific actions.The motion profile includes median per-step distance, mean absolute yaw change, dataset-level translation and yaw scales, and waypoint spacing.Source: §3.3 · pp. 5–6

3.4 TRAINING

  • Training combines latent, canonical-action, cumulative waypoint-position, and yaw losses, with extra weight for rare turning steps.The localization interface is trained separately from the end-to-end G2WAM objective.Source: §3.4 · p. 6

4 EXPERIMENTS

The experiments evaluate trajectory prediction, zero-shot transfer, closed-loop navigation, spatial-context conditioning, and runtime efficiency. G2WAM shows its strongest results in trajectory prediction and closed-loop navigation, while ablations and runtime comparisons assess its spatial inputs and efficiency.

4.1 EXPERIMENTAL SETUP

  • The evaluation uses 100 fixed queries per trajectory dataset, 100 validation episodes in Gibson’s known Mosquito scene, and shared held-out comparisons with task-specific metrics.It covers RECON, SACSoN, TartanDrive, and SCAND, plus zero-shot Go Stanford transfer; navigation success is evaluated at a 2 m geodesic threshold.Source: §4.1 · pp. 6–7

4.2 TRAJECTORY PREDICTION

  • G2WAM achieves average ATE/RPE of 1.82/0.46 across four datasets, reducing these errors by 40.1%/35.2% relative to the second-best baseline.The gains are strongest on SACSoN and TartanDrive; performance is comparable on RECON and degrades on SCAND, where localization is limited by cross-platform motion, operator variation, and pedestrians.Source: §4.2 · p. 7, Table 1 · p. 7
  • Across the four prediction datasets, G2WAM rollouts remain better aligned with ground-truth paths than baseline rollouts over the 8-step horizon.Baselines often drift from the target direction or accumulate scale error; G2WAM-GT produces the cleanest rollouts.Source: §4.2 · p. 7, Figure 3 · p. 8
  • On zero-shot Go Stanford transfer, G2WAM is comparable to baselines without spatial context; ground-truth pose assistance improves ATE/RPE by 14.0%/16.5%.The transfer setting tests whether the world-action-model backbone has learned navigation priors without spatial context.Source: §4.2 · pp. 7–8, Table 2 · p. 8

4.3 CLOSED-LOOP NAVIGATION

  • G2WAM reaches 63.0% SR and 59.2% SPL in closed-loop navigation, improving on DINO-WM by 9.0 and 22.8 percentage points, respectively.Its final DTG is 3.39 m, 25% lower than LeWM’s; G2WAM-GT reaches 71.0% SR, 70.0% SPL, and 3.04 m final DTG.Source: §4.3 · p. 8, Table 3 · p. 8

4.4 ABLATIONS

  • With R-SCoRe-estimated poses, pose and motion conditioning improve average ATE/RPE by 39.1%/35.2%; ideal pose estimation or external localization raises the gains to 73.9%/60.6%.The tested conditions are no spatial context, goal-and-current-image poses only, and poses plus goal, current, and history trajectories.Source: §4.4 · p. 9, Table 4 · p. 9

4.5 RUNTIME ANALYSIS

  • G2WAM is comparable in efficiency to state-of-the-art latent world models and has an action-selection advantage over world-model methods that use CEM planning.Localization takes 0.42 s versus EvoWorld’s 0.57 s geometric-context encoding; EvoWorld’s overall action-selection cost is 8.7 s because it re-encodes context at every CEM rollout.Source: §4.5 · p. 9, Table 5 · p. 9

5 CONCLUSION

G2WAM grounds image-goal navigation in shared-frame spatial cues and jointly predicts latent future states and actions, achieving strong trajectory prediction and navigation performance with high inference efficiency.

5 CONCLUSION

  • G2WAM anchors goal and observed frames in a common reference frame, providing stable spatial context without explicit 3D reconstruction.It jointly predicts future actions and latent future states, avoiding costly CEM-style planning and pixel-level reconstruction.Source: §5 · p. 10
  • Across experiments, G2WAM achieves state-of-the-art trajectory prediction and navigation performance with high inference efficiency.Source: §5 · p. 10
  • Future work will extend G2WAM to longer-horizon navigation and broader embodied tasks.Source: §5 · p. 10

A.1 WORLD-ACTION MODEL ARCHITECTURE

The architecture combines frozen visual patch latents with goal and relative-pose conditioning to model history and jointly predict future latent states and executable actions.

A.1 WORLD-ACTION MODEL ARCHITECTURE

  • A frozen DINOv3 encoder maps resized current, historical, and goal images to patch-level features that retain spatial structure.The transition model uses patch information to represent spatial changes across observations.Source: §A.1 · p. 12
  • The model conditions on four recent observation latents and a goal latent, which is layer-normalized, mean-pooled, and projected into a goal embedding.Source: §A.1 · p. 13
  • Layer-normalized history patches receive temporal and patch embeddings; token selection compresses the sequence into eight learned tokens before Transformer encoding and fusion back to every patch.The fused latest-frame patches query the full fused history, producing patch-level transition context for latent-state and action prediction.Source: §A.1 · pp. 13–14
  • Relative pose conditions the patch sequence, learned-token sequence, cross-attention residual, and pooled step state, while causal motion conditions only the pooled step state.This design lets relative pose constrain spatial evolution while recent motion and platform scale primarily affect action generation.Source: §A.1 · p. 14
  • A joint decoder processes action, state-summary, and latent token streams, with separate heads producing canonical waypoint increments and candidate next-step latents.A base projection supplies a direct patch-level transition, while the joint branch provides the dominant state-action prediction.Source: §A.1 · p. 14
  • The model rolls out autoregressively for eight steps without reconstructing future RGB, while a motion adapter converts canonical waypoint predictions into dataset- or platform-specific executable actions.The relative pose supplies target geometry; recent motion and platform scales determine action conversion.Source: §A.1 · pp. 14–15

A.2 TRAINING DETAILS

The world-action model uses joint latent, action, and trajectory supervision, while its localization interface is trained separately and remains fixed during navigation-model optimization.

A.2 TRAINING DETAILS

  • The objective combines patchwise latent cosine distance and a smaller mean-squared-error term with smooth-ℓ1 loss on canonical translation and yaw increments.Source: §A.2 · p. 15
  • Turning steps receive larger action-loss weights to discourage an always-forward prediction; the loss coefficients are λz = 0.3, λa = 3.0, λtraj = 2.0, and λyaw = 0.5.Source: §A.2 · p. 15
  • Trajectory and yaw losses compare cumulative executable translations and rotations across the complete rollout.Planar action increments are converted back from their normalized representation before accumulation.Source: §A.2 · p. 15
  • The four-dataset checkpoint receives 25,800 updates before Gibson adaptation, and the Mosquito checkpoint is further fine-tuned on 200 training trajectories.The Mosquito trajectories are disjoint from 100 evaluation episodes but come from the same scene; evaluation episodes are not used for model fine-tuning or localization training.Source: §A.2 · p. 15
  • The DINOv3 encoder remains frozen, and the localization interface is independently trained and not updated by world-action-model losses.Source: §A.2 · p. 15

A.3 LOCALIZATION INTERFACE

The localization interface estimates scene coordinates and camera poses, then supplies shared-frame relative poses and motion cues to the world-action model. Its navigation evaluation uses known scenes, and consistent localization remains necessary.

A.3 LOCALIZATION INTERFACE

  • R-SCoRe predicts 3D scene coordinates for query keypoints from local descriptors and retrieved global scene encodings, then uses PnP–RANSAC to recover camera pose.Candidate selection is based on geometric consistency rather than retrieval similarity alone.Source: §A.3 · p. 16
  • The original R-SCoRe evaluation reports 64.3% localization success within (0.25 m, 2°) and approximately 140–270 ms per query on an RTX 2080.These figures are from the original localization benchmarks, not the Habitat implementation; reported model-side latency excludes localization, simulation, and action execution.Source: §A.3 · p. 16
  • Current, goal, and history images are localized in one scene frame to derive goal-relative pose and consecutive history motions.Relative transformations are invariant to a common rigid change of frame, but the images must still be localized consistently.Source: §A.3 · pp. 16–17
  • The world-action model receives only relative pose and recent motion, so its localization interface can be replaced if the alternative provides poses in a consistent metric frame.The interface’s internal retrieval, scene-coordinate, and RANSAC quantities are not provided directly to the world-action model.Source: §A.3 · p. 17
  • The Habitat evaluation is a known-scene protocol using held-out trajectories from the same scene, not navigation in a completely unseen environment.Unseen-scene use would require another shared-frame localization source, such as online mapping, visual SLAM, or a map-free relative-pose estimator.Source: §A.3 · p. 17

B EVALUATION PROTOCOL AND METRIC DEFINITIONS

Trajectory prediction uses fixed eight-step rollouts on 100 test queries per dataset, while closed-loop navigation is evaluated in fixed Gibson episodes with an oracle stop threshold. The section defines the trajectory and navigation metrics and clarifies the evaluated controller and latency scope.

Trajectory prediction

  • 100 fixed test queries per dataset are evaluated over an eight-step horizon after all methods are converted to a common waypoint coordinate system.The four-dataset average is the unweighted mean over RECON, SACSoN, TartanDrive, and SCAND.Source: B EVALUATION PROTOCOL AND METRIC… · pp. 18–19
  • ATE measures complete-trajectory absolute deviation, while RPE measures consecutive relative-translation error; neither includes rotation or image-reprojection error.Both are computed from predicted and reference planar waypoint positions.Source: B EVALUATION PROTOCOL AND METRIC… · pp. 18–19

Closed-loop navigation

  • The Gibson evaluation uses 100 fixed Mosquito validation episodes, a 2 m success threshold, and at most 100 executed actions; the evaluator triggers an oracle STOP within threshold.Each cycle predicts eight latent/action steps but executes only the first waypoint before observing and replanning; geodesic distance is not a control input.Source: B EVALUATION PROTOCOL AND METRIC… · p. 19
  • SR is the successful-episode fraction, SPL weights success by shortest-path versus executed-path length, and DTG averages final geodesic distance across all episodes.Reported model-only latency excludes localization, simulation, and action execution; G2WAM-GT uses oracle pose and causal motion, unlike image-estimated deployed G2WAM.Source: B EVALUATION PROTOCOL AND METRIC… · p. 19

C SUPPLEMENTARY CODE AND NAVIGATION VIDEOS

The supplementary archive documents the evaluation pipeline and its frozen assets, while the paired navigation videos provide qualitative comparisons under controlled episode conditions. The videos are outcome-selected examples, not estimates of full-test-set performance.

Code package

  • The archive includes model definitions, evaluation entry points, interfaces, fixed test-split metadata, configurations, validation scripts, and checksums for included files and frozen assets.Raw datasets and third-party components are not redistributed; training implementation and pretrained checkpoints are withheld during anonymous review and planned for release after acceptance.Source: C SUPPLEMENTARY CODE AND NAVIGATION… · p. 19
  • The archive is not a self-contained, weight-inclusive reproduction because it omits raw datasets, third-party components, training implementation, and pretrained checkpoints.It documents evaluation and identifies frozen assets, while checkpoints and training implementation are withheld during anonymous review.Source: C SUPPLEMENTARY CODE AND NAVIGATION… · p. 19

Qualitative navigation videos

  • Ten paired videos use the same Gibson Mosquito episode, start observation, and goal image; playback aligns by executed action index, not inference time.G2WAM replays the original actions with distance traces checked at every step, while NWM is rerun with a six-step distilled checkpoint and may vary stochastically.Source: C SUPPLEMENTARY CODE AND NAVIGATION… · p. 19
  • The outcome-selected videos include four successes for both methods, five only for G2WAM, and one failure for both, so they do not estimate full-test-set SR or SPL.Quantitative comparisons should use the main evaluation table.Source: C SUPPLEMENTARY CODE AND NAVIGATION… · p. 20, Table 6 · p. 20

D AI USE STATEMENT

The authors report using generative AI tools across manuscript and research activities, while retaining responsibility for the final work. They state that AI-assisted materials were reviewed and verified.

AI use statement

  • Generative AI tools assisted with language editing, literature search, manuscript drafting, software implementation, and baseline reproduction.The authors state that all AI-assisted text, code, references, and baseline implementations were reviewed and verified.Source: D AI USE STATEMENT · p. 20
  • The authors take full responsibility for the final manuscript, methods, results, claims, and artifacts.Source: D AI USE STATEMENT · p. 20
  • AI assistance included drafting portions of the manuscript and assisting with reproduction of baseline methods.Source: D AI USE STATEMENT · p. 20

Related papers

Loading 2610.16558v1…