Source-linked AI summary
$NavA^3$: Understanding Any Instruction, Navigating Anywhere, Finding Anything
Lingfeng Zhang, Xiaoshuai Hao, Yingbo Tang, Haoxiang Fu, Xinyu Zheng, Pengwei Wang, Zhongyuan Wang, Wenbo Ding, Shanghang Zhang
TL;DR
Existing navigation tasks do not fully capture human instructions involving high-level intent and spatial requirements. NavA3 combines global instruction reasoning with local object localization trained on a 1.0 million-sample affordance dataset. The authors report state-of-the-art long-horizon navigation performance in real-world environments and successful deployment across wheeled and quadruped robots.
Problem
Existing navigation tasks focus on predefined object categories or detailed instructions, while real-world requests can require inferring target objects and their spatial relationships from high-level intent.
Source: Introduction · p. 2, Preliminaries · p. 3Method
NavA3 uses a global Reasoning-VLM policy to parse instructions and identify likely goal areas, then a local NaviAfford policy trained on 1.0 million spatial-aware affordance samples to localize objects.
Source: Introduction · p. 2Results
NavA3 achieves state-of-the-art long-horizon navigation performance in large-scale real-world environments and has been deployed on wheeled and quadruped robots.
Source: Introduction · p. 2, Conclusion · p. 8Takeaways & Limitations
The reported results show that NavA3 can handle complex spatial relations and open-vocabulary pointing across different robot embodiments.
Source: Conclusion · p. 8Adaptability in dynamic environments and integration of additional sensory inputs remain future-work targets.
Source: Conclusion · p. 8
Abstract
from arXiv · showhide
Embodied navigation is a fundamental capability of embodied intelligence, enabling robots to move and interact within physical environments. However, existing navigation tasks primarily focus on predefined object navigation or instruction following, which significantly differs from human needs in real-world scenarios involving complex, open-ended scenes. To bridge this gap, we introduce a challenging long-horizon navigation task that requires understanding high-level human instructions and performing spatial-aware object navigation in real-world environments. Existing embodied navigation methods struggle with such tasks due to their limitations in comprehending high-level human instructions and localizing objects with an open vocabulary. In this paper, we propose $NavA^3$, a hierarchical framework divided into two stages: global and local policies. In the global policy, we leverage the reasoning capabilities of Reasoning-VLM to parse high-level human instructions and integrate them with global 3D scene views. This allows us to reason and navigate to regions most likely to contain the goal object. In the local policy, we have collected a dataset of 1.0 million samples of spatial-aware object affordances to train the NaviAfford model (PointingVLM), which provides robust open-vocabulary object localization and spatial awareness for precise goal identification and navigation in complex environments. Extensive experiments demonstrate that $NavA^3$ achieves SOTA results in navigation performance and can successfully complete longhorizon navigation tasks across different robot embodiments in real-world settings, paving the way for universal embodied navigation. The dataset and code will be made available. Project website: https://NavigationA3.github.io/.
Introduction
The paper motivates long-horizon navigation for human instructions that require inferring goals and spatial relationships, beyond detailed-route instructions and category-level object finding. It presents NavA3’s global-to-local framework and a 1.0 million-sample affordance dataset, and reports state-of-the-art long-horizon navigation results in real-world environments.
Introduction
- NavA3’s global policy uses Reasoning-VLM to parse instructions and identify likely goal areas, while its local policy explores waypoints and localizes the target object.At each waypoint, NaviAfford identifies the target; if detected, its location is transformed into robot coordinates for navigation to the goal.Source: Introduction · p. 2
- NavA3 achieves state-of-the-art long-horizon navigation performance in large-scale real-world environments and exhibits cross-embodiment capabilities.The paper also reports state-of-the-art navigation performance compared with existing methods.Source: Introduction · p. 2
- The proposed task requires agents to interpret high-level human instructions and locate open-vocabulary objects with complex spatial relationships in intricate indoor environments.This contrasts with VLN’s detailed step-by-step directives and ObjectNav’s success upon finding any instance of a predefined category.Source: Introduction · p. 2
- The authors collected 1.0 million spatial-aware object-affordance samples to train NaviAfford for understanding complex spatial relationships and accurate object pointing.Examples include locating a cup by a window or empty space on a table’s left side.Source: Introduction · p. 2
Related Work
Related work covers visual-language navigation and object navigation, alongside methods for improving VLM spatial reasoning. The paper identifies remaining gaps in high-level command understanding, open-vocabulary spatial localization, and integration with long-horizon navigation.
Related Work
- VLN follows detailed instructions, while ObjectNav seeks any instance of a predefined object category, regardless of spatial context or specific requirements.The paper notes that detailed directives are seldom provided by humans in natural settings.Source: Related Work · p. 2
- Existing navigation approaches are described as lacking high-level command understanding and spatial-aware localization of open-vocabulary objects for long-horizon tasks.Source: Related Work · p. 3
- Spatial reasoning methods use object-centered point clouds, scene graphs, synthetic datasets, RGB-D data, and optimized reasoning, but the paper identifies gaps in open-vocabulary pointing and long-horizon integration.Source: Related Work · p. 3
Methodology
NavA3 frames long-horizon navigation as finding a specific object that satisfies an instruction’s semantic and spatial requirements. Its global-to-local policies combine scene-level reasoning with waypoint-based object localization.
Preliminaries
- The task requires inferring a specific target from a high-level instruction and reaching it within 1 meter while maintaining line of sight.Unlike ObjectNav, it requires multi-step reasoning about the target and its likely location.Source: Preliminaries · p. 3
- RGB images are reconstructed into 3D scenes and top-down maps; annotated regions support global reasoning, while the local policy uses an unannotated map.The global scene pairs geometric regions with semantic annotations such as room names.Source: Preliminaries · pp. 3–4
Global Policy
- Reasoning-VLM interprets instructions with the annotated global scene, infers a target object and likely region, then samples a waypoint within that region.If the target is not found, the policy can continue local exploration or select another promising room using exploration history.Source: Global Policy · p. 4, Local Policy · p. 5
Local Policy
- NaviAfford is trained on about 50K images and 1.0M question-answer pairs, using object and spatial affordances to localize targets and navigation spaces.The annotations encode directional relations between objects and identify free spaces that satisfy spatial constraints.Source: Local Policy · pp. 4–5
- At each waypoint, panoramic RGB views are used to locate the target; predicted points are averaged, transformed into robot coordinates, and used for navigation.When no target is detected, Reasoning-VLM chooses whether to keep exploring the area or move to another room.Source: Local Policy · p. 5
Experiments
NavA3 outperforms the reported baselines across the evaluated scenarios, and its ablations examine annotation, reasoning-model, and object-localization choices. Qualitative examples show affordance understanding and navigation across robot embodiments.
Experimental Details
- The benchmark spans five scenes and 50 tasks, and reports NE and SR, with success defined as reaching within 1 meter.Baselines receive explicit task instructions and top-down global 3D scene information.Source: Experimental Details · pp. 5–6
Comparisons with SOTA Methods
- 41.2 percentage points: NavA3 averages 66.4% SR versus MapNav’s 25.2%, outperforming SOTA methods across all evaluation scenarios.The reported result also includes lower navigation error in every scenario.Source: Table 1 · p. 6, Comparisons with SOTA Methods · p. 6
Ablation Study
- Full annotation yields 68.0% average SR versus 36.0% without map annotation, with improvements in both the tea room and workstation.The reported average improvement is 32.0%.Source: Table 2 · p. 6, Ablation Study · p. 7
- GPT-4o-based Reasoning-VLM reaches 68.0% SR, while Qwen2.5-VL-7B declines by 28.0% (Bai et al. 2025).The compared scenarios are the tea room and workstation.Source: Ablation Study · p. 7
- NaviAfford reaches 70.8% average affordance accuracy versus RoboPoint’s 55.9%, and 68.0% SR versus 57.5%.The authors report these results as a 13.0% affordance-accuracy improvement and a 10.5% navigation success-rate increase over RoboPoint.Source: Table 4 · p. 7, Ablation Study · p. 7
Qualitative Analysis
- Qualitative examples show spatial-affordance references localized amid clutter, multi-room task reasoning, and quadruped navigation.The examples include identifying objects by spatial relations and navigating to a specified fruit.Source: Figure 5 · p. 7, Qualitative Analysis · p. 8
Conclusion
NavA3 combines global instruction parsing and target identification with local object localization. The authors report strong navigation results and deployment on wheeled and quadruped robots, while identifying dynamic environments and additional sensing as future work.
Conclusion
- NavA3 divides navigation into a global Reasoning-VLM policy for instruction parsing and target identification and a local NaviAfford strategy for precise object localization.Source: Conclusion · p. 8
- The authors report that NavA3 outperforms state-of-the-art methods on complex spatial relations and open-vocabulary pointing, with deployments on wheeled and quadruped robots.Source: Conclusion · p. 8
- Future work will focus on improving adaptability in dynamic environments and integrating additional sensory inputs.Source: Conclusion · p. 8