Source-linked AI summary
NavProbe: Evidence-Grounded Reasoning with Active Memory Retrieval for Zero-Shot Navigation
Jingyang Liu, Sujia Yao, Jiayuan Gu, Lan Xu
TL;DR
Long-horizon navigation requires revising intermediate objectives as evidence accumulates, but full visual histories are costly and compact summaries may omit details. NavProbe couples a dynamic subgoal agenda with active retrieval from graph-structured memory. It reports strong results on VLN and ObjectNav benchmarks, including higher R2R-CE SR and SPL than the matched Uni-LaViRA rerun and the highest listed HM3D-v2 ObjectNav SR.
Problem
Full visual histories accumulate redundant or irrelevant observations, while compact memories may discard details needed to reconsider earlier navigation decisions.
Source: §I · p. 1Method
NavProbe combines a dynamic subgoal agenda with graph-structured episodic memory, retrieving historical evidence to revise objectives and consolidating reusable conclusions into entity knowledge.
Source: §I · p. 1Results
On R2R-CE, NavProbe achieves 71.7% SR and 55.8% SPL, exceeding the Uni-LaViRA rerun’s 61.7% SR and 45.0% SPL; on HM3D-v2 ObjectNav, it achieves 79.3% SR, the highest listed value.
Source: B. Main Results · p. 5Takeaways & Limitations
The evaluated results and ablations support NavProbe’s end-to-end performance and the contributions of task-progress memory, evidence retrieval, and knowledge consolidation.
Source: §V · p. 7
Abstract
from arXiv · showhide
Long-horizon navigation requires an agent to revise its intermediate objectives as evidence accumulates. Full visual histories are costly to process, while compact summaries may omit details needed to reconsider earlier decisions. We introduce NavProbe, a hierarchical zero-shot navigation agent that couples a dynamic subgoal agenda with active evidence retrieval. A compact index links summaries of visited places, transitions, and landmarks to their visual and geometric records. When the current context is insufficient, a task executive retrieves targeted evidence to generate, revise, or resolve subgoals. Reusable conclusions are used to update the index, and a skill policy converts the revised task state into parameterized navigation actions. NavProbe achieves 71.7% SR and 55.8% SPL on R2R-CE and 55.3% SR and 38.6% SPL on RxR-CE, outperforming strong zero-shot baselines. It also achieves 79.3% SR on HM3D-v2 ObjectNav, with qualitative real-robot demonstrations illustrating physical deployment. Code is available at https://github.com/liujy25/NavProbe.
I. INTRODUCTION
NavProbe addresses the need to revise navigation objectives using evidence that may no longer be visible, without retaining the full visual history or relying only on compact summaries. It couples a dynamic subgoal agenda with graph-structured episodic memory and active retrieval, and reports strong zero-shot VLN results alongside evaluation beyond VLN.
I. INTRODUCTION
- Compact summaries can omit visual details or preserve earlier interpretations, while full visual histories accumulate redundant or irrelevant observations that can distract VLM reasoning.Selective retrieval is presented as a middle ground for exposing historical observations when relevant.Source: §I · p. 1
- NavProbe couples a dynamic subgoal agenda with graph-structured episodic memory of visited places, transitions, and landmarks, indexed by compact entity summaries linked to visual and geometric records.Instruction clauses initialize the VLN agenda; ObjectNav search objectives are constructed online.Source: §I · p. 1
- The task executive uses targeted historical retrieval to generate, revise, or resolve subgoals and consolidates reusable conclusions, while the skill policy selects parameterized navigation actions.Execution extends episodic memory for subsequent decisions.Source: §I · p. 1
- NavProbe achieves the highest SR among compared zero-shot methods on R2R-CE and RxR-CE, while ObjectNav and real-robot experiments extend evaluation beyond the main VLN setting.Ablations and retrieval analysis examine task-state memory, active retrieval, and knowledge consolidation.Source: §I · p. 1
II. RELATED WORK
Prior zero-shot VLN work addresses instruction reasoning, progress tracking, and recovery through varied mechanisms. Navigation-memory work spans stored spatial and visual records, selective retrieval, and compact indexes; NavProbe combines an updatable entity index with access to underlying evidence.
A. Zero-Shot VLN and Navigation Recovery
- Zero-shot VLN methods use pretrained VLMs for high-level reasoning, with approaches including language-based history, multiple model-based experts, and multi-source value maps.Open-Nav, CA-Nav, and GC-VLN explicitly model instruction progress or constraints,,.Source: A. Zero-Shot VLN and Navigation Recovery · pp. 1–2
- Prior recovery methods use progress estimates, trajectory search, backtracking, or failed sub-trajectories; NavProbe instead focuses on representing and selectively accessing historical evidence for decisions.Examples include SmartWay’s last-state backtracking, GC-VLN’s constraint-tree backtracking, and Uni-LaViRA’s Second Chance Backtrack,, [15].Source: A. Zero-Shot VLN and Navigation Recovery · p. 2
B. Navigation Memory and Evidence Retrieval
- Navigation memories preserve different combinations of spatial, visual, temporal, and pose information, including waypoint and landmark memories and records linked to spatial or temporal context [16],, [17].Source: B. Navigation Memory and Evidence… · p. 2
- Memory systems vary in how they select evidence for reasoning, using on-demand recall, relevance and recency, task-conditioned aggregation, or multimodal retrieval [9], [18],, [20].NavProbe indexes multimodal records for places, transitions, and landmarks with updatable entity knowledge; retrieved conclusions can be consolidated for later decisions.Source: B. Navigation Memory and Evidence… · p. 2
III. METHOD
NavProbe separates subgoal management and evidence retrieval from action selection, using a repeated reasoning cycle before each navigation action. Its memory distinguishes current task state, persistent episodic records, and a compact index, while the agenda can change as evidence accumulates.
A. NavProbe Overview
- The task executive reads the current observation, task state, and memory index, retrieves historical records when needed, revises the agenda, and passes updated state and conclusions to the skill policy for action selection.Execution adds observations and trajectories to episodic memory, and consolidation preserves reusable conclusions in the index.Source: A. NavProbe Overview · p. 2
B. Hierarchical Navigation Memory
- NavProbe distinguishes dynamic task state, persistent episodic records, and compact entity knowledge that indexes those records; historical images enter the executive’s context only when explicitly retrieved.Source: B. Hierarchical Navigation Memory · p. 2
1) Task State and Dynamic Subgoal Management:
- The task state is an agenda plus execution history, and its size and contents can change as subgoals are introduced, refined, reprioritized, or removed when satisfied or no longer useful.Agenda updates remain grounded in the original goal and any instruction ordering constraints.Source: 1) Task State and Dynamic Subgoal… · p. 2
- VLN begins with instruction clauses as subgoals, whereas ObjectNav constructs intermediate search objectives online; completing a search objective does not necessarily complete the overall task.For example, inspecting a countertop may establish that the target is absent and motivate searching elsewhere.Source: 1) Task State and Dynamic Subgoal… · p. 2
- Verification predicates track whether evidence-checkable propositions are confirmed, and contradictory evidence can return a previously resolved objective to the agenda.Subgoal resolution is assessed against the full objective and available evidence.Source: 1) Task State and Dynamic Subgoal… · pp. 2–3
2) Graph-Structured Episodic Records:
NavProbe organizes navigation history as a topological graph of visited places and executed transitions, with associated landmark entities and separate spatial records.
2) Graph-Structured Episodic Records:
- Place nodes retain panoramic observations, transition edges record executed trajectories, and landmark records store detections and estimated locations as evidence behind earlier observations and actions.A separate occupancy map records free, occupied, and unexplored regions for spatial reasoning and waypoint generation; entity knowledge is linked to episodic records and revised through consolidation.Source: 2) Graph-Structured Episodic Records · p. 3
3) Entity Knowledge and the Compressed Index:
Entity summaries provide a compact interpretation of navigation history while retaining pointers to visual and geometric evidence. They complement task state, which tracks the current agenda and its execution history.
3) Entity Knowledge and the Compressed Index:
- Entity knowledge stores reusable summaries of places, transitions, and landmarks, while task state records the current agenda and its execution history.The index points to retrievable fields and their record references, allowing the executive to check the compact interpretation against evidence.Source: 3) Entity Knowledge and the Compressed… · p. 3
1) Formulating Subgoals and Probing Memory:
The task executive probes memory when an unresolved task-state question needs historical evidence. Evidence conclusions can revise the agenda, and the updated state is retained for later navigation decisions.
1) Formulating Subgoals and Probing Memory:
- The executive assesses which objectives to pursue, revise, or resolve and formulates a query when a task-state decision needs more evidence.Questions can concern subgoal completion, search-region relevance, or whether an earlier observation supports a new objective.Source: 1) Formulating Subgoals and Probing… · p. 3
- The index selects relevant entities and evidence fields, while the query specifies what to inspect and how to interpret it.The selected references return historical images and geometric records; retrieval is skipped when the initial context suffices.Source: 1) Formulating Subgoals and Probing… · pp. 3–4
- Evidence conclusions can generate, revise, or resolve subgoals and update verification predicates and execution history.A turn before the required landmark can reopen a VLN objective; a panorama showing kitchen fixtures may motivate revisiting an overlooked doorway in ObjectNav.Source: 2) Evidence Verification and… · p. 4
- Retrieved records remain in the active context only during inspection, while later reasoning uses their conclusions and references.A cumulative bird’s-eye-view representation preserves spatial context for reasoning and action selection.Source: 2) Evidence Verification and… · p. 4
- The retrieval loop ends when no further evidence is requested or the per-step budget is reached, with the final batch interpreted before state updates are finalized.The executive retains an updated task state and assessment for subsequent navigation, even if questions remain unresolved at the budget limit.Source: 2) Evidence Verification and… · p. 4
3) Memory Consolidation:
Reusable evidence conclusions are consolidated into entity summaries, and the skill policy maps the executive’s revised output to parameterized navigation actions.
3) Memory Consolidation:
- The Entity Knowledge Manager consolidates query–reference–conclusion records with existing entity knowledge, adding, revising, or removing claims without reopening retrieved images.Updated summaries enter the index for reuse in later agenda management.Source: 3) Memory Consolidation · p. 4
D. Act: Skill Selection and Geometric Execution
- The skill policy maps the current observation and executive output to a parameterized action.The output includes the updated task state, assessment, evidence conclusions, and available spatial context.Source: D. Act: Skill Selection and Geometric… · p. 4
- Action types include GoToWaypoint, VerticalMove, Backtrack, and Stop, with arguments such as a waypoint label or visited node.Geometric routines translate selected actions into low-level motion.Source: D. Act: Skill Selection and Geometric… · p. 4
1) Waypoint Grounding:
Frontier–Skeleton Sampling grounds navigation skills in visible waypoints, while backtracking, termination, and new observations support continued navigation cycles. The evaluation setup specifies the tested benchmarks and implementation conditions.
1) Waypoint Grounding:
- Frontier–Skeleton Sampling places anchors at explored–unexplored boundaries and navigable-space medial axes, then selects visible waypoints along collision-free paths subject to distance and occlusion constraints.Candidates receive consistent labels in the RGB observation and bird’s-eye view; VerticalMove temporarily treats detected stairs as traversable.Source: 1) Waypoint Grounding · p. 4
2) Recovery, Termination, and Memory Updates:
- Backtrack selects a visited place for route recovery and replanning, while Stop retains pose or performs final local positioning before termination.Executed movements add observations and trajectories to episodic memory for the next navigation cycle.Source: 2) Recovery, Termination, and Memory… · p. 4
A. Experimental Setup
- Evaluation uses R2R-CE and RxR-CE for VLN and HM3D-v2 for ObjectNav, with NavProbe results averaged across three runs on stratified 100-episode subsets.The subsets are not the full benchmark splits.Source: A. Experimental Setup · p. 4
- Table I covers R2R-CE and RxR-CE val-unseen results; bold and underlining mark the best and second-best values within the zero-shot block.The dagger marks the GPT-5.5 Uni-LaViRA rerun on the R2R-CE subset.Source: Table I · p. 5
- Navigation decisions use four-view panoramas, a retrieval budget of six, and limits of 40 decisions and 500 simulator actions per episode.Metric waypoints are executed with Habitat-Lab’s ShortestPathFollower, while waypoint selection remains NavProbe’s responsibility.Source: A. Experimental Setup · p. 5
B. Main Results
NavProbe outperforms the compared zero-shot baseline on both VLN benchmarks and achieves the highest listed success rate on HM3D-v2 ObjectNav. Ablations support contributions from explicit task-state memory, retrieval, consolidation, and selective historical context.
B. Main Results
- 71.7% SR/55.8% SPL on R2R-CE and 55.3% SR/38.6% SPL on RxR-CE; NavProbe exceeds Uni-LaViRA on both.On R2R-CE, it exceeds the GPT-5.5 Uni-LaViRA rerun by 10.0 SR and 10.8 SPL points.Source: B. Main Results · p. 5
- 79.3% SR on HM3D-v2 is the highest listed, 1.6 points above Uni-LaViRA; NavProbe's 43.8% SPL is below Uni-LaViRA's 46.1%.The success-rate advantage does not extend to SPL.Source: Table II · p. 5, B. Main Results · p. 5
C. Ablation Studies
- Without task-state memory, SR falls 71.7%→63.3%, SPL 55.8%→51.4%, and NE rises 3.45→3.94 m.The ablation removes structured task-state carryover, including subgoal statuses, verification predicates, and execution history.Source: Table III · p. 6, C. Ablation Studies · p. 6
- Without EKM, SR falls 71.7%→65.0% and SPL 55.8%→51.1%, while NE rises 3.45→4.20 m.Retrieval remains enabled; the comparison tests consolidation of retrieval-derived conclusions into persistent entity knowledge.Source: Table III · p. 6, C. Ablation Studies · p. 6
- With EKM off in both variants, retrieval raises SR 57.3%→65.0% and SPL 43.1%→51.1%, and lowers NE 4.65→4.20 m.This comparison isolates access to visual and geometric records beyond compact entity knowledge alone.Source: Table III · p. 6, C. Ablation Studies · p. 6
- Full Graph scores 62.0% SR/48.8% SPL versus NavProbe's 71.7%/55.8%, favoring query-directed selection over this full-graph context configuration.Full Graph replaces selective entity–field retrieval with full-graph history in the reasoning context.Source: Table III · p. 6, C. Ablation Studies · p. 6
D. Retrieval Analysis
Retrieval is used selectively during navigation, and its queries frequently concern prior actions and route relations. Query annotations also show that many queries address multiple semantic dimensions.
D. Retrieval Analysis
- Retrieval runs at 46.71% of 4,554 decisions, or 53.79% after excluding 600 history-free first decisions; retrieval calls average 1.82 rounds.Statistics pool 600 executions across the two VLN-CE benchmarks, including successful and unsuccessful episodes.Source: D. Retrieval Analysis · pp. 6–7
- Figure 3 pairs key execution stages with onboard observations for a multi-stage instruction on the Galaxea R1Pro.The deployment uses Voxel-SLAM for pose estimation and Nav2 to execute NavProbe's metric goals.Source: Figure 3 · p. 7
- Among 3,862 queries, action execution appears in 68.44%, spatial/route relation in 55.72%, and landmark identification in 29.05%; 48.24% cover at least two dimensions.The semantic labels are non-exclusive, and the annotations characterize what the task executive asks to check.Source: Table IV · p. 7, D. Retrieval Analysis · p. 7
E. Qualitative Results and Real-World Demonstrations
NavProbe’s qualitative examples show retrieval prompting task reassessment in VLN and ObjectNav, and its real-world deployment executes multi-stage instructions. The conclusion frames these results as evidence for coupling compact memory with on-demand access to historical evidence.
E. Qualitative Results and Real-World Demonstrations
- Retrieved transitions leave dining-room entry unconfirmed, prompting VLN route reassessment; retrieved landmarks invalidate an accepted TV detection, reopening the ObjectNav search.The agent subsequently reaches the dining room and stops near the white chair, and finds a real TV before terminating successfully.Source: E. Qualitative Results and Real-World… · p. 7
- On a Galaxea R1Pro with a ZED2 RGB-D camera and MID-360 Li-DAR, NavProbe successfully executes multi-stage real-world navigation instructions.Voxel-SLAM provides pose estimation, Nav2 executes metric goals, and subsequent observations inform task-state assessment and replanning.Source: E. Qualitative Results and Real-World… · p. 7
V. CONCLUSION
- Experiments on evaluated R2R-CE and RxR-CE subsets demonstrate strong navigation performance, while ObjectNav results and real-robot trials provide additional evidence of applicability beyond the main VLN-CE evaluation.The conclusion also reports that ablations support task-progress memory, evidence retrieval, and knowledge consolidation; retrieval analysis and a qualitative execution example illustrate historical observations supporting instruction-execution questions.Source: §V · p. 7