Source-linked AI summary
Reliability-Aware Sparse Route Memory for Round-Trip Vision-Language Navigation
Bojun Long, Lingfan Bao, Tianhu Peng, Jingcheng Sun, Chengxu Zhou
TL;DR
Continuous round-trip VLN makes the Return leg harder than standalone traversal, despite the availability of the executed Outbound route as a spatial prior. The paper records that route as sparse geometric memory and uses it through hints, action arbitration, and terminal verification. With exact route information, arbitration and verification raise Return success to 86.0%, while online matching retains a significant improvement over baseline.
Problem
Continuous Return is harder than standalone traversal, and the executed Outbound route’s potential influence on the navigation policy remains an open question.
Source: §I · p. 1Method
The system records the executed Outbound trajectory as sparse geometric memory and uses it for route hints, action arbitration, and terminal verification while retaining NaVILA as the primary controller.
Source: §I · p. 1Results
Under exact route information, terminal verification raises success from 70.5% to 86.0%, while the complete online system significantly outperforms the geometric alternative controller.
Source: B. Oracle Analysis: Information Versus… · p. 5, §VI · p. 7Takeaways & Limitations
Exact route information alone is insufficient for reliable Return behavior; the Oracle comparisons show that behavioral consistency also constrains success.
Source: §I · p. 1, §VI · p. 7Evaluation uses one base policy and hint template, approximately 40 paired Return episodes per contrast, simulation, and a single robot platform.
Source: §VI · p. 7
Abstract
from arXiv · showhide
Vision-language navigation (VLN) is typically evaluated as a one-way task, although deployed robots may need to return after reaching a goal. We study continuous round-trip VLN and diagnose failures in directional observability, deviation recovery, and termination stability. We propose a reliability-aware sparse route memory that records the executed Outbound trajectory as ordered geometric anchors and queries them in reverse through a structured hint, action-level arbitration, and terminal verification. On 50 reverse-paired episodes using NaVILA and a simulated Unitree Go2, language-only Return succeeds in 22.0% of episodes, while our online system reaches 55.1%. With exact route information, the same interfaces achieve 86.0%, showing that effective Return requires both accurate information and consistent action on that information. The remaining online gap arises mainly from geometric evidence that is too unreliable to authorise intervention. These results distinguish information quality, behavioural consistency, and online reliability as separate limits in long-horizon navigation.
I. INTRODUCTION
Continuous Return VLN is harder than the corresponding standalone traversal, and the executed Outbound route offers a spatial prior for the return journey. This paper tests how that prior should influence navigation while distinguishing information quality from policy behavior.
I. INTRODUCTION
- 40% standalone success falls to 22.0% when the same route is executed as the Return leg of a continuous round trip.The initial Return state comes from Outbound execution, errors accumulate over the longer horizon, and the route appears asymmetric in reverse.Source: §I · p. 1
- The just-executed route provides a Return spatial prior without a separate mapping or teaching run, but how it should influence the policy remains an open question.The route is available from Outbound execution and contains spatial information needed for Return.Source: §I · p. 1
- The Oracle condition holds route memory and interfaces fixed while removing localisation error, separating reference quality from the policy’s consistency in acting on it.The online experiment measures the reliability gap to this information-quality ceiling.Source: §I · p. 1
- The paper organizes Return failures as directional observability, deviation recovery, and termination stability, addressed respectively by a route hint, action arbitration, and terminal verification.NaVILA remains the primary controller.Source: §I · p. 1
II. RELATED WORK
- Unlike recovery methods that train on policy-generated deviations, this work intervenes at execution time without modifying the base policy.Related work also includes route memories and teach-and-repeat systems; this memory is built during a language-directed Outbound journey and queried immediately in reverse, while semantic decisions remain with the VLN policy.Source: §II · p. 2
- The approach connects relocalisation reliability with runtime intervention by using reliability estimates to authorize distinct linguistic, action, and terminal interfaces.The cited related work also treats VLN termination as a failure source and runtime filters as constraints on learned actions.Source: §II · p. 2
III. PROBLEM FORMULATION AND METHOD
The protocol pairs approximately reversed paths from the same scene and executes Outbound and Return continuously, without teleportation or a reset. It separately measures arrival and valid termination so termination failures can be distinguished from trajectory failures.
A. Continuous Round-Trip VLN
- Episodes pair paths that approximately reverse one another within the same scene.Each one-way episode is represented by its scene, reference path, and human instruction.Source: A. Continuous Round-Trip VLN · p. 2
- Return follows the paired reverse instruction toward the original start after Outbound execution and a deterministic 360° scan.The robot starts Return from Outbound’s actual terminal state; no teleportation or environment reset occurs.Source: A. Continuous Round-Trip VLN · p. 2
- Using an existing reverse instruction avoids the confound introduced by automatically generated language.Source: A. Continuous Round-Trip VLN · p. 2
- Return success requires the final position to lie within the 3.0 m success radius.The passage defines this radius as τ = 3.0 m.Source: A. Continuous Round-Trip VLN · p. 2
- Success requires both arrival and a valid STOP, while the termination deficit counts episodes inside the start region that fail to terminate successfully.This measure separates termination failures from failures of the Return trajectory.Source: A. Continuous Round-Trip VLN · p. 2
- The termination deficit distinguishes reaching the start region from successfully completing the Return episode.An episode can finish inside the start region without a valid STOP.Source: A. Continuous Round-Trip VLN · p. 2
B. Sparse Route Memory
Sparse route memory records the path actually executed during Outbound and reuses its anchors in reverse during Return.
B. Sparse Route Memory
- A background module creates a route anchor whenever Outbound travel exceeds the distance threshold δa.NaVILA continues following the original instruction while the memory is created.Source: B. Sparse Route Memory · p. 2
- Each anchor stores local 2-D LiDAR geometry, a topological index, cumulative route distance, and the relative transform to its neighboring anchor.The memory represents the executed trajectory rather than a global map.Source: B. Sparse Route Memory · p. 2
- After Outbound succeeds, the memory is frozen and queried in reverse anchor order, from AK to A0.Source: B. Sparse Route Memory · p. 2
C. Reliability-Aware Return Navigation
During Return, route registration estimates progress through the recorded anchors, while reliability gates govern updates and interventions. The system supplies route hints, selectively arbitrates actions, and verifies termination while retaining NaVILA as the primary controller.
C. Reliability-Aware Return Navigation
- Route tracking registers current LiDAR geometry against nearby reverse anchors, expanding to a bounded recovery set when normal tracking fails.Route-state updates require topology, multi-frame consistency, and sufficiently unambiguous candidates; otherwise the previous state is retained.Source: C. Reliability-Aware Return Navigation · p. 2
- Three calibrated reliability components estimate whether the relevant geometric signals lie within tolerance, and a joint gate authorizes route-state updates and action overrides.The deployed thresholds are rbearing ≥0.68, rd istance ≥0.74, and rpose ≥0.70; hint and arbitration also use a coarser relocalisation-confidence threshold.Source: C. Reliability-Aware Return Navigation · p. 2
- When bearing reliability is sufficient, the structured hint reports the target anchor, bearing, distance, and remaining route progress; otherwise it is withheld.The hint supplies information without constraining NaVILA’s output.Source: C. Reliability-Aware Return Navigation · p. 2
- Outbound execution records ordered geometric anchors; during Return, registration and reliability gates support the hint and action arbiter while NaVILA remains the primary controller.The system also uses a learned reliability guard on route-state updates and overrides.Source: Figure 1 · p. 3
- Action arbitration overrides a conflicting proposal only when bearing reliability is sufficient and occupancy confirms the hinted direction is traversable.It is disabled within 0.35 m of the target anchor and acts as a conservative correction rather than a local planner.Source: C. Reliability-Aware Return Navigation · p. 3
- Terminal verification accepts supported STOP requests, vetoes contradictory ones, defers when evidence is insufficient, and forces STOP after consistent evidence of arrival.Vetoes and deferrals can continue navigation; forced termination can convert arrival without STOP into success.Source: C. Reliability-Aware Return Navigation · p. 3
IV. EXPERIMENTAL SETUP
The experiments compare language-only, Oracle, and online Return on reverse-paired routes using a simulated Go2 and NaVILA. The online system reuses the executed Outbound route, while Oracle conditions separate reference quality from policy behavior.
IV. EXPERIMENTAL SETUP
- Reverse pairs use ordered nearest-neighbour matching with a 2.0 m waypoint tolerance and 0.8 coverage; both phases require a valid STOP within 3.0 m.Return success measures distance to the Outbound start, and paired tests use episodes succeeding Outbound under both configurations.Source: §IV · p. 3
- M50 contains 50 reverse-paired episodes selected from 264 candidates for high historical Outbound reliability, so absolute end-to-end success is optimistic.Selection was not based on Return outcome; conditional and within-episode paired contrasts remain the focus.Source: §IV · p. 3
- The 38-episode pilot checks the reproduced one-way baseline, while the standalone reference runs reverse routes independently without preceding Outbound execution or route memory.Standalone runs use the original human instruction and the same checkpoint, action budget, and valid-STOP requirement.Source: §IV · p. 3
- Five configurations share the policy, episodes, instructions, memory, and success criterion, varying memory access and the three Return interfaces.The online configuration uses 2-D ICP and reliability gating; full Oracle uses exact route information.Source: §IV · p. 4
- The geometry-only controller replaces Return VLM queries, hints, and arbitration with a deterministic reverse-polyline follower, retaining terminal verification.Stuck recovery is enabled only for the online system during Return and does not update relocalisation, reliability, or STOP state.Source: §IV · p. 4
- Anchor spacing, ICP settings, reliability thresholds, and arbitration thresholds were fixed before M50 evaluation.The hint and arbiter require relocalisation confidence above 0.90; route-state and override guards use separate learned reliability thresholds.Source: §IV · p. 4
A. Continuous Return Introduces an Execution Deficit
On the same routes, Return success falls from standalone one-way performance, and matched-route results show the deficit persists across scenes. The evidence locates the additional burden in continuous execution without isolating a single internal cause.
A. Continuous Return Introduces an Execution Deficit
- 40.0% of standalone one-way runs succeed on M50 routes, versus 22.0% when those routes serve as the continuous Return leg.These are 20/50 and 11/50 successes, respectively.Source: A. Continuous Return Introduces an… · p. 4
- The pilot pool records 17/38 Outbound successes (44.7%), close to the reported 50.2% Go2-Vision result, and 5/17 then succeed Return (29.4%).The authors describe the Return result as consistent with the 22.0% M50 baseline and the reproduction as not grossly losing one-way capability.Source: A. Continuous Return Introduces an… · p. 4
- Trajectory logs show oscillating turns, sustained directional deviation, stagnation, and incorrect termination, which continuous Return gives more opportunity to trigger and accumulate.These behaviors also occur occasionally during Outbound.Source: A. Continuous Return Introduces an… · p. 4
- On one reverse pair, a route succeeds independently but fails as Return even after resetting position and heading to the expert endpoint.The independent run stops 0.317 m from its goal; the Return run stops 11.351 m from the Outbound start.Source: A. Continuous Return Introduces an… · p. 4
- 45.8% of standalone runs succeed on 24 geometrically matched routes, versus 12.5% when embedded as Return across nine scenes (p = 0.021).Nine routes change from standalone success to Return failure, while one changes in the opposite direction.Source: A. Continuous Return Introduces an… · p. 4
- The matched-route comparison does not isolate a single cause; phase transition, accumulated visual context, and a longer decision chain remain plausible contributors.The evidence argues against route executability or automatically generated language alone explaining the additional burden.Source: A. Continuous Return Introduces an… · p. 4
B. Oracle Analysis: Information Versus Behaviour
Oracle conditions show that accurate route information alone leaves substantial Return failure, while arbitration and terminal verification add gains through distinct interfaces. The results motivate reliability-aware authorisation when those interfaces operate online.
B. Oracle Analysis: Information Versus Behaviour
- 37.2% succeed with exact bearing and distance supplied only through the hint, up from 22.0%; on 43 commonly entering episodes, the paired change from 23.3% to 37.2% is not significant (p = 0.180).Almost two-thirds of trials still fail after localisation error is removed.Source: B. Oracle Analysis: Information Versus… · p. 4
- Return trajectory deviation between baseline and Hint runs rises from 0.10 m during Outbound to 1.08 m during Return (p = 2.4 × 10^-8), but this does not establish correct recovery of directional semantics.The analysis cannot distinguish inconsistent execution after correct interpretation from a response to an unfamiliar prompt format.Source: B. Oracle Analysis: Information Versus… · p. 5
- 70.5% succeed with arbitration added to the fixed exact reference and hint; the paired Hint-to-Hint-Action comparison is 32.4% versus 64.9% (p = 0.008).Of 1,604 Return steps, 32.9% contain a conflicting action; traversability permits 277 overrides, while 250 conflicts are declined.Source: B. Oracle Analysis: Information Versus… · p. 5
- Terminal verification raises success from 70.5% to 86.0%, combining more arrivals with fewer unsupported terminations after arrival.Arrival rises from 81.8% to 90.7%; the termination deficit falls from 11.4 to 4.7 points. The adjacent paired comparison is not significant (p = 0.125).Source: B. Oracle Analysis: Information Versus… · p. 5
- The Oracle ladder indicates that perfect spatial information alone is insufficient; arbitration lets the reference affect policy outputs, and terminal verification addresses a separate failure dimension.These interfaces can also impose an incorrect reference repeatedly, motivating reliability-aware authorisation online.Source: B. Oracle Analysis: Information Versus… · p. 5
C. Online Reliability Deficit
Online route memory improves Return over language-only navigation, but remains well below the exact-information Oracle. The gap is driven mainly by unreliable geometric evidence that withholds intervention and by terminal decisions that can stop before arrival.
C. Online Reliability Deficit
- 55.1% online Return success trails full Oracle at 85.7% on common Outbound successes, while improving over language-only Return at 22.4%.On 49 common episodes, online raises success from 22.4% to 55.1% (p = 0.0015); on 42 common-Outbound-success episodes, online and full Oracle reach 59.5% and 85.7%, respectively.Source: C. Online Reliability Deficit · p. 5
- Online failures end a median 5.59 m from s0, while exact-information failures primarily reflect unstable arrival judgement rather than failure to arrive.Among 22 online Return failures, the maximum distance is 17.91 m and three runs end beyond 10 m; Fig. 4 distinguishes arrivals without valid termination from failures to arrive using the 3.0 m radius.Source: C. Online Reliability Deficit · p. 5, Figure 4 · p. 6
- At 979 of 1,579 Return decisions, low relocalisation confidence withholds both the hint and arbiter, restricting their use across most of the trajectory.The confidence threshold is 0.90; median per-episode override rate falls from 12.6% under Oracle to 6.9% online.Source: C. Online Reliability Deficit · p. 5
- In 18 of 22 failed online Return phases, an accepted STOP ends execution outside the true 3.0 m success region; only four failures are timeouts.The online deficit combines directional abstention that prevents arrival with over-acceptance by the terminal interface.Source: C. Online Reliability Deficit · p. 6, Figure 4 · p. 6
- Among 492 authorised, non-skipped bearing decisions, 52.0% conflict with the online hint, versus 35.1% under Oracle, so always enforcing online bearings risks sustained incorrect corrections.The comparison does not establish that the base policy degrades online, since disagreement may also reflect an incorrect estimate.Source: C. Online Reliability Deficit · p. 6
D. Alternative Controller: Geometry-Only Route Following
The deterministic geometry-only follower does not match the complete online hybrid system on Return success. The comparison supports the hybrid configuration as a whole, not an isolated causal contribution from the VLM.
D. Alternative Controller: Geometry-Only Route Following
- 54.3% online success exceeds geometry-only control at 23.9% on the 46 episodes with common Outbound success.The paired advantage is about 30 percentage points (p = 0.009); geometry-only succeeds on 11/46 valid episodes.Source: D. Alternative Controller:… · p. 6
- Geometry-only success of 23.9% does not differ significantly from language-only success of 17.4% on common-Outbound-success episodes (p = 0.648).Eleven episodes change from language-only failure to geometry-only success, while eight change in the opposite direction.Source: D. Alternative Controller:… · p. 6
- The comparison cannot isolate the VLM’s causal contribution because the geometry-only configuration also disables stuck recovery.Stuck recovery fired in only 2 of 27 successful online round trips, so its removal is unlikely to explain the performance gap.Source: D. Alternative Controller:… · p. 7
VI. DISCUSSION AND CONCLUSION
The discussion separates limits from continuous execution, information quality, behavioural consistency, and online reliability. Exact route information supports 86.0% Return success, whereas online performance is constrained by whether geometric evidence can reliably authorise intervention.
VI. DISCUSSION AND CONCLUSION
- Under exact information, selective arbitration and terminal verification raise Return success to 86.0%, above the 40% standalone one-way rate.Arbitration raises success from 22.0% to 70.5%, and terminal verification adds a further increase from 70.5% to 86.0%.Source: §VI · p. 7
- Absolute performance is qualified by one base policy and hint template, a historically selected M50 set, non-zero endpoint offsets, and dead-reckoned route state.The reliability components use a conservative joint gate; the study does not isolate per-operation specialisation against shared confidence or a no-gate control.Source: §VI · p. 7
- The evidence comes from simulation and a single robot platform, and roughly 40 paired Return episodes per contrast leave the smaller terminal increment underpowered.Future work should test across policies and on a physical mobile robot.Source: §VI · p. 7
- Language-only Return and deterministic geometric replay achieve conditional success rates of 22.0% and 23.9%, so neither route following mode succeeds reliably in isolation.The complete online system significantly outperforms the geometric alternative, but that comparison is end-to-end rather than an isolated measure of the semantic policy.Source: §VI · p. 7
- The online system loses 26.2 paired points to Oracle principally because bearing uncertainty withholds intervention.Return does not reach 100% under Oracle: arbitration corrects only about one decision in six and abstains elsewhere, while arrival without a supported STOP still fails.Source: §VI · p. 7