PaperlayerView exact revision on arXiv

Source-linked AI summary

Seek Before You Move: Evidence Seeking for Progress Grounding in Vision-Language Navigation

Zhimin Wang, Meiyuan Zhu, Duo Wu, Linjia Kang, Yajun Wang, Yuan Ni, Xiaohang Wang, Tianlu Pan, Jingyan Jiang, Yaowei Wang, Zhi Wang

arXiv:2609.37353v1cs.AI

TL;DR

Long-horizon instructions and partial observations can leave VLN agents without sufficient evidence to ground progress, while existing VLM agents may focus on reasoning from what is already observed. SeekVLN combines semantic progress reasoning with active evidence seeking, trained through FRG and SFT followed by C2PO reinforcement fine-tuning. It reports state-of-the-art results on R2R-CE and RxR-CE, alongside evidence-seeking behavior in simulated and real-world evaluations.

  • Problem

    Long-horizon instructions and partial egocentric observations can leave agents with insufficient evidence to determine progress and the next action.

    Source: §1 · pp. 1–2

  • Method

    SeekVLN combines semantic progress reasoning with active evidence acquisition, using FRG-augmented demonstrations for SFT and C2PO counterfactual comparisons for reinforcement fine-tuning.

    Source: §1 · pp. 2–3, §3.2 · p. 4

  • Results

    12.7% on R2R-CE and 7.5% on RxR-CE are SeekVLN’s success-rate improvements over the base model, achieving state-of-the-art performance on both benchmarks.

    Source: §1 · p. 3

  • Takeaways & Limitations

    Simulated and real-world evaluations show targeted evidence-seeking behavior, and the results highlight learning when to acquire and use evidence for progress grounding under partial observation.

    Source: §5 · p. 10

Abstract

from arXiv · show

Vision-Language Navigation (VLN) requires agents to continuously ground task progress from long-horizon instructions and partial egocentric observations. Existing VLM-based navigation agents typically reason only over available observations and may remain confident even when task-relevant evidence is missing. For example, an agent may confidently proceed forward and get lost even though the landmark indicating the next turn lies outside its current field of view. We term this failure mode Progress Myopia: the agent fails to recognize unreliable progress grounding and continues acting on insufficient evidence. To address it, we propose SeekVLN, an evidence-seeking framework that couples semantic progress reasoning with active acquisition of task-relevant observations. SeekVLN is trained in two stages: First, Future-guided Reverse Generation (FRG) uses future expert actions to augment offline expert trajectories with supplementary views and evidence annotations. Supervised fine-tuning on these trajectories establishes a prior for evidence seeking and progress reasoning without additional expert interaction. However, imitation alone does not reveal whether seeking improves subsequent navigation. We therefore introduce Counterfactual Contrastive Policy Optimization (C2PO) for reinforcement fine-tuning. By comparing each evidence-seeking branch with a counterfactual direct-navigation branch from the same state, C2PO uses a contrastive reward to assign credit to seeking decisions based on subsequent navigation benefit. Experiments on simulated benchmarks show that SeekVLN achieves state-of-the-art performance, improving success rate by 12.7% and 7.5% over the base model on R2R-CE and RxR-CE, respectively. Both simulated and real-world evaluations exhibit human-like evidence-seeking behaviors for more reliable progress grounding.

1 INTRODUCTION

VLN agents must ground progress across long instructions and partial observations, but task-relevant evidence can be insufficient to determine progress or the next action. SeekVLN addresses this gap with active evidence seeking and reports improved success rates on two simulated benchmarks.

1 INTRODUCTION

  • Long-horizon instructions and partial egocentric observations can leave agents without enough evidence to determine completed subgoals or where to go next.Examples include an out-of-view landmark and a fork where the instruction and observation do not indicate the correct direction.Source: §1 · p. 1
  • Existing VLM agents focus on decisions from available observations, while Progress Myopia describes failing to recognize unreliable progress grounding and continuing with insufficient evidence.The paper reports similar decision confidence on failed and successful trajectories for three representative models.Source: §1 · pp. 2–3
  • SeekVLN combines semantic progress reasoning with active acquisition of task-relevant observations, using FRG-augmented demonstrations for SFT and C2PO to compare seeking with direct navigation.FRG uses future expert actions to annotate evidence-seeking trajectories; C2PO assigns credit based on the branches’ subsequent navigation benefit.Source: §1 · pp. 2–3
  • 12.7% on R2R-CE and 7.5% on RxR-CE are SeekVLN’s success-rate improvements over the base model, establishing state-of-the-art performance.The paper also reports human-like evidence-seeking behavior in simulated and real-world evaluations.Source: §1 · p. 3

2 RELATED WORKS

Related work covers VLM-based navigation, progress grounding, and reinforcement learning for VLN. The paper positions SeekVLN as adding active evidence acquisition and matched counterfactual comparisons to these lines of work.

2.1 VLM-BASED VISION-LANGUAGE NAVIGATION

  • VLM-based navigation agents passively reason from available observations, creating an information bottleneck when task-relevant evidence is missing.SeekVLN instead actively seeks additional evidence before navigation.Source: §2.1 · p. 3

2.2 PROGRESS REASONING AND GROUNDING IN EMBODIED NAVIGATION

2.3 REINFORCEMENT LEARNING FOR VISION-LANGUAGE NAVIGATION

  • C2PO addresses evidence-seeking credit assignment by comparing matched seeking and direct-navigation branches and rewarding their difference in short-horizon progress.The paper situates this alongside RL methods for VLM navigation and step-level contrastive rewards.Source: §2.3 · p. 3

3 METHODOLOGY

SeekVLN first decides whether current observations suffice for progress grounding, then acquires task-relevant views when needed. FRG supplies evidence-seeking supervision from expert trajectories, while C2PO uses counterfactual branches to optimize seeking and navigation together.

3.1 EVIDENCE-SEEKING NAVIGATION FRAMEWORK

  • SeekVLN seeks supplementary views when observations are insufficient, reasons over them to estimate progress, and then predicts a navigation action.The SEEK interaction appends left, front, and right views at fixed relative headings.Source: §3.1 · p. 4

3.2 FUTURE-GUIDED REVERSE GENERATION

  • FRG reasons backward from future expert actions to select when and where to seek evidence, while historical and current observations support structured progress annotations.It constructs evidence-seeking supervision without additional expert interaction; selected views are used to identify task-relevant evidence.Source: §3.2 · pp. 4–5

3.3 COUNTERFACTUAL CONTRASTIVE POLICY OPTIMIZATION

  • C2PO combines a decision-level counterfactual reward for short-horizon navigation benefit with an episode-level adaptive outcome reward to jointly optimize seeking and navigation.Counterfactual branches are sampled during reinforcement fine-tuning, with no additional branch rollouts during evaluation.Source: §3.3 · pp. 5–6
  • C2PO compares SEEK and forced-NAV branches from the same state, rolling both out with the same policy for H primitive actions to isolate seeking’s local benefit.The comparison controls for the initial state and context.Source: §3.3 · p. 6

4 EXPERIMENT

SeekVLN is evaluated on R2R-CE and RxR-CE, where its two-stage training establishes state-of-the-art results and its behavioral analysis examines when seeking helps. The experiments also show adaptive triggering and qualitative evidence seeking in simulated and real-world navigation.

4.1 EXPERIMENTAL SETTING

  • Experiments use Habitat with Matterport3D scenes and the val-unseen splits of R2R-CE and RxR-CE, reporting benchmark-specific navigation metrics.SeekVLN is initialized from Aux-Think, trained with SFT on an FRG dataset of 4K R2R-CE episodes, then C2PO-RFT on 640 sampled R2R-CE episodes; navigation uses discrete forward, turn, and stop actions.Source: §4.1 · pp. 6–7

4.2 MAIN RESULTS

  • FRG-SFT already surpasses the base model on both benchmarks, and C2PO-RFT further sets a new state of the art.FRG-SFT raises R2R-CE SR from 54.8 to 61.0 and SPL from 46.9 to 55.9; RxR-CE SR from 52.2 to 55.7 and SPL from 40.2 to 47.4, using offline supervision only.Source: §4.2 · p. 7, Table 1 · p. 7
  • C2PO-RFT improves every reported metric over the SFT checkpoint, with R2R-CE SR rising from 61.0 to 67.5 (+6.5) and SPL from 55.9 to 61.4 (+5.5).On RxR-CE, SR reaches 59.7 (+4.0) and SPL 50.3 (+2.9); the passage attributes credit assignment to whether acquired evidence improves subsequent navigation.Source: §4.2 · p. 7
  • Compared with the base model, full training gains +12.7 SR and +14.5 SPL on R2R-CE, and +7.5 SR and +10.1 SPL on RxR-CE.These results establish a new state of the art among VLM-based agents on both benchmarks.Source: §4.2 · p. 8

4.3 DEEP DIVE INTO EVIDENCE-SEEKING BEHAVIOR

  • Adaptive Seek uses a 29.8% seek rate and reaches 73% SR and 67% SPL, outperforming Never Seek and Periodic Seek on the 100-episode test subset.Never Seek records 52%/48% SR/SPL and Periodic Seek 62%/53%, at seek rates of 0% and 50%, respectively.Source: §4.3 · p. 8, Figure 3 · p. 8
  • During RL updates, BACR rises from 45.6% to 55.9%, while mean short-horizon progress gain rises from 6.4 × 10−3 to 16.1 × 10−3.BACR counts triggered states where seeking changes the next action and yields greater short-horizon progress than direct navigation.Source: §4.3 · pp. 8–9, Figure 3 · p. 8
  • In simulated examples, supplementary views identify the dining area and then a hallway beyond the kitchen island, grounding the next action at each subgoal.The examples show seeking resolving specific uncertainty in progress grounding before action.Source: §4.3 · p. 9, Figure 4 · p. 9
  • In a real-world example, a side view reveals a chair to the left when the forward view and instruction do not specify the turn, and the agent completes the task.The deployment protocol is detailed in Appendix E.Source: §4.3 · p. 9, Figure 5 · p. 9

5 CONCLUSION

SeekVLN addresses Progress Myopia through active evidence seeking for reliable progress grounding. FRG establishes a prior from offline demonstrations, while C2PO learns evidence-seeking decisions through counterfactual credit assignment; experiments report improved navigation and targeted evidence-seeking behavior.

5 CONCLUSION

  • SeekVLN addresses Progress Myopia through active evidence seeking for reliable progress grounding.The framework combines semantic progress reasoning with acquisition of task-relevant observations.Source: §5 · p. 9
  • FRG derives a cold-start prior from offline demonstrations, and C2PO learns evidence-seeking decisions through counterfactual credit assignment.Source: §5 · pp. 9–10
  • Experiments on R2R-CE and RxR-CE demonstrate improved navigation, while simulated and real-world rollouts show targeted evidence-seeking behavior.The conclusion highlights learning when to acquire evidence and how to use it for reliable progress grounding and navigation under partial observation.Source: §5 · p. 10

A DETAILED ANALYSIS OF PROGRESS MYOPIA

The analysis compares decision confidence on failed trajectories with matched expert-guided successful segments to assess whether models recognize unreliable progress grounding. Its described setup covers representative agents, matched segments, action-confidence scoring, and FRG annotations for seeking, progress, and evidence.

A.2 SUCCESS AND FAILURE SEGMENTS EXTRACTION

  • The analysis compares model confidence on failed rollouts with confidence on corresponding expert-guided segments at matched critical positions.It analyzes 100 failed rollouts, locating the first segment below a path-similarity threshold and aligning it with an equal-length expert-route segment under teacher forcing.Source: A DETAILED ANALYSIS OF PROGRESS MYOPIA · p. 13, §A.2 · p. 14

A.1 SELECTED REPRESENTATIVE NAVIGATION AGENTS

  • The representative agents span direct high-level action prediction, streaming history-aware prediction, and direct navigation with explicit reasoning supervision.The selected models are NaVILA, StreamVLN, and Aux-Think, respectively.Source: §A.1 · p. 13

B.1 CANDIDATE-STATE CONSTRUCTION

  • FRG assigns target SEEK proportions by expert action, with turn targets based on cumulative rotation across consecutive same-direction turns.The described table reports target proportions by action, but the supplied passage does not provide its cell values.Source: Table 2 · p. 14, §B.1 · p. 14

A.3 DECISION CONFIDENCE.

  • Candidate actions are scored by length-normalized sequence log-likelihood, normalized over a shared action set, and confidence is measured as the top-two probability margin.Normalized action entropy is also reported, with both quantities aggregated over matched critical windows and confidence intervals estimated by resampling windows.Source: §A.3 · p. 14

B.2 PROGRESS REASONING ANNOTATION

  • Progress labels are generated only for planned SEEK states and identify the completed instruction prefix and the first unfinished subgoal.Validation requires the completed field to be a contiguous prefix and the next-task field to be the subsequent unfinished subgoal.Source: §B.2 · p. 15

B.3 EVIDENCE SEEKING ANNOTATION

  • Evidence annotations use the instruction, history, and action-conditioned current views to produce one concise key-evidence statement.The supplied views vary with the action, including front and corresponding side views for 30° and 45° turns.Source: §B.3 · p. 15

B.4 DATASET CONSTRUCTION

The FRG dataset serializes expert decisions as NAV or SEEK training examples, with SEEK examples adding evidence interaction and structured progress reasoning. Its prompts separately derive a completed-task prefix and next task, refine those labels conservatively, and annotate concise task-relevant visual evidence.

B.4 DATASET CONSTRUCTION

  • The 4K FRG prior contains 111,141 decision-level examples from 4,162 R2R-CE episodes: 68,795 NAV and 42,346 SEEK examples.NAV examples pair instruction and visual context with an expert action; SEEK examples also include evidence interaction and a think block with progress and evidence fields.Source: §B.4 · p. 15

B.5.1 PROGRESS REASONING STAGE-1 PROMPT

  • Stage 1 compares historical and current visual evidence with provided subgoals to identify the fully completed contiguous prefix and the first unfinished subgoal.Historical observations are sampled every two primitive steps; only the current observation is labeled, and the provided subgoal list defines valid instruction boundaries.Source: §B.5.1 · p. 16

B.5.2 PROGRESS REASONING STAGE-2 PROMPT

  • Stage 2 refines the initial progress response into structured completed-task and next-task labels, preserving only visually supported progress consistent with the instruction.It requires a conservative completed prefix and disallows treating partial completion or a visible landmark alone as completion.Source: §B.5.2 · pp. 16–17

B.5.3 EVIDENCE SEEKING ANNOTATION PROMPT

  • Evidence annotation uses instruction-relevant current views and history to describe visible evidence without predicting an action or inferring unseen objects.The output is a concise key-evidence statement, or an explicit statement that no clear instruction-relevant evidence is visible.Source: §B.5.3 · pp. 17–18

C TRAINING DETAILS

FRG supervised fine-tuning teaches SeekVLN to choose between NAV and SEEK and to produce navigation or structured progress reasoning after evidence acquisition. C2PO then applies decision-level PPO with counterfactual and adaptive outcome rewards, alongside adaptive KL regularization.

C.1 SUPERVISED FINE-TUNING (FRG-SFT)

  • FRG-SFT initializes SeekVLN from pretrained Aux-Think and trains the NAV/SEEK mode choice together with direct navigation or post-seeking progress reasoning and navigation.The mode loss is restricted to the two mode tokens; response training masks environment content between the seek markers while supervising the closing marker and think token.Source: §C.1 · p. 18
  • The restricted mode normalization prevents the rare mode decision from being diluted by the longer free-form response.Source: §C.1 · p. 18

C.2 REINFORCEMENT FINE-TUNING (C2PO)

  • C2PO starts from the FRG-SFT checkpoint and treats each complete decision response as one PPO policy action, using counterfactual and adaptive outcome rewards without a per-decision action penalty.Training also uses adaptive KL regularization against the frozen SFT reference policy.Source: §C.2 · p. 19

D ABLATION STUDY

The ablation study compares training components under controlled settings and evaluates them on a deduplicated R2R-CE subset.

D ABLATION STUDY

  • Ablation variants share the same backbone, training budget, and optimization settings except for the component under study.The study separately ablates FRG and C2PO.Source: D ABLATION STUDY · p. 19
  • A trajectory-deduplicated R2R-CE Val-Unseen subset retains one instruction for each of 613 expert trajectories across all 11 unseen Matterport3D scenes.This avoids evaluating the same route under multiple instructions.Source: D ABLATION STUDY · p. 20

D.1 ANALYSIS OF FRG

FRG supplies progress-reasoning and evidence-seeking supervision; ablations report that removing either signal degrades navigation metrics.

D.1 ANALYSIS OF FRG

  • FRG pairs progress reasoning about completed and upcoming subgoals with evidence-seeking examples linking trigger decisions to task-relevant visual evidence.The ablation removes each supervision signal independently.Source: §D.1 · p. 20
  • Removing progress reasoning lowers SR by 1.96 points and SPL by 2.84 points; removing evidence-seeking supervision lowers them by 2.13 and 2.54 points, respectively.NE also rises in both variants.Source: §D.1 · p. 20, Table 5 · p. 20

D.2 ANALYSIS OF C2PO

C2PO is evaluated by removing its counterfactual reward, while the real-world deployment runs policy inference remotely and executes robot commands locally.

D.2 ANALYSIS OF C2PO

  • The C2PO ablation removes the counterfactual reward while retaining FRG-SFT initialization, adaptive outcome reward, and the other reinforcement fine-tuning settings.This isolates branch-level credit assignment.Source: §D.2 · p. 20
  • Adding the counterfactual reward raises SR from 65.09% to 68.52% and SPL from 60.40% to 62.36%, while reducing NE from 3.94 m to 3.66 m.The seek rate falls from 34.3% to 29.3%.Source: §D.2 · p. 20

E REAL-WORLD DEPLOYMENT

  • SeekVLN-C2PO-RFT runs on a Unitree Go2 with robot sensing and motion and policy inference on a remote GPU server.An SSH tunnel and HTTP interface transfer observations and return discrete actions.Source: Figure 6 · p. 20, E REAL-WORLD DEPLOYMENT · p. 21
  • Visual search acquires left, forward, and right views by turning in place; the controller validates and executes one mapped command before the next observation.A stop prediction is model-declared completion, not independent verification of reaching the goal.Source: E REAL-WORLD DEPLOYMENT · p. 21
  • The deployment records images, model inputs and outputs, robot states, actions, and timing, and stops on inference, communication, observation, or control failure.Images are resized to 448 × 448 pixels, with up to nine real observations supplied as history.Source: E REAL-WORLD DEPLOYMENT · p. 21
Loading 2609.37353v1…