PaperlayerView exact revision on arXiv

Source-linked AI summary

Credit-Guided Policy Improvement for Test-time Adaptive Vision-Language Navigation

Yang Li, Sijia Zhang, Yihan Li, Aming WU, Zihao Zhang, Ziju Han, Yahong Han

arXiv:2609.37591v1cs.ROcs.AI

TL;DR

Test-time adaptation for VLN must address whether an executed action supports instruction-guided progress and whether a resulting policy update is reliable enough to retain. CGPI derives reference-relative decision credit from action-induced observations and verifies candidate updates; it improves navigation performance across evaluated settings, while robot trials illustrate zero-shot sim-to-real feasibility.

  • Problem

    Distribution shifts can make pretrained VLN action preferences unreliable, and existing adaptation signals do not directly evaluate whether an executed action contributes to instruction-guided progress or whether a candidate update should persist.

    Source: §1 · pp. 1–2

  • Method

    CGPI uses action-induced observation transitions and comparison with the frozen policy’s local reference to derive signed decision credit, propose lightweight reranker updates, and verify them against past credit-supported interactions.

    Source: §1 · p. 2

  • Results

    CGPI improves navigation performance across the evaluated discrete and continuous VLN settings with multiple pretrained backbones, while qualitative robot trials illustrate zero-shot sim-to-real feasibility.

    Source: §1 · p. 2, §4 · p. 9

  • Takeaways & Limitations

    The evaluated results support feedback-free adaptation by changing only a lightweight residual reranker while keeping the pretrained backbone frozen.

    Source: §4 · p. 9

    Deployment adaptation uses no external task-level correctness or outcome signal, and movement actions provide only the resulting observation as new environment information.

    Source: §2.1 · p. 3

Abstract

from arXiv · show

Test-time adaptation for vision-language navigation (TTA-VLN) enables pretrained policies to adapt online to unseen environments using only test-time observations and interaction history. However, distribution shifts can distort local action preferences and lead to off-course decisions. Existing methods rely on predictive uncertainty, trajectory-level feedback, or accumulated adaptation experience to correct such deviations. These signals, however, do not directly reveal whether an executed action supports instruction-guided progress toward the goal. Moreover, a plausible corrective signal does not guarantee a reliable policy update. The key challenge is thus twofold: identifying interactions that support goal-directed improvement and determining whether the resulting updates are worth retaining. We observe that each executed action induces an immediate observation transition, providing evidence of its local consequences. Based on this insight, we propose Credit-Guided Policy Improvement (CGPI), which recovers signed, reference-relative decision credit from action-induced observation transitions without external outcome feedback. With the pretrained navigation policy frozen, CGPI uses this credit to propose lightweight adaptation updates and verifies them against prior credit-supported interactions. Updates are retained only when supported and rolled back otherwise. CGPI achieves consistent gains across the evaluated VLN benchmarks and navigation backbones, while qualitative robot trials further illustrate the feasibility of zero-shot sim-to-real transfer.

1 INTRODUCTION

TTA-VLN lets pretrained navigation policies adapt online, but distribution shifts can make their action preferences unreliable and adaptation signals do not directly establish that an executed action advances the instruction. CGPI uses action-induced observations to guide candidate updates and verifies them before retaining them.

1 INTRODUCTION

  • Unseen environments and instructions can make pretrained cross-modal action preferences unreliable, while existing adaptation signals do not directly assess the executed action’s contribution to instruction-guided progress.Such distribution shifts can lead to off-course behavior.Source: §1 · pp. 1–2
  • TTA-VLN must identify interactions that support goal-directed improvement and determine whether the resulting candidate policy updates are reliable enough to retain.An action-induced observation transition supplies local evidence of the action’s consequence.Source: §1 · p. 2
  • CGPI derives signed, reference-relative decision credit from an executed action’s resulting observation, then verifies credit-guided reranker updates before retaining them or rolling them back.The pretrained navigation backbone remains frozen during adaptation.Source: §1 · p. 2
  • CGPI is evaluated on REVERIE, R2R, and R2R-CE with four navigation backbones, and qualitative Unitree Go2 trials illustrate zero-shot sim-to-real feasibility.The evaluation spans discrete and continuous VLN settings.Source: §1 · p. 2

2 METHOD

CGPI keeps the pretrained VLN backbone frozen and adapts a lightweight reranker through an identify–propose–verify–retain loop. It derives signed local credit from executed-action observations relative to the frozen policy, then retains candidate changes only after verification against earlier credit-supported interactions.

2.1 PROBLEM SETUP AND OVERVIEW

  • In streaming feedback-free TTA, each task supplies an instruction and initial observation; the backbone stays frozen while a lightweight residual reranker adapts.No external task-level correctness or outcome signal is available during deployment.Source: §2.1 · p. 3

2.2 INTERACTION-GROUNDED IMPROVEMENT IDENTIFICATION

  • For executed movement actions, CGPI compares next-observation-grounded Goal, Progress, Grounding, and Risk evidence with the frozen policy’s movement-only reference, producing signed local credit.Positive credit means the observed consequence is more favorable than the movement-only local reference; negative credit means the opposite.Source: §2.2 · p. 4

2.3 CREDIT-GUIDED CANDIDATE POLICY UPDATE

  • Eligible credits propose a lightweight reranker update: positive credit raises the corresponding decision tendency, negative credit suppresses it, and a KL term regularizes the proposal toward the frozen prior.Weak or excessively drifted updates are filtered before proposal.Source: §2.3 · p. 4

2.4 VERIFY-AND-RETAIN POLICY IMPROVEMENT

  • CGPI verifies candidates against recent credit-supported interactions, retaining them only when verification improves and the policy remains sufficiently close to the frozen prior; otherwise, it rolls back.The current interaction is added to bounded verification memory only after the retain-or-rollback decision, preventing self-validation.Source: §2.4 · pp. 4–5

3 EXPERIMENTS

Experiments evaluate CGPI across VLN benchmarks, navigation backbones, and qualitative robot trials. Ablations examine how observation-grounded credit, reference-relative comparison, and verification contribute to adaptation.

3.1 MAIN RESULTS

  • CGPI achieves the best REVERIE performance across Val Seen, Val Unseen, and Test Unseen, with gains also reported on R2R and R2R-CE.On DUET REVERIE Val Unseen, SR/SPL rises from 56.92/38.03 to 58.76/40.15, and on Test Unseen from 55.12/39.84 to 57.31/41.87 versus IDEA. R2R and R2R-CE results also improve across DUET, BEVBert, and ETPNav; a simulated-only policy’s three Unitree Go2 trajectories illustrate successful multi-stage instruction following without real-world fine-tuning or external outcome feedback.Source: §3.1 · pp. 5–6, Table 1 · p. 5, Table 2 · p. 6

3.2 CASE STUDIES OF CREDIT-GUIDED POLICY IMPROVEMENT

  • Figure 4 illustrates positive-credit updates being retained and a negative-credit update rejected, while IDEA follows an off-course branch and stops away from the target.A later positive-credit update is retained, further directing CGPI toward the target sink.Source: §3.2 · pp. 6–7, Figure 4 · p. 7

3.3 ABLATION OF IMPROVEMENT IDENTIFICATION

  • Observation grounding improves SR, SPL, and CPC over pre-action hypotheses, while reference-relative comparison further improves proposal quality; together they yield the highest navigation performance and CPC and lowest WUR.The ablation supports using the observed transition to determine what happened and the frozen-policy reference to judge whether the consequence is locally favorable.Source: §3.3 · p. 7, Table 4 · p. 7
  • Movement credit alone raises HAMT SR/SPL from 30.40/26.67 to 33.54/29.61, STOP credit alone reaches 31.46/27.55, and combining both yields 34.73/30.78.STOP ends the current episode, so a retained STOP-induced update can affect later episodes under the continual protocol.Source: §3.3 · p. 8, Table 5 · p. 8

3.4 EFFECT OF IMPROVEMENT VERIFICATION AND RETENTION

  • Credit-based verification rejects unsupported modifications, improving navigation performance and retained-candidate quality; the candidate-level KL condition reduces policy drift from 0.057 to 0.043 and raises AIR from 0.704 to 0.741.Without verification, every eligible candidate is committed, yielding higher retention but larger drift and lower AIR.Source: §3.4 · p. 8, Table 6 · p. 8

3.5 ARE RETAINED POLICY CHANGES ACTUALLY BETTER?

  • Compared with direct commitment, CGPI raises AIR from 0.548 to 0.741 and mean accepted improvement gain from 0.19 to 0.63 m; rejection precision is 0.716.Under post-hoc counterfactual evaluation, 71.6% of rejected candidates would have produced negative short-horizon oracle progress.Source: §3.5 · p. 9, Table 7 · p. 9

3.6 WITHIN-EPISODE VERSUS CONTINUAL ADAPTATION

  • Episode-reset CGPI improves HAMT SR/SPL from 30.40/26.67 to 32.62/28.41, while continual adaptation further raises performance to 34.73/30.78.Because reranker and optimizer state reset each episode, the episode-reset gain arises from updates affecting subsequent decisions within that trajectory; retained state adds a cross-episode benefit.Source: §3.6 · p. 9, Table 7 · p. 9

4 CONCLUSION

CGPI frames online VLN adaptation as policy improvement, using action-induced observation transitions for reference-relative decision credit and separating update proposal from verification and retention. Across evaluated settings and backbones it improves navigation performance, with analyses supporting its credit and retention mechanisms and robot trials illustrating feasibility in shown real-world settings.

  • CGPI improves navigation performance across evaluated discrete and continuous VLN settings with multiple pretrained backbones while adapting only a lightweight residual reranker.Source: §4 · p. 9
  • CGPI recovers local, reference-relative decision credit from action-induced observation transitions and separates credit-guided update proposal from subsequent verification and retention.Source: §4 · p. 9
  • Post-hoc analyses show that recovered credit aligns with local navigation progress and selective verification improves the quality of retained candidate updates.Source: §4 · p. 9
  • Qualitative robot trials illustrate zero-shot sim-to-real feasibility in the shown real-world settings.Source: §4 · p. 9
Loading 2609.37591v1…