PaperlayerView exact revision on arXiv

Source-linked AI summary

Asking Less, Adapting Better: Feedback-Efficient Test-Time Adaptation for Vision-and-Language Navigation

Zixu Wang, Huaquan Zhang, Shoukai Xu, Jinwu Hu, Zhiquan Wen, Mingkui Tan

arXiv:2610.30643v1cs.CVcs.AI

TL;DR

VLN agents can lose performance in unseen environments, while existing adaptation approaches face unreliable sequential updates and coarse use of human feedback. FETA selects trajectories and actions for sparse feedback, then applies discrepancy-aware updates at queried steps. Across benchmarks it reports state-of-the-art performance with fewer human queries and lower per-episode latency.

  • Problem

    In unseen environments, VLN performance can degrade, while existing human-feedback TTA uses trajectory-level signals across actions that may be locally correct or suboptimal.

    Source: §I · p. 1, §III · p. 3

  • Method

    FETA filters trajectories by normalized-entropy fluctuation, selects a critical action by entropy-gradient saliency, and applies discrepancy-aware entropy optimization only at the queried step.

    Source: C. Feedback-Driven Policy Optimization · p. 5, §IX · p. 10

  • Results

    18.30% relative SPL improvement over ATENA with DUET on the REVERIE val unseen split, with FETA reporting state-of-the-art performance across benchmarks using fewer human queries and lower per-episode latency.

    Source: B. Main Navigation Results · p. 6, §IX · p. 10

  • Takeaways & Limitations

    FETA’s feedback-efficient adaptation remains advantageous over the unadapted baseline at 60% simulated feedback accuracy and improves in each of three continual-adaptation rounds.

    Source: D. More Discussions · p. 7

Abstract

from arXiv · show

Vision-and-Language Navigation (VLN) requires agents to follow instructions and navigate to target locations. However, current VLN agents often suffer severe performance degradation in unseen environments due to distribution shifts. Test-Time Adaptation (TTA) offers a promising paradigm to bridge this gap, yet existing methods that rely on unsupervised entropy minimization can propagate errors through the trajectory, causing cascading failures in navigation. Recent methods mitigate this by incorporating human feedback, but they apply trajectory-level supervision uniformly across all steps without localizing the actions is truly informative, leading to inefficient feedback utilization and mismatched optimization granularity. To address these limitations, we propose FETA, a Feedback-Efficient Test-Time Adaptation method that improves both feedback efficiency and optimization reliability. To reduce unnecessary human interventions, we introduce a hierarchical sample selection strategy to localize informative samples. Specifically, we first filter trajectories whose normalized policy entropy exhibits large fluctuations. Within each selected trajectory, we further select the critical action via an entropy-gradient saliency criterion and query a single binary feedback on its correctness. A discrepancy-aware entropy optimization objective then updates the policy only at the queried step, with adaptive scaling that assigns larger relative weights to samples with greater prediction-feedback discrepancy. Extensive experiments across multiple benchmarks demonstrate that FETA achieves state-of-the-art performance with fewer human interventions, while remaining robust under noisy feedback and continual adaptation settings.

I. INTRODUCTION

FETA addresses unreliable and inefficient test-time adaptation in VLN by selecting informative trajectories and actions for sparse human feedback. Its discrepancy-aware updates target the queried action, and experiments report stronger navigation performance with fewer interventions.

I. INTRODUCTION

  • Prior human-feedback methods use trajectory-level labels across all actions, although successful trajectories can contain suboptimal actions and failed ones locally correct decisions.This creates inefficient feedback use and mismatched optimization granularity.Source: §I · p. 1
  • FETA filters trajectories with large fluctuations in normalized policy entropy, then uses entropy-gradient saliency to select one critical action for binary correctness feedback.This hierarchical selection localizes feedback from trajectory to action level.Source: §I · p. 2
  • At the queried step, discrepancy-aware entropy optimization uses feedback to set the update direction and gives greater relative weight to samples with greater prediction-feedback mismatch.Correct actions are reinforced; incorrect actions prompt relaxation of overconfident predictions and encouragement of alternatives.Source: §I · p. 2
  • FETA reduces human interventions through hierarchical sample selection, seeking a favorable trade-off between feedback cost and performance gain.The paper also describes the method as improving optimization reliability.Source: §I · pp. 1–2
  • 18.30% relative SPL improvement over the strongest existing navigation TTA baseline on REVERIE val unseen with DUET, while FETA uses fewer interventions and remains robust to noisy feedback and continual adaptation.The reported comparison is part of the paper’s state-of-the-art results across multiple benchmarks.Source: §I · p. 2

A. Vision-and-language navigation

VLN agents follow natural-language instructions through grounded perception and sequential actions, but deployment in unseen environments can degrade performance. Test-time adaptation faces a reliability challenge in this sequential setting, while existing human-feedback methods also use coarse trajectory-level supervision for local actions.

A. Vision-and-language navigation

  • VLN requires agents to translate natural-language instructions into grounded perception and sequential actions.The task is framed as navigation in a 3D environment using an instruction, visual observations, and action choices.Source: A. Vision-and-language navigation · p. 2, §III · p. 2

B. Test-Time Adaptation

  • Test-time adaptation updates model parameters using target-domain data during inference, but unsupervised entropy objectives can be weakly aligned with navigation and brittle under distribution shifts.In sequential navigation, an unreliable update can affect later states and optimization signals, producing cascading failures.Source: B. Test-Time Adaptation · p. 2, §III · p. 3

III. PROBLEM STATEMENT AND MOTIVATION

  • Human-feedback TTA methods use one trajectory-level signal across actions, despite successful paths containing suboptimal actions and failed paths containing locally correct ones.This coarse supervision can provide inappropriate action-level optimization signals and reduce the effectiveness of human guidance.Source: §III · p. 3

IV. FEEDBACK-EFFICIENT TEST-TIME ADAPTATION

FETA selects where to request feedback and concentrates policy updates on the queried decision. It uses trajectory-level entropy fluctuations and step-level entropy-gradient saliency to allocate sparse supervision, with progress-based binary feedback in the simulator.

IV. FEEDBACK-EFFICIENT TEST-TIME ADAPTATION

  • FETA hierarchically selects informative trajectories and decision steps for binary feedback, then uses that supervision for discrepancy-aware, step-level policy updates.The update applies directional entropy optimization with discrepancy-aware scaling.Source: §IV · p. 3, Figure 1 · p. 4

A. Preliminary

  • The simulator’s progress-based oracle marks a non-stop action positive when it reduces goal distance and STOP positive when the agent reaches the goal.Local progress approximates human judgment of action usefulness.Source: A. Preliminary · p. 3

B. Hierarchical Sample Selection

  • Selecting trajectories by normalized-entropy standard deviation is more effective than using mean entropy under closely matched feedback counts.Large fluctuations indicate alternating confident and uncertain phases, while high mean entropy reflects globally diffuse uncertainty.Source: B. Hierarchical Sample Selection · p. 4, Figure 2 · p. 5
  • One carefully selected step offers the most cost-effective trade-off: additional queries can improve success rate, but gains diminish as feedback counts rise.The first queried step yields the largest performance gain.Source: B. Hierarchical Sample Selection · p. 4
  • For each selected trajectory, FETA queries the action at the step with the largest entropy-gradient saliency score.The score measures sensitivity of normalized entropy to action logits.Source: Figure 1 · p. 4, B. Hierarchical Sample Selection · p. 5

C. Feedback-Driven Policy Optimization

FETA turns sparse binary feedback into local entropy updates: feedback discrepancy sets each queried sample’s relative weight, while the feedback label sets the update direction.

C. Feedback-Driven Policy Optimization

  • Larger prediction-feedback discrepancies receive larger relative weights, focusing optimization on queried decisions where the policy disagrees with feedback.Samples for which the policy already agrees with feedback contribute less.Source: C. Feedback-Driven Policy Optimization · p. 5
  • The method uses K = 1 to balance feedback cost and performance as the maximum number of feedback queries per trajectory.Source: Figure 3 · p. 5
  • Positive feedback concentrates the local policy around the verified action, whereas negative feedback relaxes its preference to allow probability to shift toward alternatives.Source: C. Feedback-Driven Policy Optimization · p. 5
  • FETA updates only the single queried step in trajectories that pass the trajectory-level filter, weighting samples by prediction-feedback discrepancy and using feedback to reinforce or relax the local decision.The full objective combines discrepancy-aware reweighting with directional entropy optimization.Source: C. Feedback-Driven Policy Optimization · p. 5

A. Benchmark and Metrics

FETA is evaluated on REVERIE, R2R, GSA-R2R, and R2R-CE using navigation, grounding, and feedback-cost metrics. Across reported results, it improves on ATENA, including on continuous-control navigation.

A. Benchmark and Metrics

  • The evaluation covers REVERIE, R2R, GSA-R2R, and R2R-CE, with SR, OSR, SPL, NDTW, trajectory length, and navigation error reported; REVERIE also includes grounding metrics and feedback count.R2R results are provided in Appendix A.Source: A. Benchmark and Metrics · pp. 5–6

B. Main Navigation Results

  • 18.30%, 26.21%, and 15.34% relative SPL improvements over ATENA on REVERIE val unseen are reported with DUET, GOAT, and ScaleVLN, respectively.FETA also has lower feedback counts across all reported base-model and split settings and requires less execution time per episode.Source: B. Main Navigation Results · p. 6, Table I · p. 6
  • Up to a 10.00% relative SPL improvement over ATENA is reported on the more challenging GSA-R2R non-residential split, with consistent improvements across settings.ATENA brings limited gains and degrades GR-DUET on the residential split.Source: B. Main Navigation Results · pp. 6–7, Table II · p. 6
  • Table IV reports ablations of trajectory-level selection, step-level selection, and discrepancy-aware scaling on REVERIE val unseen, with the unadapted model as baseline.Source: Table IV · p. 7
  • On R2R-CE val unseen, FETA reports relative SR improvements over ATENA of 3.51% with ETPNav and 1.67% with BEVBert.Source: Table III · p. 6, B. Main Navigation Results · p. 7

C. Ablation Study

FETA’s ablations support all three selection and scaling components. Additional evaluations report resilience to noisy feedback and continual adaptation, cross-dataset transfer, and improved real-robot navigation.

C. Ablation Study

  • Removing trajectory selection, step selection, or discrepancy-aware scaling reduces performance; each component alone also beats the unadapted baseline.The ablations replace selection with uniform random sampling under the same feedback budget or set queried-sample weights to one.Source: C. Ablation Study · p. 7

D. More Discussions

  • In physical office and laboratory navigation, FETA raises success rate by 15 and 10 percentage points, respectively, while reducing navigation error in both environments.The evaluation uses 20 episodes per environment and compares against BEVBert.Source: Table VI · p. 7, D. More Discussions · p. 8
  • Across three continual-adaptation rounds, FETA improves each round, while ATENA eventually falls below unadapted performance.The authors suggest step-local supervision may better align feedback with the action being optimized than trajectory-level feedback.Source: D. More Discussions · pp. 7–8
  • 8.97% relative SPL improvement on REVERIE Val Unseen: entropy-gradient saliency outperforms highest-normalized-entropy step selection under an equal budget.The authors suggest high-entropy steps can approach a uniform policy distribution, where the entropy gradient is weak.Source: D. More Discussions · p. 7, Table VII · p. 8
  • At 60% simulated feedback accuracy, FETA still outperforms the unadapted baseline; it falls below baseline only at 50% accuracy.Feedback labels are randomly flipped to simulate noise; the comparison is shown in Fig. 4.Source: D. More Discussions · p. 7, Figure 4 · p. 8
  • FETA shows better adaptation performance than competing methods in both R2R→REVERIE and REVERIE→R2R transfer directions.Table V reports cross-dataset adaptation with models trained on one dataset and adapted on the other.Source: Table V · p. 7, D. More Discussions · p. 8

VI. HUMAN–SIMULATOR FEEDBACK AGREEMENT

In 90 REVERIE Val Unseen queries, human annotations agree with simulator feedback on 78.9% of actions. The implementation uses online, batch-size-one adaptation and selects episodes and steps using entropy-based criteria.

VI. HUMAN–SIMULATOR FEEDBACK AGREEMENT

  • 78.9% of 90 action queries received matching human and simulator labels, with 81.8% agreement on simulator-positive and 76.1% on simulator-negative queries.Each query received one human annotation; simulator labels were 48.9% positive and annotators judged 52.2% of actions helpful.Source: §VI · p. 8

VII. IMPLEMENTATION DETAILS

  • Adaptation is online with batch size 1, and the reported results average three runs using distinct test-sample orderings.An update is triggered only when trajectory normalized-entropy standard deviation exceeds a dynamic threshold; the selected step has the highest entropy-gradient saliency.Source: §VII · pp. 8–9

VIII. QUALITATIVE ANALYSIS

Qualitative examples show FETA-guided routes reaching instruction-specified destinations missed by the base policies in simulation and on a physical robot. The conclusion describes FETA as a step-targeted, discrepancy-aware adaptation method with fewer queries and lower per-episode latency.

VIII. QUALITATIVE ANALYSIS

  • In two REVERIE examples, FETA routes reach the instructed dining-room table-support and balcony destinations that the unadapted model misses.The base policy makes a plausible but erroneous branching-point decision in both examples.Source: §VIII · pp. 9–10
  • Real-robot examples show FETA reaching a blue display board and glass door that BEVBert fails to reach.Together with the simulation examples, these provide qualitative evidence across simulated and physical environments.Source: §VIII · p. 10

IX. CONCLUSION

  • FETA combines entropy-based trajectory filtering, entropy-gradient step selection, and discrepancy-aware step-level updates; the conclusion reports fewer human queries and lower per-episode latency.Source: §IX · p. 10

APPENDIX A ADAPTATION RESULTS ON R2R

On R2R, FETA improves overall navigation performance across DUET, GOAT, and ScaleVLN backbones. On Val Unseen, it achieves higher success than ATENA while maintaining or improving path efficiency.

APPENDIX A ADAPTATION RESULTS ON R2R

  • FETA improves overall R2R navigation performance across DUET, GOAT, and ScaleVLN backbones.On Val Unseen, it achieves higher navigation success than ATENA while maintaining or improving path efficiency.Source: Table VIII · p. 13, Appendix A · p. 13

APPENDIX B DERIVATION OF ENTROPY-GRADIENT SALIENCY

The appendix derives the entropy-gradient saliency used to select critical actions from the policy’s softmax probabilities and normalized entropy. The resulting score can be computed in closed form without backpropagation through the model.

APPENDIX B DERIVATION OF ENTROPY-GRADIENT SALIENCY

  • The policy is formed by applying softmax to step-level action logits, and its entropy is normalized by the logarithm of the number of feasible actions.The derivation treats the feasible-action count at a step as constant.Source: Appendix B · p. 13
  • The derivation uses the softmax Jacobian to obtain the gradient of policy entropy with respect to the action logits.It substitutes the softmax derivative into the entropy derivative and simplifies using the probabilities’ sum-to-one property.Source: Appendix B · p. 13
  • Entropy-gradient saliency is the L2 norm of the normalized-entropy gradient and can be computed from softmax probabilities and entropy without model backpropagation.The closed-form expression is intended for the step-level selection criterion.Source: Appendix B · p. 13

Related papers

Loading 2610.30643v1…