NavBind: Diagnosing Instruction-Action Binding at Route Forks in Navigation VLAs

1 Introduction

• Success and route-fidelity metrics cannot establish whether a model follows instructions or relies on visual cues or navigation history.Route-fidelity metrics compare generated and ground-truth trajectories, providing only an indirect measure of instruction following.Source: §1 · p. 1
• NAVBIND recombines route-aligned instruction segments into 3,252 contexts with two feasible branches, matched agent state and history, and reference actions for both branches.The paired instructions share the completed-route prefix and differ in their branch suffixes.Source: §1 · pp. 1–2
• The diagnosis compares generated action sequences, fixed-reference action likelihoods, and visual attention and attribution across the paired instructions.Likelihood comparisons can reveal action-preference changes even when the generated sequence stays the same.Source: §1 · p. 2
• Higher reported navigation success does not imply more faithful instruction following across the evaluated navigation VLAs.The models also respond more strongly to opposite-turn contrasts than to forward-versus-turn contrasts.Source: §1 · p. 2

2.1 Language Diagnostics in VLN

• Prior VLN diagnostics use task metrics, supervision, modality ablations, or instruction interventions, but some tests keep intended behavior unchanged or lack matching references for modified instructions.Graph-based action tests also do not directly apply to VLAs that execute low-level actions.Source: §2.1 · pp. 2–3
• NAVBIND recombines route-aligned instruction spans into long-horizon full-instruction pairs that specify different executable branches under the same state, history, and completed prefix.Scoring both reference action sequences under both instructions tests whether local preferences shift toward the instructed branch.Source: §2.1 · p. 3

2.2 Language-Action Coupling in VLAs

• Recent VLA studies probe language effects and internal pathways mainly in manipulation, often combining diagnosis with model modification; NavBind tests frozen navigation VLAs using valid route branches, fixed-target likelihood, and visual attribution.The branch instructions share a history while specifying different executable branches.Source: §2.2 · p. 3

3.1 VLN-CE

• In VLN-CE, agents follow natural-language instructions through continuous 3D environments, receive egocentric observations, and choose actions from navigation history.The common four-action interface includes forward movement, left and right turns, and stop; exposed action logits permit exact likelihood evaluation of fixed action sequences.Source: §3.1 · p. 3

3.2 Local Route-Fork Diagnosis

• At a nonterminal route fork, NAVBIND forms full instructions from a completed-route prefix and either of two suffixes specifying executable branches.The two suffix conditions use the same pose, current observation, and observation-action history; instruction-action binding requires higher probability for the actions required by the suffix.Source: §3.2 · pp. 3–4

4.1 Route-Aligned Source Pairs

• Source routes share a directed segment to a fork, then diverge into executable, nonterminal branches, with sufficient shared path to align arrival positions and orientations.Fine-grained sub-instruction annotations place the boundary at the fork; the prefix covers the route up to it and the suffix preserves the remaining original wording.Source: §4.1 · p. 4
• Expert replay pairs each source route's completed prefix with a history ending at the fork, where both local branch targets are constructed.These history-prefix pairs provide the fixed contexts for comparing instructions and branch targets.Source: §4.1 · p. 4

4.2 Matched Interventions

• Within each context, the primary intervention swaps suffixes while holding replay history and fork state fixed; a supplementary comparison holds the suffix fixed while changing history and prefix.The retained set contains 3,252 contexts from 1,648 source pairs, shared across models, with reference sequences differing within four actions at every evaluated native action scale.Source: §4.2 · p. 4

5.1 Generated Action Sequences

• Generated sequences are compared under both instructions without teacher forcing, counting exact matches in action symbols, order, and length within horizon K.The reported Same sequence rate is the percentage of contexts with identical sequences.Source: §5.1 · p. 5

5.2 Fixed-Target Action Likelihood

• Fixed-target likelihood can reveal action-preference changes even when discrete generation remains unchanged.The comparison keeps each reference target fixed while scoring it under both instructions.Source: §5.2 · p. 5
• Fixed-target likelihoods compare each branch’s reference actions under both instructions, yielding four scores per context.For chunk policies, mean negative log-likelihood is computed over target action tokens with the same preceding target tokens under both suffixes; lower NLL means higher target likelihood.Source: §5.2 · p. 5
• Mean suffix effects are converted to relative percentage changes, with positive values indicating that matching suffixes make reference actions more likely.Both+ counts contexts where both branches’ reference actions are more likely under their matching suffixes.Source: §5.2 · pp. 5–6
• Symmetric KL divergence measures redistribution of action probabilities and is zero when those probabilities are unchanged, regardless of target alignment.Action distributions are renormalized over the four legal actions at each reference step.Source: §5.2 · p. 6

5.3 Visual Attention and GT-Conditioned Attribution

• Visual comparisons use raw attention under a shared reference sequence, while gradient-based attribution follows each instruction’s corresponding ground-truth sequence.For single-step policies, fixed-reference attention is compared at aligned ground-truth states; corresponding-GT gradients are compared only at the shared fork observation.Source: §5.3 · p. 6
• Visual IoU measures spatial change: identical selected regions score one, disjoint regions score zero, and lower values indicate greater change.Attention×gradient comparisons jointly change the instruction and the explained action.Source: §5.3 · p. 6

6.1 Data and Models

• The released checkpoints of StreamVLN, JanusVLN, and FutureNav are evaluated on the same 3,252 NAVBIND contexts, which were constructed in Section 4.Source: §6.1 · p. 6

6.2 Evaluation Protocol

• Evaluation uses a four-action horizon and each model’s native action increments.Uni-NaVid and StreamVLN generate action chunks under fixed observations; JanusVLN and FutureNav execute each predicted action before the next decision.Source: §6.2 · p. 6
• Native decoding is retained: Uni-NaVid uses sampling, while StreamVLN uses greedy decoding.Source: §6.2 · p. 6

7.1 Main Results

• 54.4% versus 54.5% SR accompanies StreamVLN’s higher d-bar (0.8360 versus 0.4411), Both+ (80.38% versus 70.60%), and KL (0.8233 versus 0.6208), so higher success does not imply stronger binding.Uni-NaVid has the highest reported RxR-CE val-unseen SR, while StreamVLN has the highest d-bar and KL.Source: §7.1 · p. 6, Table 1 · p. 7

7.2 Consistency Across Diagnostics

• For StreamVLN, unchanged action sequences coincide with lower d-bar and higher attention, gradient, and combined IoU than changed sequences.The likelihood and visual diagnostics therefore agree with the generated-action diagnostic for these groups.Source: §7.2 · p. 7, Table 2 · p. 7

7.3 Weaker Responses with Longer Prefixes

• As completed prefixes lengthen, StreamVLN has higher Same seq. and lower d-bar and KL, indicating weaker responses to the remaining instruction.The results group contexts by prefix token count, with roughly equal group sizes.Source: §7.3 · p. 7, Table 3 · p. 7

7.4 Stronger Responses to Opposite Turns

• For StreamVLN, left-right instruction pairs have lower Same seq. and higher d-bar and KL than forward-left or forward-right pairs.The model responds more strongly when instructions contrast opposite turns than when they contrast forward movement with a turn.Source: §7.4 · p. 7, Table 4 · p. 7

7.5 Attention Changes at Action Divergence

• Attention IoU reaches its minimum at the first position where the reference action sequences diverge, after higher overlap at preceding positions for later-diverging groups.For the step-2 and step-3 groups, IoU rises again at the next position.Source: §7.5 · pp. 7–8, Table 5 · p. 8

7.6 Larger Original-Pairing Advantages on Training Sources

• StreamVLN has lower reference-action NLL under original than recombined pairings in both splits, with a larger gap on train, a pattern consistent with overfitting to original pairings.The table reports mean reference-action NLL and defines the gap as recombined minus original NLL.Source: §7.6 · p. 8, Table 6 · p. 8

7.7 Stability Across Histories

• Across histories, StreamVLN preserves its generated sequence more often than JanusVLN and FutureNav (51.42% versus 25.18% and 27.67%), with the same stability ordering across action and visual diagnostics.Table 7 interprets higher Same seq. and IoU and lower KL as greater stability.Source: Table 7 · p. 8, §7.7 · p. 8

• NAVBIND combines generated behavior, fixed-target likelihoods, and visual attention and attribution to diagnose instruction-action binding under fixed navigation contexts.Source: §8 · p. 8
• Across 3,252 shared contexts, higher reported navigation success does not always mean more frequent action changes or larger shifts in action probabilities.Source: §8 · p. 8
• The findings support evaluating how language affects local decisions alongside episode-level navigation success.Source: §8 · p. 8

• Recombined source instructions may introduce ambiguity or unnatural transitions at segment boundaries.Source: Limitations · p. 8
• The four-action window excludes later instruction responses and error recovery, while model-native action increments give the window different physical extents across models.Source: Limitations · p. 8
• Visual IoU measures spatial changes in attention and attribution but does not establish region relevance to the instructed branch or a causal role in action selection.Source: Limitations · p. 8