PaperlayerView exact revision on arXiv

Source-linked AI summary

The Layer Mystery of VLA: An Information-Theoretical Analysis of VLA Latent Interface

Yuxiang Liu, Lizhi Yang, Fengze Xie, Aaron Ames, Yisong Yue

arXiv:2609.36118v1cs.AI

TL;DR

VLA latent-interface design leaves open which backbone layers a policy should use and whether it should fuse them. The paper analyzes these choices and proposes InfoNCE-based layer selection; fusion usually trails the best single-layer policy, while the InfoNCE sweep is substantially cheaper than the oracle sweep.

Abstract

from arXiv · show

Vision-language-action (VLA) policies connect a pretrained vision-language backbone to an action head through a latent interface, but which backbone layers this interface should expose remains unclear. We study single-layer selection and multi-layer fusion for frozen backbones across three pretrained models and two manipulation benchmarks, LIBERO and CALVIN, with three policy-training seeds per configuration. Across three fusion mechanisms and three layer-subset strategies, 47 of 54 configurations underperform the best observed single-layer policy. Our stastical analysis further confirms that fusion's advantage is very limited. However, the best layer varies substantially across backbones and benchmarks, making layer selection consequential and exhaustive policy sweeps expensive. We further derive a reweighting equivalence between the proposed information-bottleneck objectives for action-conditioned InfoNCE and action prediction, motivating InfoNCE as a proxy for layer quality. Empirically, InfoNCE provides the most consistent positive association with policy success among four evaluated proxies. Selecting the layer with the highest InfoNCE score requires 9-33 times less GPU compute than exhaustive policy sweeps and reduces mean selection regret from 17.89 percentage points for deepest-layer selection to 3.71 points across six settings. Its mean regret is close to the 3.28-3.50 points achieved by fixed-layer heuristics optimized retrospectively using all six oracle sweeps, without requiring closed-loop evaluations during selection.

I. INTRODUCTION

VLA latent-interface design remains unsettled: prior work disagrees on single-layer selection versus fusion, while evidence across layer choices is limited. This paper examines those design questions and frames the interface through information-bottleneck objectives for action prediction and InfoNCE.

I. INTRODUCTION

  • Prior studies offer conflicting layer and fusion choices, but typically test few layers and report only one training run per configuration.The introduction notes that reported differences of one to three points lack error-bar estimates.Source: §I · p. 1
  • The introduction cautions that interpretability findings from classification and embedding tasks do not establish which information a robot policy uses.Whether mid-depth information found decodable in those tasks is used by a robot policy is left as an open question.Source: §I · p. 1
  • The paper asks whether VLA policies should use a single backbone layer or fuse layers, and which layer to select when using a single layer.The latent interface determines which backbone information the policy can condition on and how much of the backbone must run at inference.Source: §I · pp. 1–2
  • The paper systematically compares existing fusion methods with single-layer interfaces and investigates InfoNCE as a computation-efficient layer-selection proxy.It also develops an information-bottleneck analysis relating action-prediction representations to InfoNCE.Source: §I · p. 2

A. Problem Formulation

  • The authors formulate the VLA network as an action head composed with a latent interface derived from the input, and seek an interface suited to action prediction.The backbone layers are modeled as deterministic functions in the stated setup.Source: A. Problem Formulation · p. 2

1) should solve the following optimization problem:

The paper frames desirable latent interfaces using information-bottleneck objectives for action prediction and action-conditioned InfoNCE. The InfoNCE formulation treats the loss as an N-way classification task and mirrors the action-prediction objective’s relevant-signal and irrelevant-information terms.

1) should solve the following optimization problem:

  • The action-prediction objective balances information about expert actions against action-irrelevant information retained in the latent interface.The paper identifies these terms as I(Z, A) and I(Z, X|A), respectively.Source: 1) should solve the following… · p. 2

B. The InfoNCE Objective and Its Information Bottleneck

  • InfoNCE draws one input jointly with an action and samples the remaining N−1 inputs independently from the input marginal.The jointly drawn input is indexed by the target label Y.Source: B. The InfoNCE Objective and Its… · p. 2
  • A critic scores latent–action pairs, and the InfoNCE loss applies a softmax over the N candidates.Each candidate input is passed through the latent interface to form the tuple of candidate representations.Source: B. The InfoNCE Objective and Its… · p. 2
  • InfoNCE is an N-way classification task whose latent interface should retain information needed to identify the target index while discarding other input information.The classification-relevant term mirrors action information in the action-prediction objective, while the irrelevant-information term mirrors its compression term.Source: B. The InfoNCE Objective and Its… · pp. 2–3

III. SINGLE-LAYER OR LAYER-FUSION

Across tested fusion methods and layer configurations, fusion usually underperforms the best single-layer interface. The comparison spans three fusion methods, three layer-subset strategies, three backbones, and two benchmarks, with three training seeds per configuration.

A. Experiment

  • The experiments compare APT, VLA-Adapter, and BEHAVIOR-1st, with BEHAVIOR-1st explicitly fusing learned combinations of all backbone layers.APT injects layers into the action stream, while VLA-Adapter uses action-token cross-attention to individual layers.Source: A. Experiment · p. 3
  • Three subset strategies test evenly spaced layers, a fixed number of deepest layers, or layers selected for strong single-layer performance.The best-layer subset is intended as an upper-bound setting for each fusion method.Source: A. Experiment · p. 3
  • The evaluation covers Cosmos-2B, Qwen-2B, and Abot-Pretrain on LIBERO and CALVIN, using matched training pipelines and three independent training seeds per configuration.The study controls other implementation choices to isolate fusion method and layer configuration.Source: A. Experiment · p. 3
  • Across three independent training trials per method and layer configuration, 47 of 54 fusion-method/layer-configuration combinations underperform the single optimal layer as a single-layer interface; best-layer configurations underperform it in 17 of 18 cells.Source: A. Experiment · p. 3

B. Statistical Analysis of Fusion’s Advantage

Fusion's advantage is statistically weak, and the best layer varies across backbones and benchmarks. Oracle sweeps are costly, while layer choice can also affect inference speed.

B. Statistical Analysis of Fusion’s Advantage

A. Layer-Quality Oracle

  • Fusion entries compare each method’s success rate with the optimal single-layer rate, with negative differences favoring the single layer.The comparison averages over the same three training seeds.Source: A. Layer-Quality Oracle · p. 4, Table I · p. 4
  • Oracle sweeps train and evaluate a single-layer policy at every layer, using three seeds across LIBERO, CALVIN, and three backbones.They are computationally expensive, and simulation-based policy evaluation remains only a proxy for ground-truth layer quality; hardware evaluation would cost more.Source: A. Layer-Quality Oracle · pp. 3–4
  • The optimal layer is shallower than the deepest layer, but its location varies widely across benchmarks and backbones.For Qwen-2B it shifts from layer 3 to 16 between LIBERO and CALVIN; on LIBERO, Abot prefers layer 31 of 36 while Cosmos-2B prefers layer 15 of 28.Source: A. Layer-Quality Oracle · p. 4

B. InfoNCE as Layer-Quality Proxy

InfoNCE is evaluated as a low-cost proxy for policy quality across layers, and is reported as the most reliable predictor. The paper also relates its information-bottleneck objective to that of action prediction by reweighting shared quantities.

B. InfoNCE as Layer-Quality Proxy

  • InfoNCE is the most reliable predictor of oracle layer quality among the evaluated metrics.The proxy comparison measures Spearman correlation between each metric’s layer-depth curve and the oracle-quality curve.Source: B. InfoNCE as Layer-Quality Proxy · p. 4
  • An InfoNCE sweep is an order of magnitude cheaper than an oracle sweep in GPU-hours.The paper also describes InfoNCE estimation as roughly ten times smaller in GPU-hour cost than an oracle sweep.Source: B. InfoNCE as Layer-Quality Proxy · pp. 4–5, Table II · p. 5
  • Theorem 1 states that the InfoNCE information-bottleneck objective is a reweighted version of the action-prediction objective.Both objectives use I(Z, A) and I(Z, X|A).Source: B. InfoNCE as Layer-Quality Proxy · p. 5
  • InfoNCE’s non-monotone layer trend can track oracle quality, while the information-theoretic account does not rule out other sources of optimization imperfections.The paper notes shared imperfections in InfoNCE optimization and VLA training.Source: B. InfoNCE as Layer-Quality Proxy · p. 5

C. Layer-Selection Scheme

The InfoNCE-argmax selector uses a benchmark-trained contrastive critic to choose a layer, addressing substantial variation in optimal depth across settings. It reduces selection regret relative to deepest-layer selection with far less compute than oracle sweeps.

C. Layer-Selection Scheme

  • The selector trains an InfoNCE critic for each layer with three random seeds and chooses the layer with the highest seed-averaged InfoNCE estimate.The comparison includes fixed-layer heuristics, including two optimized using oracle layer qualities across all six settings.Source: C. Layer-Selection Scheme · p. 5
  • InfoNCE-argmax has 3.71 points mean regret across six settings, versus 17.89 points for deepest-layer selection, while its proxy sweep uses approximately 9–33 times less GPU compute than oracle sweeps.Its 3.71-point regret is close to the 3.28- and 3.50-point averages of two fixed-layer heuristics optimized retrospectively with oracle data.Source: C. Layer-Selection Scheme · pp. 5–6
  • InfoNCE is reported as the most consistent predictor of oracle success across benchmarks and backbones, and its curve closely tracks LIBERO success across layers in two backbones.The figure shows LIBERO success and InfoNCE for Cosmos-2B and Abot-Pretrain; the supplied passages do not provide individual correlation values.Source: C. Layer-Selection Scheme · p. 6, Figure 1 · p. 6

V. CONCLUSION

  • The paper concludes that layer selection is consequential in the evaluated settings and calls for studying action-head capacity, backbone fine-tuning, and real-world deployment.These are identified as future directions for assessing layer quality and fusion benefits.Source: §V · p. 6

APPENDIX

The appendix develops information-bottleneck arguments for comparing layer fusion with single-layer interfaces and motivating the proposed objective. It also gives the proof strategies behind the shallow-layer equivalence and the InfoNCE/action-prediction reweighting equivalence.

A. Proposition Reasoning

  • The appendix distinguishes the proposed information-bottleneck constraint I(Z, X|A) ≤ Ic from the alternative I(Z, X) ≤ Ic in, noting that [12] motivates the conditional form as a tighter generalization bound.Source: A. Proposition Reasoning · p. 8

B. Theoretical Reasoning of Layer-Fusion

  • Theorem 2 states that a tuple of selected layer outputs has the same I(Z, A) and I(Z, X|A) as its shallowest selected layer, explaining why fusion often cannot beat the best single layer.The theorem models learnable fusion mechanisms as part of the action head, with selected layers concatenated as the interface; in 26 of 54 Table VI cells, fusion does not beat its shallowest constituent.Source: B. Theoretical Reasoning of Layer-Fusion · p. 8

C. Proof for Theorem 2

  • The proof uses deterministic mappings in both directions between the shallowest selected layer and the tuple, then applies data processing to establish equality of the information quantities.Source: C. Proof for Theorem 2 · p. 8

D. Alternative Construction of Prop. 2

  • The alternative objective construction weighs classification-relevant information differently; the paper favors preserving compact conditional action information in contrastive training.The discussion contrasts I(Z′; Y |A′) with I(Z′; A′, Y), and rejects a noise term that would discard classification-irrelevant information in A′.Source: D. Alternative Construction of Prop. 2 · pp. 8–9

E. Proof for Theorem 1

  • The proof shows that the InfoNCE information-bottleneck objective and the action-prediction objective share maximizers after reweighting, with α = βN/(1 − β(N −1)).The derivation relies on the positive coefficient condition stated for β and N.Source: E. Proof for Theorem 1 · p. 9

F. Exception Analysis

The exception analysis isolates whether APT’s gains require diverse layer inputs or can arise from its action-head architecture. Its APT-Repeat ablation indicates that architecture change alone can produce the reported gain.

F. Exception Analysis

  • The exception analysis uses Abot-Pretrain on CALVIN because five of seven fusion-win exceptions occur with each of that backbone and benchmark.Source: F. Exception Analysis · p. 9
  • Table VI compares each fusion method’s success-rate difference from its shallowest constituent across benchmarks and base models; positive values mean fusion wins.Differences are percentage points averaged over three training seeds.Source: Table VI · p. 10, F. Exception Analysis · p. 10
  • APT-Repeat[i] gives higher CALVIN success than the single-layer GR00T head at the corresponding layer, and its best performance exceeds APT-Last, APT-Best, and APT-even.Because APT-Repeat uses repeated copies of one layer’s outputs, the reported observations imply that architecture change alone can yield APT’s gain without layer fusion.Source: F. Exception Analysis · p. 10

G. Tightness Analysis of Bound 6

The tightness analysis estimates how far the optimal-critic InfoNCE score falls below mutual information and tests whether this slack changes the layerwise score pattern. The reported slack is small relative to the score and preserves its overall curve shape.

G. Tightness Analysis of Bound 6

  • The analysis defines ΔN as the amount by which the optimal-critic InfoNCE score falls below I(Z, A), quantifying the tightness of bound 6.Source: G. Tightness Analysis of Bound 6 · p. 10
  • The empirical estimate replaces the density ratio with one induced by a fitted critic and uses held-out pairs plus a per-query normalization set.Source: G. Tightness Analysis of Bound 6 · p. 10
  • The empirically estimated slack-to-InfoNCE-score ratio ranges from 0.067 to 0.099, and accounting for slack preserves the score curve’s overall non-monotone-decreasing shape.The reported observations concern LIBERO and Abot-Pretrain.Source: G. Tightness Analysis of Bound 6 · p. 10

H. Statistical Test Measure of Fusion Advantage

The section tests whether fusion outperforms the optimal single layer by comparing each fusion configuration with every layer and combining those comparisons. It also specifies the benchmark datasets, splits, and scoring used in the experiments.

H. Statistical Test Measure of Fusion Advantage

  • Fusion configurations other than APT do not demonstrate statistically significant advantage over the optimal single layer.The table reports a p-value for each fusion configuration; lower values indicate stronger evidence, and p < 0.05 indicates a nominal advantage.Source: Table VII · p. 12
  • The intersection–union test uses the least convincing fusion-versus-layer comparison, requiring every comparison to have p < 0.05 for an overall significant advantage.Comparisons are analyzed separately for each benchmark–backbone setting, using a Welch two-sample t-test adapted to three runs per configuration.Source: H. Statistical Test Measure of Fusion… · pp. 11–12
  • The analysis is approximate with three runs, and nominal p-values are not adjusted across fusion configurations; p-values do not measure advantage size.A value above 0.05 does not establish equivalence or inferiority.Source: H. Statistical Test Measure of Fusion… · p. 12

I. Benchmarks and dataset splits

  • LIBERO evaluation covers 40 tasks across four suites, while CALVIN training uses all 34 canonical tasks and closed-loop evaluation uses 27 standalone tasks.CALVIN's training split contains 4,608 whole trajectories and 274,466 frames; its evaluation excludes seven tasks that cannot be reset as standalone episodes.Source: §I · pp. 12–13
  • Both benchmarks use trajectory-level policy-training splits with a fixed split seed, and validation data are not used to update policies.Suite scores are unweighted means of task success rates: 40 task rates for LIBERO and 27 for CALVIN.Source: §I · p. 13

J. Inputs and latent-interface construction

The section specifies how images, language, state, and action sequences enter the policy, and how oracle layer sweeps and three fusion implementations are constructed. It also records shared training settings and the oracle-selection caveat.

J. Inputs and latent-interface construction

  • Each single-layer interface exposes the complete mixed text–image hidden sequence after a language layer, retaining text, image, and control tokens without mean pooling.The raw sequence is taken before action-head normalization; proxy extraction uses that same sequence.Source: J. Inputs and latent-interface… · p. 13

K. Action representation and prediction head

  • The action head receives proprioception directly and predicts 16 steps of seven normalized action scalars, executing the first eight steps before replanning.The standard head is a 16-block AlternateVLDiT conditioned on the selected VLM sequence, state, noisy actions, diffusion time, and embodiment ID; inference uses four Euler updates.Source: K. Action representation and… · p. 13

L. Policy-training configuration

  • Policies share an 8,000-step AdamW training schedule and checkpoint rule, with three common policy seeds except for the incomplete LIBERO ABot oracle sweep.That oracle sweep instead uses seeds 42, 1234, and 4321; fusion uses 42, 1234, and 2718 for every pair.Source: L. Policy-training configuration · pp. 13–14

M. Single-layer oracle sweep

  • The oracle sweep trains and evaluates every language layer for three seeds, selecting the maximum seed-mean benchmark success curve with ties resolved toward the shallower layer.The same closed-loop evaluations choose and report the maximum, so the observed oracle is an in-sample empirical maximum rather than an unbiased post-selection estimate.Source: M. Single-layer oracle sweep · p. 14

N. Fusion implementations and layer configurations

  • APT injects gated projected VLM states into VLM-token slots before expert blocks, using one self-attention stream over VLM, state, and action tokens.Its gates start at zero, so their sigmoid starts at 0.5; this implementation uses full bidirectional attention and the shared flow-matching objective.Source: N. Fusion implementations and layer… · p. 14
  • VLA-Adapter assigns one fixed selected layer to each of eight cross-attention blocks, without learned layer-mixing weights.Information can still propagate across depths through the action stream's residual state.Source: N. Fusion implementations and layer… · p. 14
  • BEHAVIOR learns separate per-block scalar mixtures of eight selected layers for keys and values, initialized one-hot with zero biases.Its VLM is frozen, and its mixture is restricted to eight selected layers rather than all depths.Source: N. Fusion implementations and layer… · pp. 14–15

O. InfoNCE critic training and scoring

The section details proxy scoring, other proxy metrics, closed-loop evaluation, statistical reporting, and compute accounting. It documents protocol boundaries that affect interpretation, including an unrecorded evaluation seed and a cost/performance seed mismatch for one setting.

O. InfoNCE critic training and scoring

  • InfoNCE uses raw pre-action-head-LayerNorm token sequences and normalized 112-dimensional action windows, with a held-out score computed as log(512) minus crossentropy.Critics use 512-sample batches, 300 epochs, and three proxy seeds; InfoNCE-argmax selects the maximum of the mean layer-score curve, breaking ties toward shallower layers.Source: O. InfoNCE critic training and scoring · p. 15

P. Other proxy metrics

  • DiME and prompt- and dataset-level matrix entropy provide alternative proxies, but the LIBERO/Cosmos DiME results use only two seeds and a different extraction protocol.Other DiME settings use three seeds and the one-forward all-layer extractor; proxy scores are averaged across samples and then seeds.Source: P. Other proxy metrics · p. 15

Q. Closed-loop evaluation procedure

  • Closed-loop evaluation requests ten episodes per LIBERO task and 15 per CALVIN task, with eight actions executed per policy query.Evaluation seeds are unrecorded; the methods share task lists and episode procedures but are not known to share identical random initial states.Source: Q. Closed-loop evaluation procedure · p. 15

R. Statistical analysis and selection regret

  • Policy means average three training seeds, while Spearman correlations compare seed-averaged policy and proxy curves rather than individual-seed curves.Fusion differences subtract independent three-seed means without a propagated uncertainty interval; regret is reported in success percentage points without a confidence interval.Source: R. Statistical analysis and selection… · p. 16
  • Training-seed variation and finite unseeded rollout variation are not separately replicated, so their contributions cannot be decomposed.Raw task-level Boolean outcomes are retained in each result JSON.Source: R. Statistical analysis and selection… · p. 16

S. Computational cost and reproducibility resources

  • Oracle cost sums successful layer-training and evaluation jobs across three seeds, whereas InfoNCE cost includes feature extraction and critic training.For LIBERO/Cosmos, the oracle cost uses timing seeds 1234, 2718, and 31415, while reported performance uses 42, 1234, and 2718.Source: S. Computational cost and… · p. 16

Related papers

Loading 2609.36118v1…