Source-linked AI summary
Wiring Matters: Injection Topology and Initialization of Affordance Heads in Vision-Language-Action Policies
Zijian An, Linhan Wang, Jiayan Wang, Shijie Geng, Ran Yang, Yiming Feng, Lifeng Zhou
TL;DR
VLA action supervision constrains trajectories but does not explicitly identify where interaction should occur, and co-training an affordance head can harm instruction following. The paper studies a stop-gradient affordance head with a learned feature-injection bridge and compares bridge wiring choices. Its residual bridge with active initialization reaches 96.2% average LIBERO success, matching AffordanceVLA’s 95.8%.
Problem
VLA action loss constrains trajectories rather than interaction locations, while reported affordance-head outcomes vary with integration details.
Source: §I · p. 1Method
The method reads intermediate backbone features through a stop-gradient and re-injects an affordance-head feature into the action expert through a learned bridge.
Source: §I · p. 1Results
96.2% average LIBERO success for the residual-active recipe, up from the headless base’s 93.1% and comparable to AffordanceVLA’s 95.8%.
Source: §I · p. 2, B. Main result: the head helps, and… · p. 5Takeaways & Limitations
With the backbone protected, bridge initialization is the dominant wiring choice, while topology contributes a smaller gain under active initialization.
Source: B. Main result: the head helps, and… · p. 7The study covers one simulated benchmark, one backbone scale, and one seed per cell, and its affordance labels use privileged simulator state.
Source: §V · p. 8
Abstract
from arXiv · showhide
Dense affordance supervision is an appealing auxiliary signal for vision-language-action (VLA) policies, yet naively co-training an affordance head can severely damage instruction following. We present a controlled study of how to wire such a head into a modern VLA on the LIBERO benchmark. Our recipe reads the backbone through a stop-gradient and re-injects an intermediate head feature into the action expert via a learned bridge. The stop-gradient is a precondition: letting affordance gradients reach the backbone drops the policy below the headless base (85.5% vs. 93.1%). With the backbone protected, a same-budget 2*2 ablation over injection topology (concatenation vs. residual) and bridge initialization (zero vs. random) shows initialization is the dominant lever. The best wiring, an actively initialized residual bridge, reaches 96.2%, matching the far more elaborate three-expert AffordanceVLA (95.8%) with under 1% extra parameters. Two probes explain the mechanism: ground-truth affordances fed as an input hurt, and inference-time zeroing shows a lazy bridge acts only as a training-time regularizer while an active bridge becomes load-bearing.
I. INTRODUCTION
Dense affordance maps could make interaction locations explicit, but attaching an affordance head can hurt instruction following. This study isolates how gradient routing and bridge wiring affect whether the signal helps.
I. INTRODUCTION
- VLA action loss constrains trajectories, not where interaction should occur, leaving object selection, grasp location, and placement to be learned implicitly.Instruction disambiguation and long-horizon tasks are identified as persistent failure modes; affordance labels offer dense, task-aligned spatial supervision.Source: §I · p. 1
- With the backbone protected, a learned bridge re-injects an intermediate affordance-head feature into the visual tokens conditioning the action expert.The stop-gradient is described as a precondition, not a stylistic choice.Source: §I · pp. 1–2
- The same-budget 2×2 ablation varies bridge topology—concatenation or residual addition—and initialization—zero or random.Initialization is reported as the dominant lever, with active initialization adding +2.0 to +2.7 points on either topology.Source: §I · p. 2
- 96.2% average LIBERO success for the residual-active recipe, up from 93.1% and comparable to AffordanceVLA’s 95.8%.The recipe uses a stop-gradient affordance head and feature re-injection, with under 1% extra parameters.Source: §I · p. 2
- Allowing affordance gradients into the backbone makes the head-equipped policy worse than the headless base: 85.5% versus 93.1%.The reported goal-conditioned success also collapses to 62%.Source: §I · pp. 1–2
II. RELATED WORK
Related work establishes modern VLAs and explores adding spatial structure to their visual inputs. This study focuses on how to attach auxiliary dense supervision, rather than on changing the backbone.
A. Vision-language-action models
- VLA work progresses from RT-2’s action-token transfer to open generalist policies, flow-matching action experts, and humanoid-oriented systems,,, [5], [6].SpatialVLA adds 3D position encodings and CoT-VLA uses intermediate visual sub-goals,.Source: A. Vision-language-action models · p. 2
- This study is orthogonal to backbone choice and examines how auxiliary dense supervision should be attached to VLAs.The related work contrasts backbone-side spatial structure, such as position encodings and visual sub-goals, with the paper’s focus on attachment.Source: A. Vision-language-action models · p. 2
B. The StarVLA platform and the QwenPI model
The StarVLA platform enables controlled changes to VLA components while keeping the rest of the pipeline fixed. The paper uses this modularity to study a compact affordance-head and bridge design.
B. The StarVLA platform and the QwenPI model
- StarVLA’s modular components let the study change the head or bridge while keeping training and evaluation otherwise frozen.The platform supports standardized evaluation across LIBERO, RoboCasa, and CALVIN.Source: B. The StarVLA platform and the QwenPI… · p. 2
- The QwenPI base pairs a Qwen3-VL-2B backbone with a layerwise flow-matching action expert that denoises continuous action chunks.The expert attends to corresponding VLM-layer hidden states, the interface into which the affordance bridge writes.Source: B. The StarVLA platform and the QwenPI… · p. 2
C. Affordances for manipulation
- The affordance design adds a head and linear bridge to a stock VLA; the paper reports matching AffordanceVLA on LIBERO with this simpler architecture.The figure description distinguishes concatenation before shared projection from residual addition of projected features to untouched visual tokens.Source: C. Affordances for manipulation · pp. 2–3
D. Auxiliary objectives, gradient interference, and bridges
Auxiliary affordance gradients can interfere with the primary policy, so the paper isolates them from the backbone and routes the head’s influence through a bridge. The method is evaluated on a flow-matching action policy.
D. Auxiliary objectives, gradient interference, and bridges
- Backpropagating the affordance loss into the backbone reproduced destructive interference with instruction following, motivating a stop-gradient and a separate injection bridge.The bridge connects to work on adding pathways to pretrained models, including zero-initialized identity-preserving connections,, [21].Source: D. Auxiliary objectives, gradient… · p. 3
III. METHOD
- The paper’s best configuration is RES-ACTIVE; three alternative bridge wirings are defined for ablation.The figure summarizes the data flow.Source: §III · p. 3
A. Base policy and notation
- The QwenPI action expert denoises demonstrated action chunks from Gaussian noise using a conditional flow-matching objective.The base policy is trained with the action loss only.Source: A. Base policy and notation · pp. 3–4
B. Affordance supervision from demonstration replay
The affordance head is trained on simulator-derived maps of the gripper’s next task-relevant contact, then reads multiple backbone layers to predict spatially precise, instruction-conditioned affordances. A top-down feature pyramid combines semantic context with localization detail, while stop-gradient prevents the auxiliary loss from updating the backbone.
B. Affordance supervision from demonstration replay
- Simulator replay labels each frame with a fixed-scale Gaussian centered on the gripper’s next task-relevant contact, projected into each camera view.The first future gripper contact during replay supplies the 3D point used to create the per-view label.Source: B. Affordance supervision from… · p. 4
- The target is a look-ahead signal that stays fixed within a task stage and switches at stage boundaries, while its fixed mass leaves location as the informative quantity.This makes the predicted map encode the immediate motor future rather than the present contact.Source: B. Affordance supervision from… · p. 4
C. Affordance head: multi-scale readout of a columnar backbone
- Uniformly spaced backbone-layer taps give the head both early appearance detail for pixel-level localization and late instruction-conditioned semantics for object selection.A single tap would force a choice between those complementary signals.Source: C. Affordance head: multi-scale… · p. 4
- A top-down pyramid reassembles tapped layers at synthetic resolutions and fuses coarse semantic features with finer spatial evidence to produce per-view contact heatmaps.The output is trained with binary cross-entropy.Source: C. Affordance head: multi-scale… · p. 4
- Because image tokens precede the instruction in the causal backbone, pooled instruction embeddings modulate readouts and fusion stages through FiLM.This supplies instruction conditioning so the head can select the commanded object among identical distractors.Source: C. Affordance head: multi-scale… · p. 4
- Stop-gradient restricts the affordance loss to head parameters, protecting the backbone while leaving backbone and action-expert parameters trainable under the action loss.The reported motivation is asymmetric protection: preserve the backbone’s language grounding while allowing the head to adapt.Source: C. Affordance head: multi-scale… · pp. 4–5
D. The affordance bridge
A learned bridge carries an intermediate fusion-stage feature, rather than the final heatmap, into the action expert’s visual tokens. The section contrasts residual and concatenation wiring and identifies bridge initialization as a consequential design choice.
D. The affordance bridge
- The bridge injects an intermediate fusion feature aligned to the backbone’s visual-token grid at every action-expert conditioning layer, rather than passing the one-channel final heatmap.The selected fusion stage combines coarse semantics with fine localization.Source: D. The affordance bridge · p. 5
- Residual injection preserves pretrained visual features and adds the head feature, whereas concatenation re-mixes the original features and measurably erodes fine-grained placement.The section frames residual wiring as additive refinement and concatenation as the more expressive alternative.Source: D. The affordance bridge · p. 5
- Zero initialization is described as lazy: the bridge starts at the base policy, but its near-zero marginal value can leave it runtime-inert; active random initialization makes the feature present from the first step.Active initialization can begin with an action-loss spike before recovery within a few hundred steps.Source: D. The affordance bridge · p. 5
- The four evaluated wirings cross concatenation versus residual injection with lazy versus active initialization, with RES-ACTIVE designated as the method.The accompanying table reports LIBERO success rates for runs sharing data, budget, and seed, with wiring as the difference.Source: D. The affordance bridge · p. 5, Table I · p. 5
E. What can the head even contribute?
Because the affordance target is deterministic from the observation and instruction, and the head reads features already seen by the action expert, its feature adds no new information. The paper therefore distinguishes optimization effects from the possibility that an explicit format eases the expert’s task at runtime.
E. What can the head even contribute?
- The affordance target is deterministic from the observation and instruction, and the head reads backbone features already available to the action expert.Thus, the feature r offers no new information in the information-theoretic sense stated by the paper.Source: E. What can the head even contribute? · p. 5
- The paper identifies optimization dynamics induced by the head’s presence as one possible source of benefit.This is presented as a mechanism to distinguish experimentally, not as an established result in this passage.Source: E. What can the head even contribute? · p. 5
- A second proposed source of benefit is that an explicit affordance format may ease the action expert’s job at runtime.The passage separates this possibility from optimization effects.Source: E. What can the head even contribute? · p. 5
IV. EXPERIMENTS
The experiments compare head wirings under a shared LIBERO training protocol. Evaluation covers four suites, with 50 trials per task.
A. Setup
- All models are fine-tuned jointly on the four LIBERO suites using the same Qwen3-VL-2B initialization.Training uses the LIBERO demonstration corpus across all four suites.Source: A. Setup · p. 5
- The runs share a global batch of 128, 20k training steps, hyperparameters, and seed; only head wiring differs.The training duration is approximately 9.4 epochs.Source: A. Setup · p. 5
- Evaluation uses 50 trials per task across LIBERO-Spatial, LIBERO-Object, LIBERO-Goal, and LIBERO-Long.That is 500 trials per suite.Source: A. Setup · p. 5
B. Main result: the head helps, and wiring decides how much
With the backbone protected, the bridge wiring determines how much affordance supervision helps: active initialization drives the largest gains, while residual injection adds a smaller benefit.
B. Main result: the head helps, and wiring decides how much
- Across the four successful rollouts, predicted contact regions follow the instruction-designated object among distractors and switch at stage boundaries.The long-horizon example switches from the stove knob to the moka pot to the burner.Source: Figure 2 · p. 6
- 96.2% vs. 93.1% average success: RES-ACTIVE gains +5.6 on Goal and +6.2 on Long, while Object and Spatial remain near saturation.The two active wirings dominate on Goal and Long; the figure reports per-suite success rates across 50 trials per task.Source: B. Main result: the head helps, and… · p. 5, Figure 3 · p. 6
- 96.2 with the gradient blocked versus 85.5 when affordance gradients reach the backbone, a 10.7-point swing; the latter falls 7.6 points below the headless base.Goal success falls from the 90s to 62.0 in the gradient-exposed variant.Source: Figure 4 · p. 7
- Active initialization improves average success by +2.0 points with concatenation and +2.7 with residual injection; lazy bridges stay at only a few percent of the identity-path magnitude.With active initialization, the bridge retains full magnitude and the action expert’s representation forms around it.Source: B. Main result: the head helps, and… · p. 7
- Residual injection adds +0.5 over concatenation only under active initialization, with LIBERO-Long rising from 89.4 to 91.4.Under lazy initialization the topologies tie at 93.8 versus 93.6; concatenation re-mixes visual tokens, while the residual identity path preserves them.Source: B. Main result: the head helps, and… · p. 7
C. Stop-gradient is the precondition
The stop-gradient protects the backbone from harmful affordance gradients, enabling useful affordance wiring. The paper also compares its lightweight design with AffordanceVLA and tests whether supplying ground-truth affordances as input helps.
C. Stop-gradient is the precondition
- 10.7 points on average separate stop-gradient RES-ACTIVE (96.2%) from gradient-exposed training (85.5%), which falls 7.6 points below the headless base.The gradient-exposed variant preserves object-level success but collapses goal-conditioned success to 62%.Source: C. Stop-gradient is the precondition · p. 7
D. Comparison with AffordanceVLA
- A stock VLA with one DPT-style head and linear bridge matches AffordanceVLA on LIBERO while adding under 1% to the policy’s parameters.The approach uses single-stage training, whereas AffordanceVLA uses a three-expert architecture and a two-stage curriculum.Source: Table II · p. 7, D. Comparison with AffordanceVLA · p. 7
E. Probe 1: ground-truth affordances as input are net harmful
- 88.3% average success—5.6 points below its matched 94.0% base—results when ground-truth affordance heatmaps are supplied as a third input image.The losses are concentrated on Goal and Long tasks.Source: E. Probe 1: ground-truth affordances… · p. 7
F. Probe 2: inference-time zeroing separates two regimes
Inference-time zeroing distinguishes a lazy bridge, whose benefit is consistent with training-time regularization, from an active bridge that becomes load-bearing. The head predicts contact regions accurately, while remaining failures concern finer control; the conclusion is limited to one simulated benchmark, one backbone scale, and one seed per cell.
F. Probe 2: inference-time zeroing separates two regimes
- Zeroing the lazy CAT bridge leaves Goal 94.8→95.6 and Long 82.6→83.2 within noise, while active-bridge Goal success collapses by 28 points.The lazy bridge still beats the base on Goal by +4.2, consistent with a training-time regularization effect rather than test-time contribution.Source: F. Probe 2: inference-time zeroing… · pp. 7–8
G. Head quality is not the bottleneck
- The affordance head predicts contact peaks within a few pixels and disambiguates duplicate objects, but remaining Long-suite failures involve second-stage placement precision.The paper identifies transducing predictions into finer control as the open problem.Source: G. Head quality is not the bottleneck · p. 8
V. LIMITATIONS AND CONCLUSION
- Under the study’s stated scope, actively initialized variants beat the base by +2.7 points for CAT and +3.1 for RES, and RES-ACTIVE is best in every comparison.The study covers one simulated benchmark, one backbone scale, and one seed per cell; affordance labels come from privileged simulator state.Source: §V · p. 8