PaperlayerView exact revision on arXiv

Source-linked AI summary

Drop-Then-Recovery: How Redundant Are Vision-Language-Action Models?

Guoheng Sun, Kaixi Feng, Shwai He, Xiaochuan Gong, Yexiao He, Ziyao Wang, Zheyu Shen, Wanghao Ye, Ramana Rao Kompella, Gaowen Liu, Ang Li

arXiv:2606.27755v1cs.ROcs.AI

TL;DR

VLAs may contain more language capacity than standard robotic manipulation requires, raising questions about architectural redundancy in closed-loop control. The paper removes transformer blocks, fine-tunes the reduced models, and finds language backbones highly recoverable across benchmarks, unlike vision and action pathways.

  • Problem

    The paper asks how much of a VLA model is needed for closed-loop control when standard manipulation instructions are often short and templated.

  • Method

    DTR removes transformer blocks and fine-tunes the remaining model to measure recoverability, while GateProbe ranks blocks by their contribution to downstream action loss.

  • Results

    Across VLA architectures and manipulation benchmarks, language backbones are highly redundant and recoverable, whereas vision and action pathways are substantially less tolerant to removal.

  • Takeaways & Limitations

    The findings support allocating VLA capacity more deliberately and designing benchmarks that place stronger pressure on compositional language grounding and OOD generalization.

  • Takeaways & Limitations

    Dropped models perform comparably to full models near training conditions but show reduced robustness under visual and physical perturbations in real-world deployment.

Abstract

from arXiv · show

Vision-Language-Action (VLA) models enable instruction-driven robotic manipulation, but they inherit oversized language backbones from pretrained VLMs whose capacity far exceeds what is needed for short robotic instructions. This raises a basic question: how much of a VLA model is actually necessary for closed-loop control? In this work, we study architectural redundancy in VLA models by using transformer block removal as a controlled intervention. We introduce \textbf{Drop-Then-Recovery (DTR)}, an analysis protocol that removes selected blocks from a pretrained VLA model and then fine-tunes the resulting model to measure whether the removed capacity was necessary for downstream control. To make this intervention reliable, we propose \textbf{GateProbe}, a one-shot virtual-gate sensitivity metric that ranks blocks by their contribution to the downstream action loss. Across multiple VLA architectures, manipulation benchmarks and even real-robot industrial scenarios, we find a strong asymmetry in post-removal recoverability: \ul{\textit{language backbones are highly redundant for standard robotic manipulation tasks, whereas vision and action pathways are substantially less tolerant to removal}}. On LIBERO, removing half of the LLM blocks even improves OpenVLA-OFT from 95.0% to 98.3% under the same downstream fine-tuning budget, and retaining only two language blocks still recovers baseline-level performance. These results suggest that current VLA benchmarks may exert limited pressure on deep language grounding and compositional instruction understanding, and that future VLA architectures should allocate capacity more deliberately across language, vision, and action components. The code is available at https://github.com/s1ghhh/VLADrop.

1 Introduction

The introduction frames VLA redundancy as a closed-loop control question: large language backbones may exceed standard manipulation needs, while action pathways may remain critical. It presents Drop-Then-Recovery (DTR) as a protocol for testing whether removed transformer capacity is necessary after downstream fine-tuning.

  • VLAs combine visual observations, natural-language instructions, pretrained vision-language backbones, and action prediction modules to directly predict robot actions.
  • Short, templated manipulation instructions motivate questioning whether inherited large pretrained backbones are excessive for standard benchmarks.
  • Redundancy must be evaluated through recovered closed-loop task success because language blocks and action modules can differ substantially in downstream importance.
  • Drop-Then-Recovery removes transformer blocks and fine-tunes the remaining model to test whether the removed capacity was needed for robotic control.A dropped model that recovers task success indicates the removed blocks were not essential for the evaluated task distribution.
  • 98.3% vs. 95.0%: OpenVLA-OFT surpasses its full-model baseline after dropping half of the LLM blocks under matched training compute.π0.5 also reaches 94.0% vs. 91.7% after the same intervention, and retaining only two language blocks still matches baseline performance.
  • Across VLA architectures and manipulation benchmarks, language backbones are highly redundant, whereas action pathways are much less tolerant to removal.Most language blocks can be removed and recovered with little or no loss in task success under current manipulation benchmarks.

2 Related work

Prior work establishes VLA models as combinations of pretrained vision-language backbones and action modules, while studying compression and efficiency across LLMs, VLMs, and VLAs. Existing findings indicate that VLA redundancy is asymmetric, and prior compression methods commonly evaluate models without recovery fine-tuning.

  • Vision-language-action models: VLA models combine pretrained vision-language backbones with action prediction modules for instruction-driven robotic manipulation.Recent scaling of data and backbone capacity has produced strong performance across diverse tasks.
  • Model compression for LLMs and VLMs: LLM and VLM compression spans quantization, unstructured or semi-structured pruning, structured pruning, and transformer block removal based on layer importance.These methods typically evaluate compressed models without recovery fine-tuning.
  • Compression and efficiency for VLA models: VLA efficiency research explores quantization, token pruning, layer skipping, and distillation, while studies report that naive pruning can harm behavior and redundancy is asymmetric across components.These efforts confirm that VLA redundancy exists.

3 Method

DTR probes VLA redundancy by physically removing selected transformer blocks and then fine-tuning the smaller dense model on the downstream task. GateProbe ranks blocks by loss sensitivity to their residual contributions, while recoverability measures post-removal performance rather than immediate degradation.

  • Drop-Then-Recovery: DTR removes selected transformer blocks before recovery fine-tuning to test whether the resulting smaller model retains downstream task performance.The protocol operates on explicit architectural units and keeps the remaining network dense.
  • Stage 1: Drop: The drop stage physically removes the K least important blocks, short-circuits their residuals, and discards their parameters.This produces proportionally reduced FLOPs, memory, and latency on any hardware.
  • Stage 2: Recovery: Recovery fine-tunes the dropped model using downstream action prediction loss, which is essential because small action degradations can compound over long control horizons.The loss may be MSE for continuous actions or flow-matching for diffusion-based heads.
  • Recoverability: Recoverability is post-recovery task performance, distinct from importance measured by the immediate zero-shot degradation after block removal.A block can cause a large initial performance drop yet remain easy to recover after fine-tuning.
  • GateProbe: GateProbe estimates each block’s loss sensitivity by placing a virtual gate on its residual branch, where α_i = 0 represents dropping and α_i = 1 preserves the original model.Its score combines the downstream gradient with the block’s residual contribution, capturing both representation change and downstream reliance.

4 Simulation experiments

Simulation experiments evaluate DTR across four VLA architectures and three benchmarks, revealing that language backbones are far more redundant than vision or action pathways. Whole-block dropping is the preferred compression granularity, and GateProbe is the most reliable block-selection metric, particularly under aggressive compression.

  • Experimental setup: DTR is evaluated on four representative VLA architectures spanning different backbone families, scales, and action-head designs.The models are π0.5, OpenVLA-OFT, Lingbot-VLA, and GigaBrain-0.
  • Experimental setup: Experiments use LIBERO for main redundancy studies and LIBERO-Plus plus RoboTwin 2.0 for robustness and cross-benchmark transfer.These are the three simulation benchmarks evaluated.
  • Component redundancy: 44.5% parameter removal in OpenVLA-OFT’s Language Drop Half matches or exceeds baseline SR, 98.3% vs. 95.0%, while Vision removes 3.6% and falls to 83.9%.Under Keep 2, language remains close to baseline, whereas Vision and Action collapse to 62.4% and 26.2% respectively on π0.5.
  • Dropping granularity: 98.3% for whole-block dropping on OpenVLA-OFT substantially exceeds MHA at 91.9% and MLP at 65.6%.On π0.5, all granularities achieve similar SR from 93.3% to 94.1%, but whole-block dropping compresses the most.
  • Block selection: GateProbe achieves the best or second-best performance across all four drop levels, with advantages of +3.9 at Drop-16 and +4.3 at Drop-17.Non-gradient metrics consistently underperform, while Taylor and IGIA degrade under extreme compression; GateProbe is used thereafter.

5 Real-world experiments

Real-world experiments show that language-block removal can preserve near-training performance, but robustness declines under stronger visual and physical distribution shifts. In parcel sorting, removing half the language blocks remains competitive with the full model, whereas retaining only two of eighteen causes moderate degradation.

  • Robustness under distribution shift: Dropped models perform comparably to the full model near training conditions but show reduced robustness under visual and physical perturbations.This transfers the observed language-backbone redundancy from simulation to real-world deployment.
  • Platform and data: The deployment uses π0.5 on a UFACTORY xArm 850 with wrist-mounted and third-person cameras, trained on approximately 110K teleoperated frames.The setup includes an NVIDIA Jetson Thor and approximately 600 grasps of training data.
  • Main results: 65.0% versus 63.3% in Env 1, Drop-9 slightly outperforms the full model, while Drop-16 falls to 55.0%.Results are averaged over 3 runs × 20 grasps.
  • Main results: 75.0% in Env 2, the full model leads Drop-9 at 71.7% and Drop-16 at 66.7%.The pattern indicates that removing half the language blocks preserves task performance, while retaining only 2 of 18 blocks causes moderate degradation.
  • Robustness under distribution shift: Under mild container-orientation perturbations, Drop-0/9/16 achieve 75%/70%/70%, remaining close to the full model.The evaluation covers six out-of-distribution conditions in Env 2, with a single run of 20 grasps.

6 Analysis and discussion

DTR’s block removal improves compute efficiency and hardware-agnostic inference, while recovery training is essential for translating per-action savings into end-to-end gains. Language redundancy persists across architectures and benchmarks, but removed capacity can also affect physical generalization and robustness.

  • Practical consequences: Drop-12 reaches 93.7% average performance, improving by +2.0 under FLOPs-matched fine-tuning; even Drop-17 recovers to 91.0%.Dropped models permit more training iterations under the same compute budget because each fine-tuning step is cheaper.
  • Practical consequences: DTR-16 delivers 1.64× task speedup and reduces memory by 42%, using a dense model that requires no specialized hardware kernels or runtime support.DTR’s acceleration is hardware-agnostic because it produces a standard dense model with fewer layers, unlike quantization or sparse pruning.
  • Practical consequences: Block Drop 4 achieves 1.05× action speedup but only 0.72× task speedup, as 78% SR degradation increases the number of required steps.The contrast with DTR shows that recovery training is necessary for end-to-end benefits, not merely per-action inference gains.
  • Cross-benchmark generalization: At Drop-9, LIBERO reaches 92.3% versus 91.7% baseline, while LIBERO-Plus drops by 3.8% and RoboTwin 2.0 Hard variants degrade by 6.6%.On LIBERO-Plus, Robot degradation reaches −10.6 versus −5.1 in Language, indicating sensitivity to physical perturbations beyond language understanding.
  • Implications: Language redundancy persists across architectures and benchmarks, but stronger language-complexity and out-of-domain-perturbation benchmarks are needed to test VLA capacity more fully.Simple instructions can leave substantial language capacity underused, while removed blocks may still support generalization to physical perturbations.

7 Conclusion

DTR and GateProbe reveal a strong capacity mismatch in current VLA models: language backbones are substantially overprovisioned for robotic manipulation, while vision and action pathways remain critical and compress poorly. The findings motivate more deliberate capacity allocation and benchmarks that better test compositional language grounding and OOD generalization.

  • Conclusion: DTR and GateProbe systematically expose architectural redundancy in VLA models.The protocols probe whether model capacity is necessary for closed-loop robotic control.
  • Conclusion: Language backbones carry more capacity than current robotic manipulation tasks demand, whereas vision and action pathways remain critical and compress poorly.The conclusion emphasizes a pronounced asymmetry across VLA components.
  • Conclusion: Future VLA architectures should allocate capacity more deliberately to match the computational demands of closed-loop control.This recommendation follows from the mismatch between today’s capacity distribution and task requirements.
  • Conclusion: Future benchmarks should place stronger pressure on compositional language grounding and OOD generalization.The proposed benchmark direction targets capabilities that current robotic manipulation evaluations may underemphasize.

A Model and benchmark details

The study evaluates DTR across four VLA architectures and three simulation benchmarks. LIBERO provides four 10-task suites with 20 trials per task, while LIBERO-Plus adds diverse perturbations and RoboTwin 2.0 targets dual-arm manipulation.

  • Models: Four VLA architectures span different backbones, scales, and action-head designs.The evaluated set includes π0.5 and OpenVLA-OFT among the four architectures.
  • Benchmarks: LIBERO contains four suites—Spatial, Object, Goal, and Long/Libero-10—with 10 tasks per suite and 20 trials per task.Each task is evaluated over 20 trials.
  • Benchmarks: LIBERO-Plus extends LIBERO with visual and physical diversity through perturbations to backgrounds, textures, viewpoints, robot poses, language, lighting, noise, and layouts.The benchmark explicitly includes perturbations across these listed conditions.
  • Benchmarks: RoboTwin 2.0 targets dual-arm manipulation.It is one of the three simulation benchmarks used in the evaluation.

B Block dropping in joint-attention architectures … G Vision drop lists

The paper specifies architecture-aware block dropping, compresses OpenVLA-OFT’s action head by reducing hidden width, and uses GateProbe to rank blocks with a single calibration forward-backward pass. It also catalogs alternative importance metrics and records dataset- and vision-specific retention indices.

  • B Block dropping in joint-attention architectures: In π0.5, joint attention requires retaining each dropped language block’s K/V projections and input normalization so the action expert can still cross-attend to language representations.The language query and output projections plus MLP are removed, while retained K/V projections receive recovery gradients as cross-attention adapters.
  • C Action head compression in OpenVLA-OFT: OpenVLA-OFT compresses its MLPResNet action head by reducing hidden width from 4096 to 2048 or 256.Drop Half reduces action-head parameters from 50.4M to 16.8M; Extreme reduces them to 1.3M.
  • D GateProbe details: GateProbe estimates each block’s loss sensitivity from the inner product between downstream gradients and the block residual, using virtual hooks without modifying the model.Its score is computed with a single forward-backward pass over a small calibration set.
  • D GateProbe details: The profiling algorithm averages absolute gate sensitivities across calibration batches after capturing block residuals and hidden-state gradients.For each block, it accumulates |⟨gi, Fi⟩| and divides by the number of calibration batches.
  • E Block importance metrics: The paper compares eight block-importance metrics across gradient requirements and parameter- versus activation-space operation, including GateProbe, Taylor, IGIA, Fisher, Hessian trace, CosSim, PPL, and Magnitude.Their costs range from zero forward passes for Magnitude to L + 1 forward passes for PPL; Hessian trace requires one forward and two backward passes.
  • E Block importance metrics: CosSim identifies near-identity blocks through low transformation distance, while CosSim (contig.) selects contiguous windows with the smallest angular distance.CosSim requires one forward pass; PPL instead evaluates individual block removal and is the most expensive non-gradient method.
  • F Drop index lookup tables: Kept PaliGemma indices are tabulated separately for LIBERO metric comparisons, LIBERO-Plus GateProbe selections, and RoboTwin 2.0 task-specific profiling.The latter two datasets use selections that differ from LIBERO’s, and each RoboTwin 2.0 task has its own profiling.
  • G Vision drop lists: Vision-dropping experiments use identical attention and MLP drop lists, with OpenVLA-OFT Keep 2 retaining DINO layer 23 and SigLIP layer 25 as endpoints.All vision-layer indices are zero-based.

H Training details · I Per-task results on LIBERO-Plus · J Full per-task results on RoboTwin 2.0

The paper uses matched recovery fine-tuning protocols to isolate the effects of block removal, with compute-adjusted budgets for fair comparisons. It reports complete per-task results for LIBERO-Plus and RoboTwin 2.0, including Easy and Hard evaluation variants across seven RoboTwin tasks.

  • H Training details: Recovery fine-tuning matches the corresponding full-model baseline whenever possible, isolating performance changes caused by dropping blocks.Data, optimization, and training protocols remain unchanged; compute-matched comparisons scale the training budget with the dropped model’s compute reduction.
  • H Training details: LIBERO recovery trains on a mixed dataset combining the Spatial, Object, Goal, and Long suites from modified LIBERO RLDS datasets.The main LIBERO training settings are summarized in Table 14.
  • H Training details: OpenVLA-OFT uses LoRA, L1 action regression, two input images, proprioceptive inputs, and image augmentation for LIBERO recovery.π0.5 uses 30K steps, global batch size 32, bfloat16 precision, and learning rate 5 × 10−5 after a 10K-step warmup.
  • H Training details: RoboTwin 2.0 recovery trains on a mixed set of trajectories from seven selected manipulation tasks.The main RoboTwin 2.0 training settings are summarized in Table 15.
  • H Training details: OpenVLA-OFT uses rank-32 LoRA, L1 action regression, three input images, proprioceptive inputs, and image augmentation for RoboTwin 2.0 recovery.π0.5 uses 30K steps, global batch size 32, bfloat16 precision, and learning rate 5 × 10−5 after a 10K-step warmup.
  • I Per-task results on LIBERO-Plus: Table 16 reports the full per-task results on LIBERO-Plus.The table provides task-level results for the benchmark’s recovery experiments.
  • J Full per-task results on RoboTwin 2.0: Table 17 reports success rates (%) for Easy (clean) and Hard (randomized) evaluation on each of seven RoboTwin 2.0 tasks under different drop levels.These are the full per-task numerical results corresponding to Figure 4.

K Full compression comparison on LIBERO-Goal · L Edge acceleration requires hardware-kernel alignment

On LIBERO-Goal, DTR uniquely improves success rate, task speed, and memory while attention dropping is tolerated but MLP dropping fails. For edge deployment, DTR offers portable acceleration because it produces a smaller dense model without requiring specialized quantization or sparsity kernels.

  • K Full compression comparison on LIBERO-Goal: DTR-16 reaches 100% SR (+2.0 over baseline), 1.64× task speedup, and 42% memory savings, improving all three axes simultaneously.No other evaluated method improves even two axes simultaneously.
  • K Full compression comparison on LIBERO-Goal: A 1.05× action speedup can become 0.72× task speedup when reduced success increases environment steps through failed episodes running to the 300-step horizon.Task Speedup is defined as Act. Speedup / Step Ratio, so per-action gains do not guarantee end-to-end gains.
  • K Full compression comparison on LIBERO-Goal: Wanda 2:4 pruning falls to 42% (LLM-only) or 28% (full-model) SR with no speedup or memory benefit, while INT4 preserves 94% SR but slows inference.INT4 reaches 0.61× action speed and 0.55× task speed because of dequantization overhead in the kernel-agnostic H200 setting.
  • K Full compression comparison on LIBERO-Goal: Attn Drop 8 removes 8 attention sublayers, or 25% of all attention, while retaining 98% SR and 1.09× task speedup.This supports substantial redundancy in attention sublayers under zero-shot cosine-similarity dropping.
  • K Full compression comparison on LIBERO-Goal: Removing 4 MLP sublayers causes 0% SR and 0.42× task speedup, despite MLP BI scores having 10× lower variance than attention BI scores.Cosine similarity therefore underestimates MLP criticality relative to attention criticality.
  • L Edge acceleration requires hardware-kernel alignment: Edge acceleration depends on matching the compressed representation to target hardware capabilities and inference-runtime kernels, because parameter or FLOP reductions need not yield proportional wall-clock gains.This matters for closed-loop control, where latency affects control bandwidth and end-to-end task completion time.
  • L Edge acceleration requires hardware-kernel alignment: DTR provides a simpler and more portable edge-acceleration path, whereas quantization and sparsity are effective when the deployment stack supports their corresponding operators and structures.Examples include supported low-bit formats and 2:4 structured sparsity rather than arbitrary zeros.
  • L Edge acceleration requires hardware-kernel alignment: DTR physically removes transformer blocks and produces a smaller standard dense model using ordinary dense operators with fewer layers.Its acceleration therefore does not rely on low-bit arithmetic, sparse Tensor Cores, or specialized sparse kernels.
Loading 2606.27755v1…