PaperlayerView exact revision on arXiv

Source-linked AI summary

StableVLA: Towards Robust Vision-Language-Action Models without Extra Data

Yiyang Fu, Chubin Zhang, Shukai Gong, Yufan Deng, Kaiwei Sun, Qiyang Min, Qibin Hou, Yansong Tang, Jianan Wang, Daquan Zhou

arXiv:2605.18287v1cs.CVcs.RO

TL;DR

VLA models are often evaluated under idealized visual conditions despite real-world disturbances, raising questions about robustness to unseen corruption. StableVLA uses an information-bottleneck adapter to filter visual noise, improving baseline performance by 35.2% in simulation and 20.4 percentage points in real-robot experiments.

  • Problem

    Existing evaluations primarily use controlled, idealized visuals, while real-world deployments encounter degradations largely absent from curated training data [12] [13], leaving a gap between benchmark and real-world performance [25] [28] [30].

    Source: §1 · p. 1

  • Method

    IB-Adapter replaces the VLA adapter with an information-bottleneck-based block between the vision branch and LLM, using independent sigmoid gates to suppress noisy channels.

    Source: §1 · p. 3, §3.2 · p. 6

  • Results

    IB-Adapter yields a 35.2% performance improvement over baseline in the simulator and 20.4 percentage points on real-robot experiments with other settings unchanged.

    Source: §1 · p. 3

  • Takeaways & Limitations

    StableVLA achieves robustness competitive with data-intensive 7B-scale state-of-the-art VLAs using a 14× smaller backbone and no external OpenX pre-training.

    Source: §5 · p. 10

    Glass Blur is evaluated only on LIBERO-Spatial; it is excluded from LIBERO-Object/Goal/Long and CALVIN because of prohibitive interaction-time computational cost.

    Source: §4.1.2 · p. 7

Abstract

from arXiv · show

It is infeasible to encompass all possible disturbances within the training dataset. This raises a critical question regarding the robustness of Vision-Language-Action (VLA) models when encountering unseen real-world visual disturbances, particularly under imperfect visual conditions. In this work, we conduct a systematic study based on recent state-of-the-art VLA models and reveal a significant performance drop when visual disturbances absent from the training data are introduced. To mitigate this issue, we propose a lightweight adapter module grounded in information theory, termed the Information Bottleneck Adapter (IB-Adapter), which selectively filters potential noise from visual inputs. Without requiring any extra data or augmentation strategies, IB-Adapter consistently improves over the baseline by an average of 30%, while adding fewer than 10M parameters, demonstrating notable efficiency and effectiveness. Furthermore, even with a 14x smaller backbone (0.5B parameters) and no pre-training on the Open X-Embodiment dataset, our model StableVLA achieves robustness competitive with 7B-scale state-of-the-art VLAs. With negligible parameter overhead (<10M), our approach maintains accuracy on long-horizon tasks and surpasses OpenPi under both synthetic and physical visual corruptions.

1 Introduction

The introduction shows that leading VLA models are vulnerable to visual disturbances and presents IB-Adapter as a data-free architectural response. It reports improved corruption robustness in simulation and real-robot experiments while retaining a lightweight adapter-based approach.

1 Introduction

  • Benchmarks typically use controlled, idealized visuals, unlike deployment disturbances such as sensor noise, motion blur, and weather effects that are largely absent from curated training data.The introduction identifies this mismatch as a gap between benchmark and real-world performance.Source: §1 · p. 1
  • StableVLA (0.5B) achieves 50% success on the Pack Doll task under physical visual corruptions, versus 20% for VLA-Adapter (0.5B) and 40% for OpenPI 0.5 (3B).The figure caption describes this as a real-world robot deployment comparison.Source: Figure 1 · p. 2
  • VLA-Adapter falls from 96% success by nearly 50% under disturbed inputs and reaches 0% under severe blur; OpenVLA, OpenVLA-OFT, OpenPi–0.5, and physical trials also show degradation.The introduction reports that vulnerability across models and in real-world experiments.Source: §1 · pp. 2–3
  • Noisy-input feature degradation is attributed to the vision-to-LLM projector, motivating IB-Adapter, an information-bottleneck block connecting the vision branch and LLM backbone.The contribution summary associates current VLA vulnerability with the projection module.Source: §1 · p. 3
  • IB-Adapter yields 35.2% average improvement across synthetic corruptions, a 31.7-percentage-point pick-and-place gain, and 20.4 percentage points in real-robot experiments.The 35.2% and 31.7-point results are reported for StableVLA; the 20.4-point result is reported under zero-shot settings with other experimental settings unchanged.Source: §1 · p. 3
  • Without extra training data, adapter replacement retains VLA-Adapter’s lightweight schedule and surpasses OpenVLA with 14× more parameters and π-0.5 trained on substantially more data.The introduction also cites the prohibitive combinatorial space of real-world corruptions and the risk that augmented training memorizes specific noise patterns.Source: §1 · pp. 2–3

2 Related Work and Preliminaries

Prior VLA robustness strategies rely on costly data-centric methods, while frozen vision encoders and simple projectors can let unseen visual noise propagate. StableVLA instead grounds modality alignment in the Information Bottleneck principle and uses attention-inspired channel filtering.

2.1 Robustness in Vision Language Models

  • Frozen vision encoders and simple MLP projectors can propagate visual corruption, while augmentation and domain randomization are costly and generalize poorly to unseen perturbations.StableVLA reconstructs the modality-alignment interface around the Information Bottleneck principle to filter perturbations without exhaustive noise-pattern simulation.Source: §2.1 · p. 3

2.2 Attention Mechanism from the Perspective of Information Bottleneck

  • IB theory links self-attention to iterative optimization [1] [35] [43], while XCiT and FAN [43] motivate channel-wise grouping that StableVLA applies through multi-head covariance.ViTs are more robust to visual corruptions than CNNs, a property linked to self-attention's semantic grouping of visual tokens [43].Source: §2.2 · p. 4

3 Method

StableVLA adds an Information Bottleneck Adapter to filter visual nuisances while preserving task-relevant semantics, then fuses it with an MLP to retain fine-grained spatial information for manipulation.

3.1 Modality Alignment From an Information Bottleneck Perspective

  • The IB formulation treats modality alignment as compressing visual tokens to filter nuisances while preserving task-relevant semantics, with channel-wise grouping motivated by noise and semantics being distributed across channels.This channel-wise filtering aims to suppress noisy channels and accentuate stable features before they reach the policy.Source: §3.1 · pp. 4–5

3.2 Information Bottleneck Adapter (IB-Adapter)

  • IB-Adapter models inter-channel covariance and uses independent sigmoid gates to suppress uncorrelated noise without forcing competition between channels, alongside nonlinear feature transformation.Its identity-key design grounds covariance in visual-token geometry to preserve high-frequency spatial cues.Source: §3.2 · pp. 5–6

3.3 Hybrid Architecture for Balancing Robust Semantics and High-frequency Details

  • The Fused IB-Adapter combines a denoising IB pathway with a high-fidelity MLP pathway to balance robust semantics and precise spatial execution.Stochastic Pathway Dropout is calibrated by task: pdrop ≈0 retains the MLP for spatially demanding LIBERO-Long tasks, while pdrop ≈0.3 encourages robust features for CALVIN and LIBERO-Object.Source: §3.3 · p. 6

4 Experiments

StableVLA improves robustness to visual corruptions across simulated benchmarks and real-world manipulation tasks. Feature visualizations and architecture ablations support the proposed IB-Adapter’s role in filtering noise.

4.1.1 Setup

  • Experiments used four LIBERO suites and zero-shot CALVIN, testing clean inputs and corruption severities 3–5 against VLA-Adapter, OpenVLA, OpenVLA-OFT, and OpenPi–0.5.StableVLA was trained from scratch on LIBERO and CALVIN without exposure to the evaluated corruptions; LIBERO reports success rates and CALVIN average completed tasks.Source: §4.1.1 · p. 7

4.1.2 Experiment Results.

  • At severity 5, StableVLA improves over VLA-Adapter by 40.2%–139.6% across four LIBERO suites and remains competitive with or surpasses large-scale baselines.Across corruption levels, it scores best or second best on LIBERO and completes more CALVIN tasks than VLA-Adapter; Figure 5a shows its advantage across most corruption categories.Source: §4.1.2 · p. 7, Figure 5 · p. 8
  • Under impulse noise, Fused IB-Adapter produces coherent object-centric semantic groupings, whereas the standard MLP’s features are diffuse and conflate task regions with background.The paper attributes this contrast to covariance-based Sigmoid gating suppressing low-correlation stochastic noise.Source: §4.1.2 · p. 8, Figure 5 · p. 8

4.2.1 Experiment Setup

  • Real-world evaluation covered four manipulation tasks over 10 trials each, testing Gaussian noise, defocus blur, oil, and plastic-cover obstructions against π0.5 and VLA-Adapter.The tasks span basic manipulation to long-horizon planning; high-severity Gaussian noise was excluded after preliminary tests showed catastrophic degradation across baselines.Source: §4.2.1 · pp. 8–9

4.2.2 Experiment Results

  • StableVLA maintains the smallest performance drop across all real-world tasks, with particular resilience to physical oil and plastic-cover interferences.π0.5 achieves high clean-data success rates but degrades under visual corruptions, while VLA-Adapter shows similar fragility.Source: §4.2.2 · p. 9, Table 2 · p. 9
  • Replacing Sigmoid with Softmax drops LIBERO corrupted-data performance by 16.3 percentage points and reduces CALVIN completed tasks from 2.13 to 0.46.The authors attribute Sigmoid’s advantage to independent channel suppression, unlike Softmax’s enforced competition.Source: §4.2.2 · p. 10, Table 3 · p. 10

5 Conclusions

IB-Adapter uses a dual-stream architecture guided by the Information Bottleneck principle to address VLA vulnerability to visual corruption. Despite a 14× smaller backbone and no external OpenX pre-training, it achieves robustness competitive with data-intensive 7B-scale state-of-the-art models.

5 Conclusions

  • Despite a 14× smaller backbone and no external OpenX pre-training, IB-Adapter achieves robustness competitive with data-intensive 7B-scale state-of-the-art models.Extensive evaluations show superior zero-shot robustness across diverse benchmarks.Source: §5 · p. 10
  • IB-Adapter is a general-purpose dual-stream architecture that combines high-fidelity spatial processing with covariance-based spectral filtering, guided by the Information Bottleneck principle.Source: §5 · p. 10
  • As a parameter-efficient and potentially model-agnostic component, IB-Adapter offers an alternative to standard projectors and promotes robust architectural inductive biases for embodied AI.Source: §5 · p. 10

without Extra Data … B Implementation Details

The theoretical derivation frames IB-Adapter as information-bottleneck compression of visual channels into semantic clusters. It compares categorical and independent Bernoulli assignments, with the latter allowing low-covariance noise channels to be gated near zero.

A Theoretical Derivation

  • The method treats each visual channel as a data point assigned to one of D semantic groups, using a smoothed Gaussian model and iterative soft assignments.The target of compression is the clean code S.Source: A Theoretical Derivation · p. 14
  • The derivation approximates cluster-conditional distributions with Gaussians and assumes shared covariance across clusters; the partition function determines normalization and the resulting attention mechanism.It assumes the smoothing parameter is sufficiently small.Source: A Theoretical Derivation · pp. 14–15
  • Under the categorical latent structure, each channel must be assigned to exactly one cluster, enforcing competition among cluster assignments.The cluster centers are used as the output representation.Source: A Theoretical Derivation · p. 15
  • Under the independent Bernoulli structure, each channel–cluster association is a separate binary decision, with a learnable bias representing the activation threshold.The output representation is defined from updated cluster centers, as in the categorical case.Source: A Theoretical Derivation · pp. 15–16
  • The derivation establishes IB-Adapter as an attention-like module with learnable WQ, WK, WV parameters and a learnable bias.Its queries are formed from the visual features and inverse shared covariance, while keys use the cluster centers.Source: A Theoretical Derivation · pp. 15–16
  • Independent Bernoulli assignments let each channel–cluster pair be evaluated separately, so uncorrelated noise channels with low covariance to semantic clusters receive near-zero gates.The derivation contrasts this with categorical assignments, which force channels to compete for a single cluster.Source: A Theoretical Derivation · p. 16

B.1 VLM Pre-training (Alignment Stage) … C.1 Full Comparison on LIBERO

StableVLA aligns Fused IB-Adapter visual tokens with the LLM embedding space before robotic fine-tuning, then evaluates methods on LIBERO grouped by training paradigm. The appendix also documents its fine-tuning setup, baseline protocols, and comprehensive quantitative results.

B.1 VLM Pre-training (Alignment Stage)

  • Before robotic fine-tuning, StableVLA aligns Fused IB-Adapter visual tokens with the LLM embedding space because its hybrid projector differs from standard open-source checkpoints.The alignment follows the Prismatic VLMs [19] protocol and uses LLaVA-LVIS4V-LRV for general-purpose visual reasoning.Source: §B.1 · p. 17

B.2 Robotic Fine-tuning & Baselines

  • After alignment, robotic fine-tuning uses Fused IB-Adapter’s dual pathways, controlled by fusion coefficient λ and optimized with Stochastic Pathway Dropout (pdrop).Hyperparameters were tailored across benchmark suites; Table 4 summarizes pre-training and fine-tuning configurations.Source: §B.2 · p. 17, Table 4 · p. 17
  • Baselines followed the evaluation protocol using official checkpoints or weights for OpenVLA [21], OpenVLA-OFT [22], and OpenPi (π0.5).VLA-Adapter [38] used official LIBERO checkpoints; for CALVIN it was retrained with official code under identical configurations, and peak performance was reported.Source: §B.2 · p. 17
  • 4.14 was the best VLA-Adapter [38] checkpoint score among evaluated stages, and was reported as the baseline’s upper-bound capability.The evaluated scores were 3.601, 4.14, 3.628, 3.92, and 4.097.Source: §B.2 · p. 17

C Detailed Experimental Results

C.1 Full Comparison on LIBERO

  • Table 5 presents the full LIBERO comparison by training paradigm and reports success rates, but the supplied passages do not provide method scores.The groups are OpenX Pretrain, OpenX + Web Co-train, and VLM Direct FT.Source: §C.1 · p. 18, Table 5 · p. 18

C.2 Per-Method Detailed Results · C.3 Ablation Study Details · D Radar Chart Details

The appendix supplies per-corruption results for the evaluated methods, architecture ablations, and details of how the radar-chart robustness scores are computed and indexed.

C.2 Per-Method Detailed Results

  • Per-method tables provide detailed results for each corruption type, reporting performance across all 19 corruption types (18 for tasks without Glass Blur) at severity levels 3, 4, and 5, along with clean (uncorrupted) performance.Source: §C.2 · p. 18
  • StableVLA and VLA-Adapter-Pro each have detailed LIBERO and CALVIN results, with VLA-Adapter-Pro identified as the strongest baseline sharing StableVLA’s VLM direct fine-tuning paradigm.The respective tables report LIBERO success rate and CALVIN average completed tasks.Source: §C.2 · p. 18, Table 6 · p. 19, Table 7 · p. 20
  • Separate detailed LIBERO results are provided for OpenVLA, OpenVLA-OFT, and OpenPI, the latter of which leverages internet-scale co-training.The tables report success rate (%).Source: §C.2 · p. 18, Table 8 · p. 20, Table 9 · p. 21

C.3 Ablation Study Details

  • The architecture ablation tables compare IB-Adapter using only the image bridge without feature fusion, and Fused IB-Adapter-softmax using softmax rather than sigmoid normalization.Both tables report LIBERO success rate and CALVIN average completed tasks.Source: §C.3 · p. 18, Table 11 · p. 22, Table 12 · p. 23

D Radar Chart Details

  • Radar-chart scores average clean-image and severity 3–5 results for each of 18 corruption types, then normalize scores so the best method equals 1.Table 13 lists the corruption-type indices; LIBERO-Spatial includes Glass Blur as index 19.Source: D Radar Chart Details · p. 24, Table 13 · p. 24

E Qualitative Results of Real-world Experiments · F Detailed Related Work and Preliminaries · F.1 Vision-Language-Action (VLA) Models

Figure 7 shows StableVLA successfully executing five real-world manipulation tasks; the related-work discussion motivates projector redesign because frozen vision encoders propagate corruption and standard MLP projectors lack intrinsic noise filtering.

E Qualitative Results of Real-world Experiments

  • Figure 7 shows StableVLA successfully executing sequences for five different real-world manipulation tasks.Source: Figure 7 · p. 24

F.1 Vision-Language-Action (VLA) Models

  • Using pre-trained VLMs [3] [10] [23] [26] [27] [39] [44] for robotic control has become a dominant embodied-intelligence paradigm [7] [21] [34].Source: §F.1 · p. 25
  • Training these models from scratch relies on massive datasets such as Open X-Embodiment and AgiBot, requiring substantial computational resources.Source: §F.1 · p. 25
  • VLA-Adapter [38] bypasses pre-training on large-scale datasets to transfer VLMs’ general perceptual capabilities to specific robotic domains efficiently.Source: §F.1 · p. 25
  • Because standard VLA models typically freeze the vision encoder [32] [41], input-level noise or corruption propagates through the visual backbone.Source: §F.1 · p. 25
  • Standard MLP projectors preserve spatial details but lack intrinsic filtering of task-irrelevant nuisances, motivating this work’s noise-suppressing projector redesign.Source: §F.1 · p. 25

F.2 Robustness in Vision and Robotics · F.3 Attention Mechanism from the Perspective of Information Bottleneck

StableVLA targets unseen visual perturbations through an Information Bottleneck architecture rather than exhaustive noise-pattern simulation. It extends channel-wise information filtering to VLA modality alignment for robust embodied decisions.

F.2 Robustness in Vision and Robotics

  • Robustness in VLA policies is tied to success rate under visual perturbations, while classification commonly uses mean classification error on benchmarks such as ImageNet-C [16].ImageNet-C includes noise, blur, weather, and digital corruptions.Source: §F.2 · p. 25
  • Data augmentation is a mainstream robustness strategy, with simulated corruptions in computer vision [17] and Domain Randomization in robotic learning [36].Domain Randomization randomly perturbs visual textures or physical parameters in simulation.Source: §F.2 · p. 25
  • StableVLA uses Information Bottleneck-based modality alignment to filter visual perturbations without exhaustive noise-pattern simulation.The paper presents this as intrinsic robustness, contrasting it with data-centric approaches that require costly augmented datasets and may memorize specific noise patterns.Source: §F.2 · p. 25

F.3 Attention Mechanism from the Perspective of Information Bottleneck

  • [43] proves that, under Gaussian assumptions, iterative Information Bottleneck optimization is mathematically equivalent to self-attention.The passage relates this result to self-attention's visual grouping, in which tokens aggregate into semantic clusters.Source: §F.3 · p. 25
  • Vision Transformers exhibit greater robustness than CNNs against various corruptions, a property attributed to self-attention-driven visual grouping [43].The passage describes tokens aggregating into semantic clusters.Source: §F.3 · p. 25
  • StableVLA extends channel-wise information filtering to modality alignment with a multi-head covariance mechanism for robust semantic propagation in embodied decisions.The mechanism filters noisy channels.Source: §F.3 · p. 25
  • FAN [43] frames Cross-Covariance Attention as subspace clustering that uses Information Bottleneck across channels to identify semantic subspaces and suppress noisy channels.The passage says [2] introduced Cross-Covariance Attention to compute channel-wise interactions and reduce computational complexity.Source: §F.3 · p. 25

Related papers

Loading 2605.18287v1…