PaperlayerView exact revision on arXiv

Source-linked AI summary

StableVLA: Towards Robust Vision-Language-Action Models without Extra Data

Yiyang Fu, Chubin Zhang, Shukai Gong, Yufan Deng, Kaiwei Sun, Qiyang Min, Qibin Hou, Yansong Tang, Jianan Wang, Daquan Zhou

arXiv:2605.18287v1cs.CVcs.RO

TL;DR

VLA models face a robustness gap when deployed with visual disturbances absent from curated training and benchmark conditions. StableVLA addresses this with an information-bottleneck adapter that filters visual noise, improving performance by 35.2% in simulation and 20.4 percentage points on real-robot experiments without extra data.

  • Problem

    Existing VLA evaluations emphasize idealized visual conditions, leaving robustness to real-world disturbances such as sensor noise and motion blur insufficiently characterized.

  • Method

    The IB-Adapter uses information-bottleneck-based channel selection to suppress task-irrelevant visual noise during vision-language alignment.

  • Results

    35.2% improvement in simulation and 20.4 percentage points on real-robot experiments are achieved over the baseline under zero-shot visual-corruption settings.

  • Takeaways & Limitations

    Despite using a 14× smaller backbone without external OpenX pre-training, StableVLA achieves robustness competitive with data-intensive 7B-scale state-of-the-art models.

  • Takeaways & Limitations

    Glass Blur is excluded from evaluation on most LIBERO and CALVIN tasks because of its prohibitive computational cost during interaction.

Abstract

from arXiv · show

It is infeasible to encompass all possible disturbances within the training dataset. This raises a critical question regarding the robustness of Vision-Language-Action (VLA) models when encountering unseen real-world visual disturbances, particularly under imperfect visual conditions. In this work, we conduct a systematic study based on recent state-of-the-art VLA models and reveal a significant performance drop when visual disturbances absent from the training data are introduced. To mitigate this issue, we propose a lightweight adapter module grounded in information theory, termed the Information Bottleneck Adapter (IB-Adapter), which selectively filters potential noise from visual inputs. Without requiring any extra data or augmentation strategies, IB-Adapter consistently improves over the baseline by an average of 30%, while adding fewer than 10M parameters, demonstrating notable efficiency and effectiveness. Furthermore, even with a 14x smaller backbone (0.5B parameters) and no pre-training on the Open X-Embodiment dataset, our model StableVLA achieves robustness competitive with 7B-scale state-of-the-art VLAs. With negligible parameter overhead (<10M), our approach maintains accuracy on long-horizon tasks and surpasses OpenPi under both synthetic and physical visual corruptions.

1 Introduction

The introduction identifies a major gap between VLA performance in idealized benchmarks and robustness under real-world visual disturbances, linking this vulnerability to the vision-to-LLM projection module. It proposes the data-free IB-Adapter, which improves corruption robustness while preserving lightweight training and model size.

  • Motivation: 96% clean-input success can drop by nearly 50% under disturbed inputs and reach 0% under certain corruptions.This result is reported for VLA-Adapter evaluated with synthetic natural visual corruptions.
  • Method: IB-Adapter uses information bottleneck theory to selectively connect the vision branch and LLM backbone without extra data.It replaces the original adapter and is retrained under the same settings.
  • Results: 35.2% average performance improvement across synthetic visual corruptions is achieved, while real-robot pick-and-place performance improves by 31.7 percentage points.The synthetic result comes from replacing VLA-Adapter’s original adapter; the real-robot result is reported for the pick-and-place task.
  • Motivation: Current state-of-the-art VLA models are highly vulnerable to visual disturbances despite strong performance under clean benchmark conditions.The vulnerability appears in both synthetic and real-robot scenarios.
  • Analysis: The observed vulnerability is closely associated with the projection module bridging the vision encoder and LLM backbone.The analysis reports substantial feature degradation under noisy inputs and motivates intervention at this connection.
  • Results: StableVLA consistently outperforms selected strong baselines across LIBERO, CALVIN, and real-robot evaluations while maintaining a significantly smaller model size.Evaluated baselines include VLA-Adapter, OpenVLA, OpenVLA-OFT, and π.

2 Related Work and Preliminaries

Pre-trained vision-language models have become a dominant paradigm for robotic control, but training from scratch typically requires massive datasets and substantial computation. Vision Transformers show greater robustness to visual corruptions, linked to self-attention’s semantic token grouping and its connection to Information Bottleneck optimization.

  • Vision-Language-Action Models: Pre-trained VLMs are a dominant paradigm for robotic control, while training from scratch typically requires massive datasets and substantial computational resources.The passage names Open X-Embodiment, DROID, and AgiBot as examples of such datasets.
  • Information Bottleneck and Visual Robustness: Vision Transformers are more robust to visual corruptions than CNNs, attributed to self-attention aggregating tokens into semantic clusters.Under Gaussian assumptions, self-attention is theoretically equivalent to iterative Information Bottleneck optimization.

3 Method

StableVLA formulates VLA modality alignment as an Information Bottleneck problem and introduces IB-Adapter to suppress channel-wise visual nuisances while preserving task-relevant semantics. Fused IB-Adapter combines denoising with a high-fidelity pathway to retain spatial details needed for precise, long-horizon manipulation.

  • Information Bottleneck Formulation: Existing MLP projectors act as all-pass filters, indiscriminately maximizing I(Xv; Z) instead of filtering task-irrelevant visual perturbations.The IB formulation defines Z as a compressed representation that filters nuisances while retaining the clean task-relevant code S; β controls this trade-off.
  • Information Bottleneck Formulation: IB-Adapter applies Information Bottleneck optimization across channels because visual semantics and noise are heterogeneously distributed across the encoder’s output channels.The method dynamically suppresses noisy channels and accentuates stable features before representations reach the policy model.
  • IB-Adapter: IB-Adapter combines subspace covariance modeling, sigmoid-based gating, and nonlinear feature transformation to identify robust semantic signals and filter visual nuisances.Its multi-head design captures correlations across semantic subspaces, while sigmoid gating permits independent channel selection under a Bernoulli latent assumption.
  • Fused IB-Adapter: Fused IB-Adapter addresses high-frequency detail attenuation by decoupling robust semantic understanding from precise spatial execution through dual parallel pathways.The standard MLP path preserves raw high-frequency details, while the IB-Adapter path extracts covariance-filtered semantic features.
  • Fused IB-Adapter: SPD calibrates pathway retention during fine-tuning, with pdrop ≈0 recommended for extreme spatial-fidelity tasks such as LIBERO-Long.In these tasks, the IB-Adapter serves as a robustness residual while the MLP pathway preserves cues required for precise execution.

4 Experiments

Experiments evaluate StableVLA under synthetic and physical visual corruptions across LIBERO, CALVIN, and real-world manipulation tasks. StableVLA consistently improves robustness over baselines, while feature visualizations and ablations support the Fused IB-Adapter’s mechanism and design.

  • Synthetic robustness: StableVLA achieves superior zero-shot robustness across LIBERO and CALVIN corruption settings, typically ranking best or second best against large-scale VLA baselines.LIBERO reports per-task success rates, while CALVIN reports average completed tasks, with metrics averaged over corruption types.
  • Synthetic robustness: StableVLA surpasses VLA-Adapter across most corruption categories and remains competitive with or better than OpenPi–0.5 and OpenVLA-OFT despite a smaller backbone.The comparison spans all four LIBERO task suites and supports robust semantic feature extraction by the IB-Adapter.
  • Mechanism analysis: K = 2 clustering shows standard MLP outputs as diffused, whereas Fused IB-Adapter outputs retain coherent semantic grouping under impulse noise.The visualization uses output features from LIBERO-Object manipulation scenes corrupted by impulse noise.
  • Real-world robustness: StableVLA maintains the smallest performance drop across real-world tasks under Gaussian noise, defocus blur, Oil, and Shelter physical interferences.Real-world evaluation uses the Astribot S1 and four manipulation tasks, with each model tested over 10 trials per task and corruption setting.
  • Ablation: 16.3 percentage points is the corrupted-data drop for Softmax versus Sigmoid on LIBERO, while CALVIN average completed tasks fall from 2.13 to 0.46.The result supports the proposed independent Bernoulli latent structure and the use of Sigmoid activation in Fused IB-Adapter.

5 Conclusions

The paper addresses VLA modality alignment vulnerabilities under visual corruptions by introducing the IB-Adapter, an Information Bottleneck-based dual-stream architecture. Evaluations show superior zero-shot robustness across diverse benchmarks.

  • The work targets a notable vulnerability of VLA modality alignment to visual corruptions.
  • The IB-Adapter is a general-purpose dual-stream architecture guided by the Information Bottleneck principle.It combines high-fidelity spatial processing with covariance-based spectral filtering.
  • Extensive evaluations confirm superior zero-shot robustness across diverse benchmarks.

without Extra Data … F Detailed Related Work and Preliminaries

The supplied material is a contents outline spanning appendices, theoretical derivation, detailed experiments, and related work and preliminaries. It lists implementation, comparison, ablation, qualitative, and background subsections without providing their substantive text.

  • C Detailed Experimental Results: Detailed experimental results cover full LIBERO comparisons, per-method results, and ablation-study details.These are listed as C.1, C.2, and C.3.
  • C Detailed Experimental Results: The experimental-results section additionally lists radar-chart details and qualitative results from real-world experiments.These are identified as sections D and E in the contents.
  • F Detailed Related Work and Preliminaries: Detailed related work and preliminaries address vision-language-action models and robustness in vision and robotics.These topics are listed as F.1 and F.2.
  • F Detailed Related Work and Preliminaries: The related-work and preliminaries material also includes attention mechanisms from an information-bottleneck perspective.This topic is listed as F.3.

A Theoretical Derivation · B Implementation Details

The section derives the IB-Adapter by clustering visual channels under categorical or independent Bernoulli latent structures, then establishes its attention-based functional form. It argues that the Bernoulli formulation filters uncorrelated noise through near-zero gates while preserving semantic signals.

  • A Theoretical Derivation: The derivation treats each visual channel as a data point clustered into D semantic groups while optimizing an information-bottleneck representation Z.The visual encoder output is X = [c1, ..., cD] ∈ R^N×D, and the target clean code is S.
  • A Theoretical Derivation: The channel-clustering formulation assumes p(s|j) = N(s|cj, ϵ^2I), uses soft assignments, and approximates p(s|c) with Gaussian cluster conditionals.The approximation assumes sufficiently small ϵ and yields a closed-form KL divergence.
  • A Theoretical Derivation: The derivation absorbs cluster-independent terms into the partition function and assumes shared covariance Σc = Σ across clusters.This simplification follows from the quadratic form in the KL divergence.
  • A Theoretical Derivation: The partition function determines normalization and the resulting attention mechanism, motivating separate categorical and independent Bernoulli latent cases.The categorical case enforces exactly one cluster assignment per channel.
  • A Theoretical Derivation: The categorical formulation updates cluster centers into the output representation and expresses the computation through Q, K, V, and learnable bias parameters.Here, Q = WQX, K contains the previous and current cluster centers, and WQ, WK, WV are learnable.
  • A Theoretical Derivation: The independent Bernoulli formulation assigns each channel-cluster pair an independent binary latent variable rather than enforcing exclusive cluster membership.The variable ajc ∈ {0, 1} indicates whether channel j is associated with cluster c, with b representing the off-state activation threshold.
  • A Theoretical Derivation: The Bernoulli case similarly produces updated cluster-center representations and the same Q, K, V parameterization, establishing the proposed IB-Adapter functional form.The formulation includes learnable parameters WQ, WK, WV and bias b.
  • A Theoretical Derivation: The categorical structure forces channel competition, whereas independent Bernoulli gating lets each channel-cluster pair be evaluated independently and suppresses uncorrelated noise.Noise channels have low covariance with semantic clusters, producing gate values near zero (σ ≈0) without suppressing legitimate semantic signals.

B.1 VLM Pre-training (Alignment Stage) … C.1 Full Comparison on LIBERO

StableVLA aligns Fused IB-Adapter visual tokens with the LLM embedding space before robotic fine-tuning, then evaluates the resulting model against established baselines and training paradigms on LIBERO. The appendix reports comprehensive quantitative comparisons and documents the evaluation configurations used across stages and methods.

  • B.1 VLM Pre-training (Alignment Stage): StableVLA performs Vision-Language Alignment before robotic fine-tuning because its Fused IB-Adapter projector differs from standard open-source checkpoints.The alignment targets the LLM’s embedding space for visual tokens produced by the hybrid Fused IB-Adapter projector.
  • B.1 VLM Pre-training (Alignment Stage): The alignment stage strictly follows the Prismatic VLMs protocol and uses the LLaVA-LVIS4V-LRV dataset.This setup is intended to ensure general-purpose visual reasoning capabilities.
  • B.2 Robotic Fine-tuning & Baselines: After alignment, StableVLA is fine-tuned for robotic manipulation using Fused IB-Adapter’s dual-pathway mechanism.The pathways are controlled by fusion coefficient λ and optimized with Stochastic Pathway Dropout (pdrop).
  • B.2 Robotic Fine-tuning & Baselines: The evaluation aligns all baselines with a common protocol for fair and reproducible comparison, including OpenVLA, OpenVLA-OFT, VLA-Adapter, and OpenPi.Official checkpoints or weights are used where available; VLA-Adapter is retrained for CALVIN under identical configurations.
  • B.2 Robotic Fine-tuning & Baselines: 4.14 is the reported VLA-Adapter baseline result, selected as the upper bound after evaluating checkpoints with scores of 3.601, 4.14, 3.628, 3.92, and 4.097.The reported value is the best result across checkpoints sampled along the convergence trajectory.
  • C Detailed Experimental Results: The appendix provides comprehensive quantitative results for all evaluated methods and presents a complete LIBERO comparison grouped by training paradigm.Table 5 reports success rate (%) across OpenX Pretrain, OpenX + Web Co-train, and VLM Direct FT methods.

C.2 Per-Method Detailed Results · C.3 Ablation Study Details · D Radar Chart Details

The supplementary results detail per-corruption performance for multiple VLA methods and adapter variants across LIBERO and CALVIN, while radar charts summarize normalized robustness across corruption types. The ablation comparisons isolate the image-bridge design and fusion normalization choices.

  • C.2 Per-Method Detailed Results: Detailed results span 19 corruption types, or 18 for tasks without Glass Blur, at severity levels 3–5 and under clean conditions.These tables provide per-corruption-type performance for each method.
  • C.2 Per-Method Detailed Results: StableVLA results are reported on both LIBERO and CALVIN benchmarks.Table 6 contains the detailed results for StableVLA.
  • C.2 Per-Method Detailed Results: VLA-Adapter-Pro is evaluated as the strongest baseline sharing StableVLA’s VLM direct fine-tuning paradigm.Its detailed LIBERO and CALVIN results appear in Table 7.
  • C.2 Per-Method Detailed Results: OpenVLA, OpenVLA-OFT, and OpenPI receive separate detailed evaluations, with OpenPI using internet-scale co-training.OpenVLA and its orthogonal fine-tuning variant are reported on LIBERO, as is OpenPI.
  • C.3 Ablation Study Details: The ablation study examines adapter architecture design through detailed comparisons of alternative variants.The reported metrics are LIBERO success rate (%) and CALVIN average completed tasks.
  • C.3 Ablation Study Details: IB-Adapter uses only the image-bridge component without feature fusion, whereas Fused IB-Adapter-softmax replaces sigmoid with softmax normalization in fusion.These variants are reported in Tables 11 and 12.
  • D Radar Chart Details: Figure 5a summarizes robustness across 18 corruption types by averaging clean-image and severity 3–5 scores, then normalizing each corruption type so the best method equals 1.The corruption-type indices are provided in Table 13; LIBERO-Spatial includes Glass Blur as index 19.

E Qualitative Results of Real-world Experiments · F Detailed Related Work and Preliminaries · F.1 Vision-Language-Action (VLA) Models

StableVLA is shown executing five real-world manipulation tasks successfully, while the related work frames VLA development around resource-efficient training and the robustness limitations of frozen vision encoders and simple MLP projectors.

  • E Qualitative Results of Real-world Experiments: StableVLA successfully executes five different real-world robot manipulation tasks.Figure 7 presents successful execution sequences for all five tasks.
  • F.1 Vision-Language-Action (VLA) Models: Pre-trained Vision-Language Models have become a dominant paradigm for robotic control in embodied intelligence.The passage cites prior work using pre-trained VLMs for robotic control.
  • F.1 Vision-Language-Action (VLA) Models: Training VLA models from scratch requires massive datasets such as Open X-Embodiment and AgiBot, creating substantial computational demands.The passage identifies both datasets as examples of the scale needed for pre-training from scratch.
  • F.1 Vision-Language-Action (VLA) Models: VLA-Adapter proposes a resource-efficient alternative architecture to mitigate the computational burden of large-scale pre-training.This approach is presented as diverging from standard paradigms.
  • F.1 Vision-Language-Action (VLA) Models: Standard VLA models typically freeze the vision encoder during end-to-end training to preserve semantic priors.The passage attributes this design to existing VLA models and cites prior work.
  • F.1 Vision-Language-Action (VLA) Models: Frozen vision encoders allow input-level noise or corruption to propagate through the visual backbone, exposing a gap in architectural robustness.The passage identifies this propagation as a critical limitation despite advances in training efficiency.
  • F.1 Vision-Language-Action (VLA) Models: Existing VLA models use simple MLP projectors to align visual features with the downstream policy’s action space.The passage describes these projectors as the interface intended to suppress disturbances before downstream effects.

F.2 Robustness in Vision and Robotics · F.3 Attention Mechanism from the Perspective of Information Bottleneck

StableVLA targets unseen visual-corruption robustness through intrinsic architectural design rather than data augmentation. Its information-bottleneck perspective motivates channel-wise semantic grouping that suppresses noisy visual features during VLA modality alignment.

  • F.2 Robustness in Vision and Robotics: Robotic perceptual robustness is evaluated through policy success rates under visual perturbations, unlike computer vision’s Mean Classification Error metric.Relevant perturbations include noise, blur, weather, and digital corruptions.
  • F.2 Robustness in Vision and Robotics: Data augmentation and Domain Randomization improve robustness by simulating visual or physical perturbations during training.These strategies are used in computer vision and robotic learning, respectively.
  • F.2 Robustness in Vision and Robotics: Data-centric methods impose substantial training costs and may memorize specific noise patterns, limiting generalization to unseen corruption types.StableVLA therefore pursues intrinsic robustness through architectural design.
  • F.3 Attention Mechanism from the Perspective of Information Bottleneck: Vision Transformers show superior corruption robustness compared with CNNs, attributed to self-attention’s aggregation of tokens into semantic clusters.This visual grouping is connected to the Information Bottleneck principle’s trade-off between input compression and relevant-information preservation.
  • F.3 Attention Mechanism from the Perspective of Information Bottleneck: Recent methods extend visual grouping from spatial attention to channel interactions, including XCiT and Cross-Covariance Attention.Cross-Covariance Attention significantly reduces computational complexity by computing channel-wise interactions.
  • F.3 Attention Mechanism from the Perspective of Information Bottleneck: FAN interprets channel-wise attention as subspace clustering that identifies coherent semantic subspaces while suppressing noisy channels under the Information Bottleneck principle.StableVLA extends this insight to VLA modality alignment with a multi-head covariance mechanism.
Loading 2605.18287v1…