PaperlayerView exact revision on arXiv

Source-linked AI summary

DS-VLA: A Dendritic-inspired Vision-Language-Action Model for Robust Action Control

Yaxing Lyu, Jingyi Li, Mingkun Xu, Yujie Wu

arXiv:2609.32253v1cs.ROcs.AIcs.NE

TL;DR

Nominal VLA success does not necessarily translate to robust closed-loop behavior when executed actions are corrupted. DS-VLA incorporates dendritic spiking dynamics and inhibitory gating into VLA control. It reports 91.6% average nominal success and 87.35% perturbed success on LIBERO, retaining 95.4% of nominal performance.

  • Problem

    Nominal task success does not establish whether a VLA policy can recover closed-loop behavior after action corruption alters later states.

    Source: Introduction · p. 1, Main Results · p. 5

  • Method

    DS-VLA uses sparsely connected dendritic branches with distinct temporal responses and a neuron-wise inhibitory gate to regulate evidence admitted into action states.

    Source: Introduction · p. 2

  • Results

    91.6% average nominal success and 87.35% average perturbed success; DS-VLA retains 95.4% of nominal performance on LIBERO.

    Source: Conclusion · p. 7

  • Takeaways & Limitations

    DS-VLA associates selective evidence regulation and behavioral recovery with robust performance under the evaluated action perturbations.

    Source: Introduction · p. 2, Conclusion · p. 7

    At Difficulty 3, perturbations can create unrecoverable states requiring a new task-level strategy beyond the action head's correction of moderate deviations.

    Source: Robustness under Increasing Difficulty · p. 7

Abstract

from arXiv · show

Vision-language-action (VLA) models have achieved strong performance in language-conditioned manipulation, yet success under nominal evaluation does not necessarily translate into robust closed-loop behavior when executed actions are transiently corrupted. We introduce DS-VLA, a dendritic-inspired action architecture that incorporates dendritic spiking dynamics into VLA control to address this limitation. Specifically, to enable modularized feature processing and temporal information integration, DS-VLA equips action neurons with multiple sparsely connected dendritic branches, each featuring heterogeneous, learned decay factors. Furthermore, to suppress unreliable state updates while preserving task-relevant historical information, we introduce a neuron-wise inhibitory gate that adaptively regulates the admission of new multimodal evidence into dendritic states prior to somatic dynamics. We evaluate DS-VLA on all four LIBERO suites under both nominal rollouts and a unified closed-loop action-perturbation protocol. DS-VLA achieves a 91.6\% average nominal success rate and an 87.35\% average perturbed success rate, retaining 95.4\% of its nominal performance. Under the same reported perturbation setting, OpenVLA-OFT, FAST, $\pi_0$, and GR00T achieve 39.45\%, 23.90\%, 28.55\%, and 30.75\%, respectively. A controlled ablation isolates the contribution of neuron-wise shared inhibition, while analyses of neural dynamics and post-perturbation trajectories associate robust performance with selective evidence suppression and effective behavioral recovery. Together, these results demonstrate that integrating brain-inspired computational mechanisms offers a promising architectural prior for robust embodied intelligence beyond merely scaling vision-language backbones or generative action decoders.

Introduction

Nominal task success does not establish closed-loop robustness when execution errors alter later robot, object, and sensory states. DS-VLA brings dendritic integration, spiking dynamics, and inhibitory modulation into VLA action generation and evaluates the design under task and perturbation settings.

Introduction

  • Action disturbances can shift subsequent observations away from the intended trajectory, so nominal success alone does not reveal closed-loop robustness.Robust control must preserve task intent, limit disruptive state updates, and adapt actions to the changed environment.Source: Introduction · p. 1
  • Dendritic branches locally integrate inputs before somatic combination, offering branch-wise organization across input subspaces and timescales.The paper motivates sparse branch-specific receptive fields, distinct dendritic states, and somatic integration as a dynamical prior for action modeling.Source: Introduction · p. 2, Figure 2 · p. 2
  • The proposed inhibitory gate is motivated by dendritic inhibition, while its roles in selecting action-relevant evidence and limiting perturbation-induced updates are empirical hypotheses.The paper notes that dendritic inhibition can strongly influence excitability and gate burst-related dynamics.Source: Introduction · p. 2
  • DS-VLA combines multiple dendritic branches with distinct temporal responses, somatic spiking dynamics, and a neuron-wise gate shared by each neuron's two branches.The gate regulates incoming evidence to reduce irrelevant or phase-inconsistent input and limit anomalous state updates while protecting accumulated task information.Source: Introduction · p. 2
  • DS-VLA's evaluation framework covers standard tasks, multiple perturbation levels, and controlled architectural ablation; it reports retention of over 95% of nominal performance under standardized action perturbations.The framework examines task performance, perturbation resilience, and inhibitory gating's role.Source: Introduction · p. 2

Related Work

Related work spans varied VLA action-generation designs, methods for robustness in closed-loop control, and biological and computational studies of dendritic processing and inhibition. The paper identifies systematic investigation of these mechanisms in embodied intelligence as an open area.

Vision–Language–Action Models

  • VLA action generation ranges from FAST frequency-space tokens and OpenVLA-OFT parallel action chunks to π0 flow matching and GR00T dual-system inference.Prior work also includes modular development frameworks and a neuromorphic hierarchy spanning planning, stabilization, and spiking execution.Source: Vision–Language–Action Models · pp. 2–3

Closed-Loop Robustness in VLA Control

  • Closed-loop robustness research includes training randomization, online motor adaptation, generative controllers, corruption curricula, and optimization against worst-case action noise.The paper focuses on how policies adapt subsequent actions to new observations after action disturbances change the physical state.Source: Closed-Loop Robustness in VLA Control · p. 3

Spiking Neural Networks, Dendritic Computation, and Inhibitory Modulation

  • Prior neuroscience and spiking-network work motivates dendritic heterogeneity and localized inhibition, but their computational use in embodied intelligence has yet to be systematically investigated.This work connects branch-related regulation and temporal dendritic processing with practical embodied control.Source: Spiking Neural Networks, Dendritic… · p. 3

Method

DS-VLA conditions multimodal action features on robot state before processing them with a dendritic spiking action head. The head uses branch-specific temporal integration and shared neuron-wise inhibition, then decodes continuous actions from membrane states.

Multimodal and State-Conditioned Features

  • A VLM and action queries extract multimodal features, while a GRU encodes robot-state history and FiLM modulates the query features before the dendritic action head.The method uses two prediction rounds, updating the state tensor after the first prediction before producing the final action sequence.Source: Method · p. 3, Multimodal and State-Conditioned… · pp. 3–4

Method

  • The overall method maps visual observations, language, and robot-state history to a continuous action sequence through two state-conditioned prediction rounds and a dendritic spiking action head.The model is optimized end to end with L1 action regression and uses no auxiliary gate, firing-rate, or dendritic regularization loss.Source: Method · p. 3, Continuous Action Readout and Training · p. 5

Sparse Dendritic Integration and Inhibitory Gating

  • Each somatic neuron uses two branches with complementary masked inputs and a shared, input-dependent inhibitory coefficient that regulates new evidence while preserving relative branch organization.The coefficient modulates incoming evidence; branch-specific decay factors provide heterogeneous temporal responses.Source: Sparse Dendritic Integration and… · p. 4, Dendritic and Somatic Dynamics · p. 4

Dendritic and Somatic Dynamics

  • Three dendritic spiking layers integrate branch states at the soma, apply learned somatic decay and a bounded membrane skip, then generate spikes with subtractive reset.The somatic decay is learned independently for each neuron and initialized in [0.95, 0.99].Source: Dendritic and Somatic Dynamics · pp. 4–5

Continuous Action Readout and Training

  • Continuous actions are decoded from the cumulative temporal mean of the final-layer membrane state, retaining subthreshold and temporal information beyond the binary spike representation.Neural states propagate across query positions, with gradients detached every eight internal steps without resetting states.Source: Continuous Action Readout and Training · p. 5

Experiments

The experiments assess DS-VLA on standard manipulation, action perturbations of increasing severity, and inhibitory-gating ablations. Success rate is the primary metric, with successful-trajectory and action-head analyses as complements.

Experiments

  • The evaluation covers standard manipulation, action perturbations of increasing severity, and inhibitory-gating ablations.Source: Experiments · p. 5
  • Success rate is the primary evaluation metric.Source: Experiments · p. 5
  • Successful-trajectory and action-head analyses complement the success-rate metric.Source: Experiments · p. 5

Experimental Setup

The evaluation covers four LIBERO suites, compares DS-VLA with four action-generation baselines, and measures task success under nominal execution and progressively stronger action perturbations.

Benchmark and Tasks

  • Each LIBERO suite has ten tasks and 50 episodes, for 2,000 episodes total, with suite-specific horizons ranging from 220 to 520 steps.The suites target spatial, object, goal, and long-horizon generalization; images are resized to 224×224 and robot-state history contains 16 observations.Source: Benchmark and Tasks · p. 5

Baselines

  • Four baselines—OpenVLA-OFT, FAST, π0, and GR00T—share task definitions, episode counts, and the success interface with DS-VLA.The baselines represent parallel continuous regression, autoregressive action tokens, flow matching, and dual-system diffusion control; the ungated variant isolates inhibitory gating.Source: Baselines · p. 5

Evaluation Metric

  • Success rate is the primary metric, aggregated within each suite and macroaveraged across the four suites.It counts successful episodes relative to evaluated episodes.Source: Evaluation Metric · p. 5

Action-Perturbation Protocol

  • The perturbation protocol injects translation/rotation noise, overwrites translation commands, and forces gripper closure for increasingly long periods across three difficulty levels.Difficulty 1 also flips the gripper command during steps 10–19; the protocol provides no perturbation indicator or hand-designed recovery signal.Source: Action-Perturbation Protocol · p. 5

Main Results

DS-VLA retains strong task success under action perturbation, while its nominal performance remains competitive. Trajectory examples and the described mechanisms connect this robustness to recovery and selective regulation of state updates.

Main Results

  • 91.6% average nominal success and 98.6% on Object leave DS-VLA behind OpenVLA-OFT, FAST, and π0 on average but ahead of GR00T.The comparison is reported for standard LIBERO tasks.Source: Main Results · p. 5, Table 1 · p. 5
  • 87.35% average success under Difficulty 1 retains 95.4% of DS-VLA’s clean performance, with only 4.25 points of degradation versus at least 55.75 for baselines.DS-VLA exceeds the strongest perturbed baseline by 47.9 points and reaches 77.8% on Long, while all baselines remain at or below 1.2%.Source: Main Results · p. 5, Table 2 · p. 6
  • On the same Goal task, the successful trajectory compensates after perturbation and completes placement, whereas the failed trajectory does not recover before termination.Figure 4 marks injected action displacement with red dashed arrows.Source: Main Results · p. 5, Figure 4 · p. 6
  • DS-VLA combines branch-specific states with an input-dependent inhibitory coefficient shared across each neuron’s two branches, limiting abrupt updates while preserving task-relevant information.The described results support combining dendritic temporal integration with neuron-wise inhibitory regulation.Source: Main Results · pp. 5–6

Robustness under Increasing Difficulty

DS-VLA success declines as action perturbations become longer and more severe. Among successful trajectories, recovery involves longer completion horizons while per-step computation and action variation remain stable; the most severe setting can create unrecoverable states.

Robustness under Increasing Difficulty

  • 87.35% success at the lower perturbation level falls to 71.65% and then 46.00% as disturbances become longer and more severe.Figure 5 reports DS-VLA success rates across increasing perturbation difficulty.Source: Robustness under Increasing Difficulty · p. 6, Figure 5 · p. 6
  • Completion horizon grows from 145.4 to 175.4 steps as severity increases, while action frequency, latency, and action variation remain stable.The analysis covers successful trajectories; recovery therefore relies primarily on additional closed-loop corrections rather than increased per-step computation or action magnitude.Source: Robustness under Increasing Difficulty · p. 6, Table 3 · p. 6
  • Branch-specific states and heterogeneous decay provide multiple timescales, while inhibitory gating limits abnormal overwriting during repeated corrections.The reported trajectory statistics describe recovery rather than unconditional efficiency.Source: Robustness under Increasing Difficulty · p. 6
  • At Difficulty 3, forced actions may create unrecoverable states that require a new task-level strategy beyond dendritic dynamics and inhibitory gating.The paper identifies stronger failure recognition, recovery reasoning, and replanning as complementary capabilities.Source: Robustness under Increasing Difficulty · p. 7

Inhibitory-Gating Ablation

The inhibitory-gating ablation shows that removing the gate lowers both clean and perturbed success, with the largest gains from gating on Long tasks.

Inhibitory-Gating Ablation

  • Removing inhibitory gating lowers clean and perturbed averages by 18.0 and 20.9 percentage points, respectively.Figure 6 reports success rates for the inhibitory-gating ablation.Source: Inhibitory-Gating Ablation · p. 7, Figure 6 · p. 7
  • On Long, inhibitory gating improves clean and perturbed success by 48.4 and 42.2 points, respectively.The ablation retains dendritic compartments and spiking dynamics, isolating the added value of input-dependent inhibition.Source: Inhibitory-Gating Ablation · p. 7
  • Perturbed Object is the sole exception, with the ungated variant scoring 0.8 percentage points higher.Source: Inhibitory-Gating Ablation · p. 7

Action-Head Efficiency

The action-head analysis reports lower DS-VLA parameter and dense-MAC counts than OFT, alongside estimated arithmetic-energy savings and measured GPU power.

Action-Head Efficiency

  • The cost comparison includes DS-VLA, OpenVLA-OFT, and GR00T action heads, while FAST is excluded because it has no separable action head.The arithmetic-energy proxy assumes firing rates of 0.10 and 0.05 and is not measured GPU energy.Source: Action-Head Efficiency · p. 7
  • 57.8% fewer parameters and dense MACs than OFT are reported for DS-VLA.Under ideal event-driven assumptions, estimated arithmetic energy is 5.90× lower than OFT and 1,377.17× lower than ten-step GR00T.Source: Action-Head Efficiency · p. 7, Table 4 · p. 7
  • On an RTX 4090 at batch size one, DS-VLA draws 104.27 W on average, versus 248.13 W for OFT and 109.64 W for GR00T.The authors note possible additional event-driven hardware gains depend on spike sparsity, hardware mapping, and native deployment.Source: Action-Head Efficiency · p. 7

Conclusion

DS-VLA combines dendritic computation and inhibitory regulation for robot control, with reported success under nominal and perturbed LIBERO evaluation. The authors frame the spiking, sparse architecture as a potential path toward event-driven execution.

Conclusion

  • DS-VLA partitions inputs across dendritic branches with distinct states and temporal responses, then integrates their activity through a spiking soma.The neuron-wise shared gate regulates current entering both branches, abstracting rather than simulating complete biophysics.Source: Conclusion · p. 7
  • DS-VLA achieves 91.6% average success on standard LIBERO tasks and 87.35% under Difficulty 1 action perturbations, retaining 95.4% of nominal performance.It outperforms the strongest perturbed baseline by 47.9 percentage points.Source: Conclusion · p. 7
  • The authors present the spiking, sparse dendritic architecture as a path toward efficient event-driven execution.They characterize DS-VLA as a step toward uniting brain-inspired computing and embodied intelligence.Source: Conclusion · p. 7

Related papers

Loading 2609.32253v1…