Source-linked AI summary
Mamba-based Selective State Space Modeling Improves the Accuracy-Complexity Tradeoff of SmolVLA Vision-Language-Action Experts
Farida Mohsen, Thowayba Elkaffash, Mohammad Reza Chalak Qazani, Mohamed Mabrok, Nader Meskin, Ali Safa
TL;DR
VLA control must balance task success against policy-call frequency, since short execution horizons improve responsiveness but increase inference overhead. This paper replaces SmolVLA’s causal self-attention temporal mixers with Mamba selective state-space mixers and evaluates both experts across LIBERO horizons. Mamba retains the Transformer’s performance at N = 1 while delivering larger success advantages at longer horizons and reducing model complexity.
Problem
VLA policies face a tradeoff between success rate and policy-call frequency when choosing how many predicted actions to execute before replanning.
Method
The paper replaces SmolVLA’s causal self-attention layers with Mamba-based selective state-space mixers while keeping the remaining architecture and training protocol fixed, then evaluates three execution horizons on LIBERO.
Results
Mamba improves average success over the Transformer by 3.7 points at N = 25 and 7.8 points at N = 50, while performing comparably at N = 1.
Takeaways & Limitations
The Mamba expert offers stronger success retention under longer open-loop horizons and reduces overall model weight complexity by 24%.
Abstract
from arXiv · showhide
Vision-language-action (VLA) models face a crucial tradeoff between their task success rate and the policy-call frequency. Executing a single action per inference ($N=1$) enables accurate robot control but comes at the cost of huge compute time overheads, making real-time implementation infeasible. On the other hand, executing longer action horizons before replanning ($N\gg1$) reduces compute complexity, but inevitably degrades the system's success rate. In order to improve the VLA accuracy-complexity tradeoff, this paper investigates Mamba's selective state-space modeling as an alternative to causal self-attention within the action expert of the popular SmolVLA model, widely used as a reference model for its highly accurate yet low complexity nature. We evaluate both the Mamba- and Transformer-based experts on the widely-adopted LIBERO benchmark suites across three execution horizons $N\!\in\!\{1,25,50\}$, respectively corresponding to high, moderate and low compute complexities. Our results remarkably show that the advantage of the Mamba expert increases with the execution horizon, indicating significant success retention under long execution horizons $N = 50$ and $N = 25$. When $N = 50$ actions are executed before replanning (i.e., corresponding to feasible real-time deployment), the Mamba expert outperforms the Transformer baseline by $7.8\%$. In addition, when $N = 25$ actions are executed before replanning, our Mamba expert outperforms the Transformer baseline by $3.7\%$. Finally, under per-action replanning ($N=1$), our Mamba variant matches the Transformer-based mean success rate while significantly reducing the overall model parameter complexity by $24\%$ thanks to Mamba's compute-efficient nature.
I. INTRODUCTION
This paper investigates replacing SmolVLA’s causal self-attention temporal mixers with Mamba-based selective state-space mixers to improve the accuracy-complexity tradeoff across action execution horizons.
- Motivation: The execution horizon N trades control accuracy against latency: smaller N increases policy calls, whereas larger N reduces calls but acts longer on stale observations.This tradeoff directly affects real-time deployment feasibility.
- Motivation: Mamba is investigated because its input-dependent causal recurrence may provide a useful inductive bias for coordinating ordered actions within a chunk.The open question is whether this improves the accuracy of later executed actions.
- Motivation: The study examines Mamba-based selective state-space modeling as an alternative to causal self-attention within SmolVLA’s flow-matching action expert.The remaining architecture, objective, and optimization protocol are kept fixed for comparison.
- Study design: The two experts are evaluated across three execution horizons using the standard LIBERO benchmark suites.The comparison covers both the Mamba-based model and the Transformer-based reference.
II. RELATED WORK
VLA models adapt pretrained vision-language representations to robot control through discrete action tokens or continuous action generators.
- VLA architectures: OpenVLA produces tokenized actions, whereas π0 and SmolVLA use flow-matching action experts to generate continuous action sequences.SmolVLA alternates cross-attention grounding with causal temporal mixing, making it suitable for the paper’s comparison.
B. Action chunking and the execution horizon
Action chunking separates action generation from execution by applying only a prefix of each predicted chunk before replanning from a new observation.
- Action chunking: Action Chunking with Transformers showed that predicting multiple future actions can improve the temporal coherence of imitation-learning policies.This motivates generating temporally structured action chunks rather than isolated commands.
- Execution horizon: Diffusion Policy introduced receding-horizon execution, where only a prefix of each predicted chunk is applied before the plan is updated.The prefix length is the execution-horizon variable swept in this study.
- Execution-time methods: Real-time chunking constrains newly generated chunks to remain compatible with actions already committed for execution, while related monitoring methods truncate unreliable chunks.These approaches address errors during the open-loop interval at execution time.
- Policy formulation: The policy maps the current robot observation and a natural-language instruction to a chunk of future actions.Observations include camera views and robot state.
- Action representation: SmolVLA uses chunks of H = 50 actions, each represented by a 7-dimensional vector containing translation, rotation, and gripper commands.Each action has three translation, three rotation, and one gripper component.
- SmolVLA: The reference policy combines a pretrained vision-language backbone with a conditional flow-matching action expert whose encoded features condition the expert through cross-attention.The backbone is frozen during policy training.
B. Conditional Flow-Matching AE
The conditional flow-matching action expert interpolates demonstrated action chunks with noise and trains a velocity field to recover action chunks from noisy inputs.
- Training examples pair observations, language, and demonstrated action chunks with Gaussian noise for flow-time interpolation.The interpolation is defined from data at τ = 0 to noise at τ = 1.
- The action expert vθ is trained by minimizing a flow-matching objective shared with the original SmolVLA setup.The flow-time sampling distribution p(τ) is shared by the baseline and Mamba-based variant.
- At inference, sampling starts from Gaussian noise and integrates the learned velocity field from τ = 1 to τ = 0 using ten Euler steps.This procedure produces the predicted action chunk.
C. Mamba-Based Intra-Chunk Temporal Mixing
M1 replaces SmolVLA’s causal self-attention temporal mixers with Mamba selective state-space mixers while preserving the rest of the action-expert design.
- M1 retains all eight cross-attention grounding layers and replaces the eight causal self-attention layers with Mamba-1 temporal mixers.The replacement targets intra-chunk temporal mixing in the action expert.
- The backbone, grounding pathway, embeddings, token width, chunk length, and output head remain unchanged between B0 and M1.This isolates the temporal-mixer substitution between the two variants.
- The Mamba selective scan processes action-token representations across the H ordered positions of each action chunk.The sequence entering temporal layer k consists of representations for the chunk positions i = 1, …, H.
- Mamba’s discretization step and input projections depend on the current token representation, allowing information retention, updating, or suppression across chunk positions.The continuous state-transition parameters are learned and shared across sequence positions.
D. Execution Horizon
Execution horizon N controls how many actions from a predicted chunk are applied before the policy receives a new observation and replans.
- Both experts predict fixed action chunks of H = 50 actions, while N determines the executed prefix length.The applied actions are [a_t, …, a_t+N−1] = FirstN(A_t), with N ∈ {1, 25, 50}.
- At N = 1, the policy replans after every action; at N = 50, it executes the complete predicted chunk before replanning.These settings represent the shortest and longest execution prefixes tested.
- The horizon sweep keeps trained weights fixed and tests success when increasingly long chunk prefixes execute without observation-conditioned correction.
A. Implementation Details
The study trains and evaluates matched SmolVLA experts on LIBERO while varying execution horizon, using identical pretrained visual-language backbones and controlled optimization settings.
- M1 reduces the trainable expert from 99.9M to 76.3M parameters, a 23.6% reduction, while leaving the frozen backbone identical.B0 uses causal self-attention temporal layers, whereas M1 uses Mamba SSM mixers.
- Both variants use 30,000 AdamW updates, global batch size 64, cosine learning-rate decay, ten Euler inference steps, and N ∈ {1, 25, 50}.
- LIBERO evaluation covers Spatial, Object, Goal, and Long suites, totaling 40 tasks with ten rollout trials per task.The benchmark uses robosuite with a Franka Panda arm and the released demonstrations.
- Figure 3 reports four-suite mean success rate versus execution horizon, averaged across three seeds with ±1 standard-deviation bands.Annotations show the M1−B0 gap at each horizon.
C. Evaluation Metrics
The evaluation uses task success rate as the primary metric and summarizes performance across tasks, seeds, horizons, and model differences with bootstrap uncertainty.
- Success is computed per task and training seed, then averaged across the ten tasks in each LIBERO suite.
- Reported suite values are means and standard deviations across three training seeds.
- The analysis treats 40 tasks as paired observations at each horizon and reports the mean per-task difference between M1 and B0.
- 95% confidence intervals for model differences are derived from 10,000 bootstrap resamples of the tasks.
- Sensitivity to reduced replanning is summarized for each model, with smaller values indicating stronger success retention.
V. EXPERIMENTAL RESULTS
Across LIBERO suites, M1 increasingly outperforms B0 as the execution horizon lengthens, while retaining comparable performance under per-action replanning.
- A. Impact of the Execution Horizon on the Success Rates: 7.8 points: M1 reaches 61.7% versus 53.9% for B0 at N = 50 across the four-suite mean.
- A. Impact of the Execution Horizon on the Success Rates: 3.7 points: M1’s four-suite mean advantage over B0 at N = 25 exceeds the 0.3-point difference observed at N = 1.
- A. Impact of the Execution Horizon on the Success Rates: At N = 1, M1 and B0 achieve on-par results under per-action replanning.
- A. Impact of the Execution Horizon on the Success Rates: M1 loses 14.5 points from N = 1 to N = 50, compared with 22.0 points for B0, retaining 7.5 additional points.
- B. Per-Suite Structure: At N = 50, M1’s gains are 17.7 points on Object, 10.0 on Spatial, and 4.7 on Long, while B0 leads Goal by 1.0 point.
- B. Per-Suite Structure: Object reverses from a 6.0-point B0 lead at N = 1 to a 17.7-point M1 lead at N = 50.
C. Discussion on the results
The study attributes M1’s stronger long-horizon retention to its Mamba temporal mixer and frames the result as an improved accuracy-complexity tradeoff for open-loop deployment.
- M1 improves average success over B0 by 3.7 points at N = 25 and 7.8 points at N = 50, while performing comparably at N = 1.
- M1 reduces model weight complexity by 24% while preserving comparable N = 1 success and improving longer-horizon retention.
- Longer execution reduces scheduled policy-call frequency but delays incorporation of new observations.
- Mamba’s input-dependent causal recurrence may provide an inductive bias for coordinating ordered action positions within a chunk.
- The conclusion identifies additional VLA architectures and physical-robot validation as future evaluation targets.