PaperlayerView exact revision on arXiv

Source-linked AI summary

MambaAct: Efficient Mamba Action Experts for Flow-Matching Vision-Language-Action Models

Khuyen Pham, Tran Nguyen Le, Edoardo Fazzari, Cesare Stefanini

arXiv:2610.22951v1cs.ROcs.CV

TL;DR

Flow-matching VLAs repeatedly evaluate action experts during denoising, making inference costly. MambaAct retains π0.5’s pretrained visual-language encoder and replaces its Transformer action expert with a compact, context-conditioned Mamba expert. It uses over 10× fewer action-expert parameters than π0.5 while improving mean LIBERO success by 1.08 percentage points across three fine-tuning seeds.

  • Problem

    Flow-matching VLAs incur substantial inference cost because their action expert is evaluated repeatedly across denoising steps.

    Source: §I · p. 1

  • Method

    MambaAct retains the pretrained π0.5 visual-language encoder and replaces its Transformer action expert with a compact Mamba expert conditioned through a Context Compressor and gated FiLM interface.

    Source: §I · p. 1

  • Results

    MambaAct uses over 10× fewer action-expert parameters than π0.5 and improves mean LIBERO success by 1.08 percentage points across three fine-tuning seeds.

    Source: §I · p. 1

  • Takeaways & Limitations

    The results support action-expert replacement as a complementary efficiency axis, with strong robustness on LIBERO-Plus and real-world robotic tasks.

    Source: §I · p. 1, §V · p. 8

    Real-world experiments are limited to two UR5 tasks under primary spatial distribution shifts, and compatibility with visual-token pruning or denoising-step compression remains unevaluated.

    Source: §V · p. 8

Abstract

from arXiv · show

Vision-language-action (VLA) models have shown strong promise for generalist robotic manipulation, but their inference cost remains a major bottleneck for real-time control. This limitation is especially pronounced in flow-matching VLAs such as $\pi_{0.5}$, where the action expert is evaluated repeatedly during iterative denoising. We introduce MambaAct, an efficient Mamba action expert for flow-matching VLA policies. MambaAct retains the pretrained visual-language prefix encoder of $\pi_{0.5}$, while replacing its transformer-based action expert with a compact Mamba-based action expert. Within this expert, Mamba serves as the action mixer that models dependencies across the noisy continuous-action sequence. To preserve visual-language conditioning without repeated token-level cross-attention, MambaAct uses a lightweight Context Compressor to map Vision-language model (VLM) prefix tokens into a compact conditioning signal that gates and modulates each Mamba action block. MambaAct uses a $10\times$ smaller action expert and reduces flow-matching denoising latency by $64.8\%$ on LIBERO and $58.8\%$ in real-world deployment, while improving mean LIBERO success by $1.08$ percentage points across three independent fine-tuning seeds and maintaining gains on LIBERO-Plus and real-world benchmarks. Our code and model weights are publicly available at https://github.com/giakhuyendihoc/MambaAct.

I. INTRODUCTION

Flow-matching VLAs repeatedly evaluate the action expert during denoising, making action generation a major inference cost. MambaAct replaces that expert with a compact Mamba design while retaining the pretrained visual-language backbone and standard flow-matching formulation.

I. INTRODUCTION

  • MambaAct replaces the repeatedly evaluated Transformer expert with a compact Mamba expert, retaining pretrained visual-language representations and standard flow matching without additional pretraining or distillation.Its Context Compressor and gated FiLM interface provide compact conditioning from the visual-language prefix.Source: §I · p. 1
  • Over 10× fewer action-expert parameters than π0.5 accompanies a 1.08-percentage-point improvement in mean LIBERO success across three fine-tuning seeds.The comparison concerns action-expert parameters and mean LIBERO success.Source: §I · p. 1
  • Ablations examine the mixer and conditioning interface, while results show MambaAct composes with VLM-depth reduction and transfers to SmolVLA.These evaluations address the efficiency-performance trade-off and generalization across flow-matching VLA backbones.Source: §I · p. 1

A. Efficient Vision-Language-Action Models

B. Mamba in Vision-Language-Action Models

Prior Mamba-based policies target multimodal reasoning, action denoising, or end-to-end policies; MambaAct instead replaces only the action expert in a pretrained flow-matching VLA. The section also describes flow-matching inference, where repeated expert evaluations make action generation a substantial latency source.

B. Mamba in Vision-Language-Action Models

  • RoboMamba targets VLM reasoning, Mamba Policy action denoising, and SUREFlow [20] an end-to-end multimodal policy; MambaAct replaces only the expert in a pretrained flow-matching VLA.MambaAct retains the π0.5 VLM and conditions its Mamba expert through compressed context and gated FiLM modulation.Source: B. Mamba in Vision-Language-Action… · p. 2

A. Preliminaries

  • In π-family flow matching, the expert predicts action velocity; inference starts from Gaussian noise and integrates backward using Ndenoise Euler steps.The action chunk is repeatedly updated during denoising.Source: A. Preliminaries · p. 2
  • At ten denoising steps, action generation accounts for 70.1% of π0.5 latency and 86.4% of SmolVLA latency.The figure decomposes latency on the LIBERO benchmark.Source: A. Preliminaries · p. 2, Figure 1 · p. 3
  • The SISO Mamba3 mixer operates along the action-token dimension, using selective state-space recurrence to model ordered sequences.Mamba replaces pairwise self-attention with selective state-space recurrence.Source: A. Preliminaries · p. 2, B. Mamba in Vision-Language-Action… · p. 2

B. MambaAct

MambaAct retains and jointly fine-tunes the pretrained π0.5 visual-language backbone while replacing its Transformer action expert with a compact Mamba expert. It compresses prefix tokens into conditioning context and processes noisy action tokens through four Mamba blocks to predict flow velocity.

  • MambaAct replaces the 311.5M-parameter Transformer action expert with a 29.7M-parameter Mamba expert while retaining and jointly fine-tuning the pretrained π0.5 visual-language backbone.Source: B. MambaAct · p. 3
  • The Context Compressor projects and pools visual and language prefix tokens, adds learned-query global summaries, and forms context and condition vectors.Source: B. MambaAct · p. 3
  • The noisy action chunk is projected into an initial H-token action representation, with executable actions padded to dimension da = 32.Source: B. MambaAct · p. 3
  • Four Mamba blocks process the action-token sequence, while block-specific FiLM parameters provide scale, shift, and residual-gate modulation.The FiLM scale and shift outputs are zero-initialized, and gate logits initialize the residual multiplier to 0.2; these are initialization choices, not tuned optima.Source: B. MambaAct · p. 3
  • The final velocity head is trained using mean flow-matching error, with the clean ground-truth action chunk and sampled Gaussian noise defining the training setup.Source: B. MambaAct · p. 3

C. Experimental Setup

The experiments compare MambaAct with a matched π0.5 baseline across simulated, robustness, and physical manipulation evaluations. The setup uses joint fine-tuning, multiple seeds for the main LIBERO comparison, and latency measurements on task-specific hardware.

  • The evaluation covers LIBERO task performance and latency, LIBERO-Plus zero-shot robustness, and two real-world UR5 tasks: tube placing and nut pouring.Source: C. Experimental Setup · p. 3
  • The matched baseline is OpenPI π0.5, and MambaAct retains the pretrained PaliGemma visual-language backbone while replacing the Transformer action expert.Source: C. Experimental Setup · p. 4
  • MambaAct uses all 18 VLM layers and a four-block, width-512 Mamba expert; its Context Compressor produces 64 context tokens.The 64 tokens comprise 44 visual, 16 language, and 4 learned-query global-summary tokens.Source: C. Experimental Setup · p. 4
  • For the main LIBERO comparison, MambaAct and the matched baseline are independently fine-tuned with seeds 17, 27, and 42; policies are trained for 30,000 optimizer steps.The full VLM backbone and action expert are optimized jointly.Source: C. Experimental Setup · p. 4
  • LIBERO reports suite success rates and their unweighted average across four suites, using 50 trials per task and three fine-tuning seeds for the main comparison.Source: C. Experimental Setup · p. 4
  • LIBERO-Plus evaluation uses one rollout for each of 10,030 test-only instances across seven perturbation categories, with category-level and overall success rates reported.Source: C. Experimental Setup · p. 4
  • Real-world evaluation uses two UR5 robots and evaluates 28 in-distribution and 52 out-of-distribution trials per task; success scoring gives partial credit for meaningful intermediate completion.Source: C. Experimental Setup · pp. 4–5
  • Latency is reported for prefix encoding and context construction, all ten denoising steps, and end-to-end policy queries including I/O and scheduling overhead.Simulation and real-world measurements use an H100 and an RTX 5090, respectively.Source: C. Experimental Setup · p. 5

1) LIBERO:

On LIBERO, MambaAct improves average success while substantially shrinking the action expert and reducing denoising and end-to-end latency. It is also compared with recent efficient VLA methods under heterogeneous evaluation settings.

1) LIBERO:

  • 97.80±0.43% mean LIBERO success versus 96.72±0.14%, alongside a 90.5% smaller action-expert core and 64.8% lower denoising latency.End-to-end latency falls by 52.9%; the table reports success over three seeds and latency on seed-42 checkpoints.Source: Table I · p. 5, 1) LIBERO · p. 5
  • MambaAct has the second-highest average success listed among efficient VLA methods, with 97.80 ± 0.43% across three fine-tuning seeds.Its seed-42 run averages 98.4% across the four suites; SnapFlow reports 98.8%, but uses 10 trials per task and does not report seed variability.Source: Table II · p. 5, 1) LIBERO · p. 5
  • MambaAct retains standard flow-matching inference and avoids a separate distillation stage, unlike SnapFlow’s one-step denoising and progressive self-distillation.The comparison uses each method’s own baseline for speedup because hardware and implementation settings differ.Source: 1) LIBERO · p. 5, Table II · p. 5

2) LIBERO-Plus:

On LIBERO-Plus, MambaAct slightly exceeds the matched π0.5 baseline overall and shows gains across several perturbation conditions. Observation noise is the main reported degradation, while camera robustness is essentially unchanged.

2) LIBERO-Plus:

  • 79.7% overall score versus the matched π0.5 baseline’s 78.7%, with models trained on standard LIBERO and evaluated without LIBERO-Plus fine-tuning.The matched π0.5 and MambaAct results use the same training and evaluation protocol.Source: 2) LIBERO-Plus · p. 5, Table III · p. 6
  • MambaAct achieves the highest listed lighting robustness score at 95.2% and matches the strongest layout result at 86.4%; camera robustness is essentially unchanged, while observation-noise performance drops from 77.4% to 73.5%.The results indicate that strong standard LIBERO performance does not come at the cost of substantially reduced robustness to perturbations.Source: 2) LIBERO-Plus · p. 6
  • 73.5% under observation noise versus 77.4% for π0.5, while camera robustness is essentially unchanged.Observation noise is the main reported degradation among the perturbations described.Source: 2) LIBERO-Plus · p. 6

3) Real-world benchmarks:

On two real-world UR5 tasks, MambaAct improves out-of-distribution success over π0.5 while reducing denoising and end-to-end latency. The reported comparison includes inference on an RTX5090.

3) Real-world benchmarks:

  • MambaAct achieves 95.2% OOD success on nut pouring and 82.7% on tube placing, versus π0.5’s 86.5% and 76.9%, respectively.The reported evaluation also finds reductions in model size, GFLOPs, and training time.Source: 3) Real-world benchmarks · p. 6, Table IV · p. 6
  • Both models achieve 100% in-distribution success on the two real-world tasks.The latency measurements run on an RTX5090; end-to-end latency includes prefix encoding, denoising, data transfers, communication, and runtime scheduling.Source: 3) Real-world benchmarks · p. 6, Table IV · p. 6
  • 58.8% lower denoising latency and roughly 33% lower end-to-end latency on both tasks, alongside higher OOD success than π0.5.Nut pouring OOD success rises from 86.5% to 95.2%, and tube placing from 76.9% to 82.7%; both models achieve 100% ID success.Source: Table IV · p. 6, 3) Real-world benchmarks · p. 6

B. Ablation Studies

The ablations test Mamba’s action mixer and gated conditioning, then whether the expert composes with VLM-depth reduction and transfers to SmolVLA. They report gains across LIBERO, real-world tasks, and a second flow-matching backbone.

Architectural source of gain

  • In the Table V LIBERO ablation, Mamba achieves 98.4% average success versus 85.7% for a size-matched Transformer under gated conditioning.The same trend holds with cross-attention (96.7% vs. 75.6%), supporting the action mixer’s importance under aggressive compression.Source: B. Ablation Studies · pp. 6–7, Table V · p. 7
  • With Mamba, gated conditioning raises average success from 96.7% to 98.4% and reduces latency from 119.66 to 107.61 ms; its accuracy advantage is confounded by larger expert size.The gated models use 30M parameters versus 17–18M for cross-attention models.Source: B. Ablation Studies · p. 7

Composability

  • Using the first 12 rather than all 18 VLM layers, MambaAct reduces LIBERO training time from 13.78 to 10.81 hours and E2E latency from 107.6 to 102.91 ms, with 97.5% versus 98.4% average success.This tests compatibility with prefix-side acceleration; the result suggests the expert can use intermediate VLM representations.Source: Table VI · p. 7, B. Ablation Studies · p. 7

Real-world evaluation

  • The real-world L12 model reduces size from 2.96B to 2.30B parameters and latency from 83 ms to 74 ms, preserving 100% ID success on both tasks.It reaches 95.0% final success on nut pouring and 90.0% on tube placing.Source: B. Ablation Studies · p. 7

Transferability

  • Under the same 25k-step, batch-32, seed-1000 budget, SmolVLA-MambaAct reaches 84.95% average success (+3.65 percentage points), reducing denoising latency by 63.3% and E2E latency by 52.0%.The released SmolVLA checkpoint reaches 81.30% under the evaluation protocol; the Mamba expert uses 64.31M rather than 98.25M parameters.Source: B. Ablation Studies · pp. 7–8

Summary

  • The ablation summary finds Mamba effective with reduced VLM depth and on a second flow-matching VLA, while gated conditioning’s accuracy advantage remains partly confounded by expert capacity.The authors interpret the results as consistent with benefits from selective action-sequence modeling and lightweight visual-language conditioning beyond compression alone.Source: B. Ablation Studies · pp. 7–8

V. CONCLUSION

MambaAct replaces the repeatedly evaluated Transformer action expert with a compact Mamba expert while retaining the pretrained VLM. The conclusion reports compression with improved LIBERO performance and identifies limits in real-world scope and compatibility testing.

V. CONCLUSION

  • MambaAct reduces the π0.5 action-expert parameter count from 311.5M to 29.7M (approximately 10×) while lowering denoising latency and improving mean LIBERO performance across three fine-tuning seeds.The authors also report strong robustness on LIBERO-Plus and real-world robotic tasks.Source: §V · p. 8
  • The action-expert replacement integrates with VLM-depth reduction and transfers effectively to SmolVLA.MambaAct retains the pretrained visual-language backbone and standard flow-matching objective, without additional pretraining or distillation.Source: §V · p. 8
  • Real-world experiments cover only two UR5 tasks under primary spatial shifts; compatibility with visual-token pruning and denoising-step compression remains unevaluated.The authors also leave deeper analysis of gated conditioning and selective Mamba state dynamics to future work.Source: §V · p. 8

Related papers

Loading 2610.22951v1…