PaperlayerView exact revision on arXiv

Source-linked AI summary

Global Prior Meets Local Consistency: Dual-Memory Augmented Vision-Language-Action Model for Efficient Robotic Manipulation

Zaijing Li, Bing Hu, Rui Shao, Gongwei Chen, Dongmei Jiang, Pengwei Xie, Jianye Hao, Liqiang Nie

arXiv:2602.20200v2cs.ROcs.AIcs.CV

TL;DR

VLA action generation is limited by inefficient noise-to-action transformation and weak temporal awareness. OptimusVLA addresses these issues with Global Prior Memory and Local Consistency Memory, and reports stronger simulation and real-world performance with faster inference.

  • Problem

    VLA policies face inefficient generation from a large isotropic-noise-to-action gap and weak temporal robustness from conditioning only on current observations.

  • Method

    OptimusVLA retrieves task-level priors with Global Prior Memory and models recent executed actions with Local Consistency Memory to constrain generation temporally.

  • Results

    OptimusVLA outperforms strong baselines across three simulators and real-world suites while delivering 2.9× inference speedup.

  • Takeaways & Limitations

    Memory-driven prior initialization and temporal constraints improve VLA efficiency and robustness without sacrificing generalization.

  • Takeaways & Limitations

    GPM depends on trajectory-memory coverage and quality, while fixed-length, local action modeling may be insufficient for very long-horizon or delayed-effect tasks.

Abstract

from arXiv · show

Hierarchical Vision-Language-Action (VLA) models have rapidly become a dominant paradigm for robotic manipulation. It typically comprising a Vision-Language backbone for perception and understanding, together with a generative policy for action generation. However, its performance is increasingly bottlenecked by the action generation proceess. (i) Low inference efficiency. A pronounced distributional gap between isotropic noise priors and target action distributions, which increases denoising steps and the incidence of infeasible samples. (ii) Poor robustness. Existing policies condition solely on the current observation, neglecting the constraint of history sequence and thus lacking awareness of task progress and temporal consistency. To address these issues, we introduce OptimusVLA, a dual-memory VLA framework with Global Prior Memory (GPM) and Local Consistency Memory (LCM). GPM replaces Gaussian noise with task-level priors retrieved from semantically similar trajectories, thereby shortening the generative path and reducing the umber of function evaluations (NFE). LCM dynamically models executed action sequence to infer task progress and injects a learned consistency constraint that enforces temporal coherence and smoothness of trajectory. Across three simulation benchmarks, OptimusVLA consistently outperforms strong baselines: it achieves 98.6% average success rate on LIBERO, improves over pi_0 by 13.5% on CALVIN, and attains 38% average success rate on RoboTwin 2.0 Hard. In Real-World evaluation, OptimusVLA ranks best on Generalization and Long-horizon suites, surpassing pi_0 by 42.9% and 52.4%, respectively, while delivering 2.9x inference speedup.

1. Introduction

OptimusVLA addresses action generation as the main VLA bottleneck by combining task-level prior retrieval with lightweight temporal consistency modeling. Its evaluations report improved efficiency, robustness, and benchmark performance.

  • Action generation is the dominant bottleneck for VLA efficiency and robustness.
  • Large prior–target gaps require multi-step denoising and can initialize infeasible actions, while current-observation policies miss task phase and trajectory context.
  • OptimusVLA combines Global Prior Memory, which retrieves task-level priors, with Local Consistency Memory, which models recent actions and injects temporal constraints.
  • 98.6% average success rate on LIBERO, 13.5% improvement over π0 on CALVIN, and 38% average success rate on RoboTwin 2.0 Hard were reported.
  • In real-world evaluation, OptimusVLA ranked best on Generalization and Long-horizon suites, surpassing π0 by 42.9% and 52.4%, respectively, with 2.9× inference speedup.

2. Preliminaries and Related Work

Prior VLA methods face efficiency costs from noise-to-action generation and temporal limitations from current-observation conditioning. OptimusVLA narrows the source–target gap with task-level priors while adding progress-aware temporal constraints.

  • VLA architectures include single-stream models that autoregressively generate discretized action tokens and hierarchical models combining vision–language backbones with generative policies.
  • Even flow-matching policies require multiple function evaluations to produce high-quality actions, limiting achievable speedup.
  • Conditional Flow Matching transports noise x_0 to target actions x_1 through a learned time-conditioned velocity field, commonly using a straight-line Optimal Transport path.
  • Global Prior Memory replaces an isotropic Gaussian source with a task-level prior built from semantically similar action trajectories to narrow the source–target gap.
  • Current-observation conditioning hampers phase awareness and consistency, whereas long observation histories increase latency and memory usage and simple concatenation lacks global temporal context.

3. OptimusVLA

OptimusVLA combines Global Prior Memory and Local Consistency Memory to improve action-generation efficiency and temporal robustness. GPM retrieves task-level priors for initialization, while LCM models action history to provide progress awareness and smoother trajectories.

  • OptimusVLA Overview: At each time step, GPM samples an action chunk and LCM produces a consistency bias that is added to form the flow policy input.The flow policy then transforms this combined input into the next action chunk.
  • Global Prior Memory: GPM retrieves semantically similar trajectories and forms task-level priors instead of starting flow generation from isotropic Gaussian noise.The memory bank retrieves trajectories using task embeddings, and weighted action blocks form the prior distribution.
  • Global Prior Memory: GPM uses similarity-adaptive noise and adaptive NFE to initialize generation near the target action distribution, reducing required NFEs and infeasible samples.The Prior-Aware Sampler controls initialization noise and function evaluations based on retrieved-prior similarity.
  • Local Consistency Memory: LCM processes recent action chunks with a Consistency Layer and Dynamic Awareness Module to infer progress and enforce temporal consistency.The Dynamic Awareness Module models inter-chunk dependencies, while the Consistency Layer captures dependencies within action chunks.
  • Training Details: Training first pretrains a hierarchical VLA, then learns task-discriminative prior retrieval and trains LCM to predict residuals between prior means and ground-truth action chunks.GPM is frozen before LCM training.

4. Experiments

Experiments evaluate OptimusVLA across simulation and real-world manipulation, including benchmark performance, ablations, memory scaling, and efficiency. The framework achieves strong success rates and improves robustness and inference efficiency through GPM and LCM.

  • Evaluation setup: Experiments cover LIBERO, CALVIN, RoboTwin 2.0, and real-world Generalization and Long-horizon suites.The evaluation includes simulation benchmarks, real-world manipulation, ablations, and efficiency comparisons.
  • Simulation results: 98.6% average success rate on LIBERO, with 3.2 NFEs for OptimusVLA versus 10.0 for π0.5.GPM stabilizes long-horizon generation by initializing flow closer to the target manifold.
  • Simulation results: 38% average success rate on RoboTwin 2.0 Hard, including 58% on Stack Bowls Two and a +28% success-rate improvement over RDT.LCM supplies consistency constraints for smooth, coordinated bimanual trajectories.
  • Real-world results: 85.0% and 64.0% average success rates on real-world Generalization and Long-horizon Tasks, respectively.On Long-horizon Tasks, OptimusVLA outperforms π0 by 52.4%; the Generalization suite varies lighting and scenes.
  • Ablation study: Removing GPM drops CALVIN performance by 3.8% and Generalization performance by 9.4%, while removing LCM drops LIBERO-Long by 1.7%.The ablations associate GPM with cross-environment generalization and LCM with temporal consistency and progress awareness.
  • Memory ablation: LIBERO-Long performance scales with memory-bank richness, while retrieving k = 8 trajectories supports a robust Gaussian mixture prior.One trajectory makes the prior too deterministic, whereas small k overfits to a single retrieved trajectory.
  • Efficiency analysis: OptimusVLA reaches 97.6% on LIBERO-Goal in 18,000 training steps, versus 26,000 steps for π0.5 to reach a similar level.It also achieves 6.5× faster inference time and 3.1× fewer NFEs on LIBERO than π0.5.

5. Conclusion

OptimusVLA combines Global Prior Memory and Local Consistency Memory for robotic manipulation. The framework improves performance and inference efficiency across simulation and real-world experiments.

  • Conclusion: OptimusVLA combines GPM, which retrieves task-level priors, with LCM, which models executed-action histories and enforces temporal coherence.Together, the memories address generative efficiency and trajectory consistency.
  • Conclusion: Extensive simulation and real-world experiments demonstrate superior performance together with substantially higher inference efficiency.The conclusion attributes the framework’s design to improvements in both efficiency and robustness.

Supplementary Material

The supplementary material provides additional information on limitations, Global Prior Memory, training, evaluation, and case studies.

  • Supplementary organization: Section A covers limitations and future work.
  • Supplementary organization: Section B covers Global Prior Memory.
  • Supplementary organization: Section C covers training details, while Section D covers evaluation.
  • Supplementary organization: Section E covers case studies.

A. Limitation and Future Work

OptimusVLA’s memory components face scope boundaries: GPM depends on trajectory-memory coverage and quality, while LCM’s fixed-length, local action modeling may not support very long-horizon dependencies. The paper identifies adaptive memory updates and end-to-end joint training as future directions.

  • Limitations: GPM effectiveness is constrained by the coverage and quality of its trajectory memory bank, especially when tasks or scenes differ substantially from stored experiences.Retrieved priors may then be misleading and bias the VLA model toward suboptimal behaviors.
  • Limitations: LCM’s fixed-length, action-centric modeling may be insufficient for very long horizons, multi-stage dependencies, or delayed effects.The design keeps computational overhead small but does not directly model all long-range task dependencies.
  • Future Work: Adaptive memory mechanisms with online consolidation, forgetting, and uncertainty-aware retrieval are proposed to support continual learning under distribution shift.These mechanisms are presented as a natural future direction rather than an evaluated component of the current system.
  • GPM Construction: GPM retrieves semantically similar trajectories, aligns their action chunks to execution progress, and constructs a task-level prior for inference.The retrieved chunks are interpolated to the model horizon and approximated with a moment-matched Gaussian.
  • Similarity-Adaptive Sampling: Retrieval confidence controls GPM’s noise scale and discretization steps, with higher confidence permitting lower noise and fewer steps.For novel scenarios, the system falls back to higher noise and more computation; cached retrieval avoids repeated index queries within an episode.

C. Training Details

The first training stage pretrains a hierarchical VLA backbone and flow policy using the π0.5 architecture and a Conditional Flow Matching objective, without attaching GPM or LCM.

  • Stage I: The first stage pretrains a hierarchical VLA model following π0.5’s architecture and training protocol.GPM and LCM are absent during this stage so the base VLA can serve as the backbone for later training.
  • Stage I: The VLM maps each instruction and observation to a multimodal representation that is fed into the flow policy.Ground-truth action chunks are extracted from the dataset for policy training.
  • Stage I: The flow policy is trained with a Conditional Flow Matching objective on ground-truth action chunks.This establishes the action-generation policy before the memory modules are attached.

C.2. Stage II: GPM Training

Stage II trains the GPM Prior Head to organize multimodal representations by task semantics while keeping the pretrained VLA parameters frozen; later stages use retrieved priors and action history for LCM training.

  • Stage II: Stage II attaches a lightweight Prior Head while freezing all parameters of the pretrained base VLA model.Only the Prior Head is updated during this stage.
  • Stage II: The Prior Head is trained with an InfoNCE task-contrastive loss using batches containing positive trajectory pairs from the same task.Other-task trajectories act as hard negatives, ensuring each anchor has an in-batch positive.
  • Stage III: LCM training retrieves top-k GPM priors, computes their Gaussian mean, and predicts the residual between the global prior and ground-truth actions.The LCM is optimized by unrolling along the trajectory and using previous action chunks as input.
  • Stage III: A cold-start strategy masks the previous action chunk with probability p_cold so LCM can operate when inference begins without history.The predicted bias is trained against its target with mean squared error.

D.1. Evaluation on RoboTwin 2.0

RoboTwin 2.0 evaluates bimanual manipulation across diverse objects, tasks, and robot embodiments, using Hard-setting rollouts to compare OptimusVLA with baselines including π0.5.

  • Benchmark: RoboTwin 2.0 contains 731 annotated objects across 147 categories, over 100k expert dual-arm trajectories, and 50 collaborative dual-arm tasks.The benchmark uses five distinct robot embodiments.
  • Protocol: The main simulation protocol trains each task in a single-task manner on the Aloha–AgileX dual-arm platform with 50 clean expert demonstrations.Evaluation uses the benchmark’s Hard setting with domain-randomized clutter, lighting, textures, and height variations.

D.2. Evaluation on Real-World

OptimusVLA is evaluated in real-world Generalization and Long-horizon task suites on the Galaxea R1 Lite platform, using varied scenes, lighting conditions, object instances, and randomized object positions. The reported results and qualitative examples indicate strong performance across both suites.

  • Platform and evaluation: Galaxea R1 Lite is a mobile, wheeled humanoid robot with a bimanual upper body for human-centric indoor manipulation.Its embodiment includes two 6-DoF arms, a 3-DoF torso, and a 6-DoF omnidirectional base.
  • Platform and evaluation: The real-world evaluation uses Generalization Tasks for varying scenes, lighting conditions, and object instances, and Long-horizon Tasks for sequence stability and robustness.Objects are randomly initialized at different spatial locations in each evaluation episode.
  • Results: OptimusVLA demonstrates superior performance in real-world environments across the Generalization and Long-horizon task suites.The paper reports quantitative results in Tables 8 and 9 and provides additional qualitative visualizations.
  • Qualitative results: Qualitative examples cover four Generalization Tasks, including placing fruit, plates, bottles, and cups under third-person and wrist-camera views.The examples include placing one fruit on a plate, placing a plate on a tablecloth, standing a bottle upright, and placing a cup on a tablecloth.
  • Qualitative results: Qualitative examples cover four Long-horizon Tasks involving multiple objects and sequential placement actions under third-person and wrist-camera views.Examples include placing all fruits on a plate, placing objects across a tablecloth and bowl, placing a cup and apple on a plate, and placing three blocks on a tablecloth.
Loading 2602.20200v2…