PaperlayerView exact revision on arXiv

Source-linked AI summary

Remember What You Did: Action-History Memory with Dual-Expert Denoising for Long-Horizon Vision-Language-Action Policies

Yaxin Zhao, Dianye Huang, Chenwei Wang, Chenguang Yang, Zhongliang Jiang

arXiv:2609.37307v1cs.RO

TL;DR

Similar current observations can require different actions at different task stages, while many VLAs lack explicit interaction history. ActMem-VLA uses executed-action memory to steer early denoising before handing off to a frozen action expert. It improves task completion across simulated and real-world evaluations.

  • Problem

    Many VLAs lack explicit interaction history, leaving similar observations at different task stages ambiguous despite requiring different next actions.

    Source: §I · p. 2

  • Method

    ActMem-VLA uses Mamba-encoded executed-action history to condition a PreAction Expert for early denoising, then hands the intermediate action state to a frozen Action Expert for refinement.

    Source: §I · pp. 2–3

  • Results

    Across ten simulated tasks, ActMem-VLA achieves 80.8% average success, exceeding π0.5 and MemoryVLA by 15.6 and 31.3 percentage points; in four real-world tasks, it improves success on all four and raises the average from 35.0% to 63.75% relative to π0.5.

    Source: §V · p. 7

  • Takeaways & Limitations

    Training only the memory module and PAE while keeping the fine-tuned base VLA frozen provides a parameter-efficient path to temporal adaptation for long-horizon manipulation.

    Source: §V · p. 7

Abstract

from arXiv · show

Vision-language-action (VLA) models have driven rapid progress in robotic manipulation, demonstrating strong fine-grained control and promising performance on long-horizon tasks. However, many existing VLAs lack explicit access to interaction history, making them vulnerable to perceptual aliasing: similar current observations and robot states at different task stages may induce action ambiguity and lower success rate. Existing methods incorporate temporal or progress cues through feature conditioning, action-prior modification, or sampling guidance. However, methods that jointly fine-tune memory modules and the base VLA incur additional policy-training costs, motivating the separation of trainable history-conditioned steering from frozen base-policy refinement. We propose ActMem-VLA, a dual-expert handover architecture that augments a frozen, fine-tuned VLA with a memory plugin comprising a Mamba-based memory module and a lightweight PreAction Expert (PAE). Specifically, Mamba encodes executed-action history into memory that conditions PAE alongside current context. With these inputs, PAE steers task progression during early, high-noise denoising, then passes the partially denoised action to the frozen Action Expert (AE) to refine action details during the remaining low-noise steps. The fine-tuned base VLA remains frozen throughout training, while only the Mamba module and PAE are jointly optimized. On LIBERO-Mem, ActMem-VLA achieves 80.8\% average success across all ten tasks, compared with 65.2\% for $\pi_{0.5}$ and 49.5\% for MemoryVLA, while introducing only 3.45\% additional parameters. Across four real-world tasks, it improves the average success rate over $\pi_{0.5}$ by 28.8\%.

I. INTRODUCTION

ActMem-VLA uses executed-action history to steer a frozen VLA through ambiguous task stages, separating history-aware early denoising from frozen action refinement. It reports gains in simulation and real-world tasks.

I. INTRODUCTION

  • Many VLAs use only current observations and robot state, so visually similar inputs at different task stages can leave the appropriate next action ambiguous [4], [10], [11],, [13].For example, the robot may need to lower an apple onto a scale before weighing, but move it toward a basket afterward.Source: §I · pp. 1–2
  • Prior approaches condition on historical observations, memory-conditioned features, or recurrent latent summaries, but raw past frames add latency and redundant visual content [7], [14]–[17], [18]–[21], –[26].Existing memory methods often couple memory modeling to action generation, requiring further policy training to equip a fine-tuned VLA with memory [14],, [28], [29].Source: §I · p. 2
  • ActMem-VLA encodes executed-action history into progress memory that conditions a PreAction Expert (PAE) for early, high-noise denoising, then hands off to the frozen Action Expert (AE) for low-noise refinement.Only the Mamba memory module and PAE are optimized; the base VLA remains frozen.Source: §I · pp. 2–3, Figure 2 · p. 2
  • The memory plugin adds 3.45% parameters relative to the base VLA while keeping its fine-tuned policy frozen.This design uses action history as compact task-progress information rather than redundant historical visual observations.Source: §I · p. 3
  • 80.8% average success across LIBERO-Mem’s ten tasks exceeds π0.5 (65.2%) and MemoryVLA (49.5%).Across four real-world tasks, ActMem-VLA improves average success over π0.5 by 28.8%.Source: §I · p. 3

II. RELATED WORK

The paper builds on flow-matching VLAs and addresses ambiguity from policies that lack explicit history by adding history-conditioned early denoising while freezing the original VLM and AE.

A. Vision-Language-Action Policies

B. Memory Mechanisms in Robotic Policies

ActMem-VLA represents task progress with executed-action history and divides one denoising trajectory between a history-conditioned expert and the frozen action expert. The section frames this design against history-representation approaches and task-stage ambiguity.

B. Memory Mechanisms in Robotic Policies

  • Existing robotic-policy memory approaches preserve observations, salient events, retrieved information, or compact latent memories, while recurrent mechanisms compress interaction history across decisions [7], [14]–[17], [19], [20], [23]–[26].ActMem-VLA instead uses executed-action history as a progress-relevant signal because past visual content is redundant, costly, and potentially ambiguous.Source: B. Memory Mechanisms in Robotic Policies · p. 3

C. Structured Denoising for Action Generation

  • ActMem-VLA assigns early, high-noise denoising to a memory-conditioned PAE and low-noise refinement to the frozen AE within one trajectory.This division steers the overall action trajectory while retaining the pretrained expert’s fine-grained control capability.Source: C. Structured Denoising for Action… · p. 3

A. Problem Description

  • Similar observations can recur at different task stages while requiring different actions, but memoryless policies do not condition on the distinct action histories.The policy predicts action chunks and executes the first s ≤ H actions before querying again.Source: A. Problem Description · p. 3
  • LIBERO-Mem comprises ten memory-dependent manipulation tasks, and its taskwise success comparisons use task-specific step limits, counting unfinished episodes as failures.Source: Figure 3 · p. 4, Figure 4 · p. 4
  • ActMem-VLA encodes executed-action history into memory that conditions the PAE with current context; the PAE denoises first, then the frozen AE continues the same trajectory without reinitializing noise.Only the execution memory module and PAE are trained; the base VLM and AE remain frozen.Source: A. Problem Description · pp. 3–4

B. Mamba-Based Execution Memory

The execution-memory module processes executed actions with Mamba-2, producing a memory representation that is projected into keys and values for conditioning the PAE.

B. Mamba-Based Execution Memory

C. PreAction Expert and Denoising Handover

A lightweight PAE handles early denoising before handing the intermediate action state to the frozen AE for low-noise refinement.

C. PreAction Expert and Denoising Handover

D. Joint Training and Efficient Implementation

Training optimizes the memory module and PAE on demonstrations while keeping the VLM and AE frozen, with prefix KV reuse available to accelerate training.

D. Joint Training and Efficient Implementation

  • The memory module and PAE are trained on demonstration episodes while the VLM and AE remain frozen, using executed actions preceding time t as memory.Memory updates follow the inference-time s-action blocks and pad incomplete blocks.Source: D. Joint Training and Efficient… · p. 4
  • For each query, training samples a flow time τ ∈ [0, r] and Gaussian noise, constructs noisy actions, and conditions the PAE velocity prediction on current context and memory.The memory module and PAE are optimized with a flow-matching loss.Source: D. Joint Training and Efficient… · p. 4
  • Selected prefix KV from the frozen VLM can be precomputed and reused to accelerate training.Source: D. Joint Training and Efficient… · p. 4

IV. EXPERIMENTS

The experiments evaluate ActMem-VLA on ten simulation tasks and four real-world tasks, alongside component and hyperparameter ablations.

IV. EXPERIMENTS

  • The evaluation covers ten simulation and four real-world tasks, with ablations examining component contributions and key hyperparameter choices.Source: §IV · p. 5

A. Experimental Settings

  • Simulation uses LIBERO-Mem's ten memory-dependent tasks, while real-world trials use a DOBOT Nova arm with wrist- and head-mounted cameras and 50 demonstrations per task.The real-world robot is controlled through ROS from a workstation with an NVIDIA RTX5090 GPU.Source: A. Experimental Settings · p. 5
  • The four real-world tasks test weighing-stage discrimination or prescribed repetitions followed by a distinct terminal action.The tasks are two- and three-cycle bowl return, mango weighing, and triple button press.Source: A. Experimental Settings · p. 5
  • Baselines are memory-free π0.5 and observation-retrieval MemoryVLA; simulation uses three training seeds and 180 trials per task per method, while real-world evaluation uses 20 trials per task.Simulation checkpoints are evaluated with three seeds and 20 trials per task per evaluation seed; ablations use one training seed.Source: A. Experimental Settings · p. 5

B. Simulation Results

Under task-specific limits, ActMem-VLA leads the ten-task comparison, while the standard 600-step protocol may not reveal whether success depends on tracking task progress.

1) Comparison under the LIBERO-Mem 600-Step Episode Limit:

  • The 600-step protocol can count cyclic action sequences as success, so its success rates do not reliably indicate whether a policy tracks task progress.The memoryless π0.5 baseline reaches nearly 100% on several tasks, including T1–T3.Source: 1) Comparison under the LIBERO-Mem… · p. 5

2) Comparison under Task-Specific Step Limits:

C. Ablation Studies

The ablations test whether executed-action memory and a trainable PreAction Expert (PAE) contribute to performance. Their benefits are task-dependent, with the clearest gains on repetition-intensive T6.

C. Ablation Studies

  • The evaluation targets repetition tracking and multi-stage object rearrangement across T6–T8.T6 involves seven bowl repetitions; T7 and T8 involve bowl swapping and cyclic rearrangement.Source: C. Ablation Studies · p. 6

1) Component Ablations:

  • On T6, removing execution memory cuts success from 80.0% to 38.3%, while removing the PAE cuts it to 58.3%.Across T6–T8, the full model has the highest average success among the component variants, though both ablations outperform it on T8.Source: 1) Component Ablations · p. 6
  • Removing execution memory reduces average success from 55.0% to 40.6%, while removing the PAE reduces it to 53.3%.The variants remove Mamba or the trainable early-denoising expert, respectively, while keeping the base VLA frozen.Source: 1) Component Ablations · p. 6

2) Hyperparameter Sensitivity:

Hyperparameter tests vary the handover point and the capacities of the memory module and PAE. The tested default configuration uses four PAE steps followed by six frozen-AE steps, with a two-layer, width-1024 Mamba and a four-layer PAE.

2) Hyperparameter Sensitivity:

  • Among tested handover ratios, r = 0.4 performs best on all three tasks, with 55.0% average success.This setting allocates four denoising steps to the PAE and six to the frozen AE.Source: 2) Hyperparameter Sensitivity · p. 6
  • Increasing Mamba depth from two to four lowers average success from 55.0% to 43.3%, while increasing width to 1536 raises it to 55.6%.The wider Mamba decreases T6 success from 80.0% to 76.7%; the authors retain the narrower model for similar average performance with less computation.Source: 2) Hyperparameter Sensitivity · p. 6
  • A four-layer PAE has the highest average success among tested depths: reducing depth to two yields 43.9%, while increasing it to six yields 53.9%.The six-layer PAE improves T8 but reduces success on T6 and T7.Source: 2) Hyperparameter Sensitivity · p. 6

D. Real-World Results

Across four real-world tasks, ActMem-VLA improves average success over π0.5 and outperforms it on every task. The conclusion reports gains on tasks requiring stage discrimination, repetition tracking, and transitions, while keeping the fine-tuned base VLA frozen.

D. Real-World Results

  • 63.75% average success for ActMem-VLA versus 35.0% for π0.5 across four real-world tasks, an improvement of 28.8 percentage points.ActMem-VLA succeeds in 51 of 80 trials, compared with 28 of 80 for π0.5.Source: D. Real-World Results · p. 7
  • ActMem-VLA outperforms π0.5 on all four real-world tasks, with the largest gain on mango weighing, where similar configurations recur before and after weighing.The authors report that reliable execution of the three-repetition tasks remains challenging: success is 35% on R3 and 55% on R4, versus 100% on R1.Source: D. Real-World Results · p. 7

V. CONCLUSION

  • The conclusion reports improved completion across ten simulated and four real-world tasks while training only the memory module and PAE and keeping the base VLA frozen.The reported improvements concern tasks requiring stage discrimination, repetition tracking, and transitions to subsequent actions.Source: §V · p. 7
Loading 2609.37307v1…