PaperlayerView exact revision on arXiv

Source-linked AI summary

DRAM: Delta-rule Recurrent Associative Memory for Robot Manipulation Policies

Xinyu Zhao, Yixiang Shan, Tao Yang, Runyu Lei, Yiming Zhao, Jiaxin Fan, Zongbao Feng, Peng Jia

arXiv:2609.32453v2cs.ROcs.AI

TL;DR

Robot policies that use only current observations or short windows may miss history needed for partially observable manipulation. DRAM adds a fixed-size recurrent memory to pretrained policies, and the paper reports consistent improvements across diverse tasks, including real-robot evaluations.

  • Problem

    Many robot policies use only current observations or short windows, although partially observable manipulation can require retrieving information from earlier interactions.

    Source: §1 · p. 1

  • Method

    DRAM adds a fixed-size recurrent associative memory that corrects stored associations and retrieves preceding-frame context through a plug-and-play interface.

    Source: §1 · p. 2

  • Results

    DRAM consistently improves pretrained policy performance across diverse memory-dependent tasks, including real-robot evaluations.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Gains are most pronounced on longer multi-stage tasks in the reported simulated and real-world evaluations.

    Source: §6 · p. 10

    Evaluation resets memory at episode boundaries; memory management across successive tasks remains unexplored.

    Source: E LIMITATIONS · p. 17

Abstract

from arXiv · show

Robotic manipulation is inherently history-dependent, yet most pretrained robotic policies condition on only the current observation or a short temporal window. Equipping such policies with long-term memory remains challenging: existing approaches either feed the backbone multi-frame observation windows, which substantially increase inference cost, or rely on pre-defined semantic features, which limit task generality and may also require the retraining of the backbone to adapt to the memory. We introduce DRAM (Delta-rule Recurrent Associative Memory), a plug-and-play memory module that can be attached to a wide range of pretrained robotic policies, endowing them with long-horizon memory without architectural modification or backbone retraining, requiring only task-specific post-training of the memory module and action expert. DRAM maintains a fixed-size associative memory using gated delta-rule linear attention, with a modified update that incorporates all tokens within each frame in parallel. An architecture-agnostic readout integrates historical context into action prediction across different policy architectures. Experiments show that DRAM consistently improves frozen pretrained policies over short-context baselines and alternative compact memory designs, validating its effectiveness as a fixed-size, post-hoc memory module trained with the backbone frozen.

1 INTRODUCTION

DRAM adds a fixed-size recurrent memory to robot policies so they can use task-relevant history beyond short observation windows. Its delta-rule updates and plug-and-play interface support historical context across policy backbones, with consistent improvements reported across diverse tasks.

1 INTRODUCTION

  • Prior methods use recurrent summaries or explicit memory banks, motivating a bounded-state mechanism with an architecture-agnostic interface.Examples include RoboTTT (Jiang et al., 2026), ReMem-VLA (Li et al., 2026), and MemoryVLA (Shi et al., 2026).Source: §1 · p. 1
  • DRAM maintains a fixed-size recurrent state by correcting existing associations with prediction residuals rather than unconditionally accumulating new values.The update retrieves the value currently associated with a key, then writes the residual between that prediction and the new value.Source: §1 · p. 2
  • Its patch-level read-before-write mechanism retrieves historical features from preceding frames before jointly writing the current frame’s information.All patches query the shared pre-update memory; their distinct historical features are fused with current representations for action prediction.Source: §1 · p. 2
  • DRAM’s lightweight residual interface preserves policy components while supporting different backbones, and it consistently improves performance across diverse memory-dependent tasks, including real-robot evaluations.The memory module and action expert require task-specific post-training, while the backbone remains frozen.Source: §1 · p. 2

2 PRELIMINARIES

The preliminaries formulate manipulation as history-dependent action prediction and introduce recurrent memory as a fixed-size representation of past inputs. They then connect linear attention to that recurrence and describe delta-rule updates as correcting stored associations.

2.1 PROBLEM FORMULATION

  • Because current inputs may omit task-relevant information from earlier interactions, memory augments action conditioning with a persistent representation of observation history.At each environment step, the policy receives a visual observation, proprioceptive state, and task instruction, and predicts an action chunk of length C.Source: §2.1 · p. 2

2.2 RECURRENT MEMORY

  • A recurrent memory carries state across successive inputs, with fixed-size storage independent of the number of preceding steps.Source: §2.2 · pp. 2–3
  • Linear attention provides an associative implementation of recurrent memory, compressing historical key–value associations in a matrix-valued state.The paper adopts this formulation while noting that recurrent mechanisms can also use gated vector states or Transformer memory tokens.Source: §2.2 · p. 3

2.3 DELTA-RULE ASSOCIATIVE MEMORY

  • The delta rule updates associations using the residual between a stored value prediction and the incoming value, while a gate controls retention.The method adapts this principle to joint updates over the tokens of each environment frame.Source: §2.3 · p. 3

3 METHOD

DRAM maintains a fixed-size historical state with frame-level recurrence and token-level access. Its read–fuse–write design retrieves past context before jointly updating memory with the current frame.

3.1 FRAMEWORK OVERVIEW

  • Each frame advances one fixed-size state while individual visual patches independently access shared associative memory.This separates the temporal granularity of recurrence from the spatial granularity of memory access.Source: §3 · p. 3

3.2 MEMORY ACCESS AND HISTORICAL FUSION

  • Queries retrieve from the pre-update state, and a gated residual fuses historical readouts into the query stream while preserving the host policy’s conditioning interface.Separate-source hosts consume readouts as additional context; shared-source hosts consume the fused token stream.Source: §3.1 · p. 4, §3.2 · p. 5

3.3 FRAME-SYNCHRONOUS JOINT WRITE

  • One frame triggers a joint delta-rule correction over its key–value pairs, avoiding serialization-induced order bias while keeping repeated evidence accumulation bounded and saturating.The joint write is invariant to within-frame token permutations in exact arithmetic.Source: §3.3 · pp. 5–6

3.4 TRAINING WITH TRUNCATED BACKPROPAGATION THROUGH TIME

  • Training freezes the vision tower and VLM while optimizing memory, adapter, and action expert through truncated BPTT under the control objective, without auxiliary memory-specific losses.At deployment, each observation triggers one read–write transition; memory persists across action chunks and resets across episodes.Source: §3.4 · pp. 6–7

4 EXPERIMENTS

Across LIBERO and RMBench, DRAM performs strongly, including on tasks requiring longer execution or episodic memory, though its advantage is not uniform. Results also show benefits across policy backbones, sensitivity to memory resets, and real-world gains on a multi-stage task.

4.1 SETUP

  • The benchmark covers four LIBERO suites, and comparisons include memory approaches using different backbones and history representations.LIBERO-10 is identified as especially relevant to longer execution horizons; published methods have different training and evaluation protocols.Source: §4.1 · p. 7, §4.2 · p. 7

4.2 MAIN RESULTS

  • DRAM averages 99.00% success across LIBERO, exceeding HAMLET’s 97.60%; it reaches 99.00% on LIBERO-10, 3.20 points above µVLA, but trails HAMLET on LIBERO-Goal.It matches the best reported results on LIBERO-Spatial and LIBERO-Object.Source: §4.2 · p. 7
  • On LIBERO, adding DRAM raises DiT’s average success from 84.31% to 94.63% and π0.5’s from 97.50% to 99.00%.DiT’s LIBERO-10 success rises from 75.00% to 91.00%, a gain of 16.00 percentage points; π0.5 maintains 98.00% on Goal.Source: §4.2 · pp. 7–8
  • On RMBench, DRAM averages 57.8%, 15.8 percentage points above Mem-0, and leads on eight of nine tasks, though the comparison uses different published protocols.It trails Mem-0 on Put Back Block, 65% versus 90%.Source: §4.2 · p. 8, Table 3 · p. 8

4.3 FLEXIBILITY OF MEMORY INTEGRATION

  • Among three π0.5 memory write-source configurations, Native KV averages 99.00%, 1.50 percentage points above baseline; Hidden Tokens reaches 98.63%, while Last Hidden matches the 97.50% baseline.The results indicate that write-source choice affects DRAM’s benefit even when the memory update rule is unchanged.Source: §4.3 · pp. 8–9, Table 4 · p. 8

4.4 SENSITIVITY TO MEMORY RESET INTERVAL

  • On LIBERO-10, resets every 8, 32, or 128 frames yield approximately 73–76% success, versus 99.0% with full memory and 96.0% for π0.5 without DRAM.A 256-frame reset interval reaches 91.3%, still 7.7 points below full memory and below the no-memory baseline; the experiment measures sensitivity to interruptions, not a precise memory horizon.Source: §4.4 · p. 9, Figure 2 · p. 9

4.5 REAL-WORLD EXPERIMENTS

  • In real-world trials, DRAM raises insertion success from 90.0% to 100.0% and full-task success from 0.0% to 85.0%.The larger gain is on the multi-stage full task.Source: §4.5 · p. 9, Table 5 · p. 9

5 RELATED WORK

Prior robotic-policy memory approaches represent and integrate history in different ways, including feature banks, compressed tokens, and recurrent states. DRAM uses fixed-size token-level associations and frame-wise joint updates, retrieving from the pre-update state.

5.1 MEMORY FOR ROBOTIC MANIPULATION

  • Robot-policy memory methods include explicit feature banks, compressed history tokens, and recurrent states; DRAM instead uses token-level key–value associations and retrieves before a joint frame update.MemoryVLA uses a feature bank; HAMLET and ContextVLA compress history into tokens; µVLA and ReMem-VLA propagate recurrent states; MEM combines visual context with a recurrent language summary.Source: §5.1 · pp. 9–10

5.2 RECURRENT ASSOCIATIVE MEMORY

  • Linear attention stores historical key–value associations in a fixed-size recurrent matrix, while delta-rule updates correct existing predictions rather than repeatedly accumulating complete values.Gated DeltaNet combines associative correction with data-dependent forgetting.Source: §5.2 · p. 10

5.3 FRAME-WISE DELTA UPDATES

  • Because tokens within a frame are simultaneously available and their serialization has no temporal meaning, DRAM retrieves from pre-update memory and incorporates the frame through one joint update.Related concurrent work also considers frame-wise writing or a per-frame ridge problem; DRAM’s joint write is in external policy-agnostic memory attached to a frozen policy and trained with the control objective.Source: §5.2 · p. 10, §5.3 · p. 10

6 CONCLUSION

DRAM adds persistent historical context to pretrained robot policies through fixed-size recurrent associative memory. Its frame-wise joint update and separable policy integration support varied interfaces, with experiments reporting gains on simulated and real-world tasks, especially longer multi-stage tasks.

6 CONCLUSION

  • DRAM augments pretrained robot policies with persistent historical context using token-level associations in fixed-size memory, reading before a joint frame update without imposing token order within a frame.Separating memory operations from policy integration supports different representation interfaces while preserving host-policy components; experiments show gains on simulated and real-world tasks, most pronounced on longer multi-stage tasks.Source: §6 · p. 10

7 AUTHOR CONTRIBUTIONS

The author-contribution statements assign experimental, analytical, and writing roles, while the included integration-pattern passage describes architecture-dependent interfaces for the same memory operations.

7 AUTHOR CONTRIBUTIONS

  • Xinyu Zhao led experiments and contributed to analysis and manuscript writing; Yixiang Shan led experimental design, analysis, and the initial draft, while other authors reviewed and revised the manuscript.Zongbao Feng and Peng Jia supervised the research.Source: §7 · p. 10

B FLEXIBLE INTEGRATION PATTERNS

  • DRAM supports separate-source and shared-source integration patterns; the host architecture determines the pattern, while READ, WRITE, and FUSE operations remain the same.The paper instantiates the patterns with π0.5 and DiT hosts.Source: B FLEXIBLE INTEGRATION PATTERNS · p. 12

B.1 SEPARATE-SOURCE INTEGRATION: THE π0.5 HOST

DRAM can integrate historical context through separate-source or shared-source designs while preserving each host policy’s token stream or original interface. The π0.5 host uses layer-aligned memory and prefix readouts, whereas the DiT host mixes history into its frame tokens.

B.1 SEPARATE-SOURCE INTEGRATION: THE π0.5 HOST

  • The π0.5 design decouples learned query slots from backbone write sources, leaving the host token stream untouched.Memory layers align with VLM backbone layers; write sources can be Native KV, Hidden tokens, or Last hidden states.Source: §B.1 · p. 12, Figure 3 · p. 13
  • Fused π0.5 readouts enter the action expert as prefix slots, with a zero-initialized projection that initially leaves query representations unchanged.The prefix slots use a reserved RoPE index while the action expert’s original tokens retain their position indices.Source: §B.1 · p. 12

B.2 SHARED-SOURCE INTEGRATION: THE DIT HOST

  • The DiT host uses current-frame tokens as DRAM queries, keys, and values, then passes history-fused tokens through the action head’s unchanged cross-attention interface.Its memory depth can be chosen independently of the host, and the fused sequence preserves token count, ordering, and positional encoding.Source: §B.2 · p. 13
  • Across both integration patterns, history-carrying tokens preserve the host’s positional scheme, while readout granularity is chosen independently of write-source choice.Layer-aligned readouts suit deep hosts; a single final-layer readout can suit shallow hosts or single-layer memory.Source: §B.2 · p. 13

C DERIVATION AND PROPERTIES OF THE FRAME-LEVEL JOINT WRITE

The frame-level joint write fits all current key–value associations against one retained state, avoiding order-dependent sequential updates. An anchor term preserves unaddressed history and makes the objective uniquely solvable, yielding aggregate and correction forms of the update.

C.1 NORMAL EQUATION, UNIQUENESS, AND CLOSED FORMS

  • The joint write evaluates every frame token’s prediction error against one candidate state, avoiding the order dependence of sequential token updates.The objective balances fitting current associations against staying close to the retained state, leaving components unaddressed by current keys unchanged.Source: §C.1 · p. 14
  • The anchor makes the objective strictly convex and its minimizer unique, even when current keys are repeated or rank-deficient.Without the anchor, the fit term alone may have non-unique solutions when keys do not span the key space.Source: §C.1 · p. 15
  • The update has aggregate and correction forms; the correction form adds a coupled correction from prediction residuals to the retained state.The correction form is computed in practice and is equivalent to a single Newton step that reaches the quadratic objective’s minimizer.Source: §C.1 · p. 15

C.2 SINGLE-TOKEN CORRESPONDENCE

The frame-synchronous joint write reduces exactly to the gated delta rule for one association and has order-invariant, bounded behavior for multiple tokens. The appendix also describes masking and numerical procedures for solving and differentiating through the update.

C.2 SINGLE-TOKEN CORRESPONDENCE

  • The frame-synchronous joint write exactly recovers the gated delta rule with correction strength β̂, so the gated delta rule is the N = 1 special case of the joint write.The cap β̂ ≤ 0.99 bounds w ≤ 99, keeping A well conditioned.Source: §C.2 · p. 16

C.3 PERMUTATION INVARIANCE AND BOUNDED ACCUMULATION

  • Permuting a frame’s key–value tokens and their weights leaves the unique written state unchanged.Thus spatial raster, camera identity, or language order cannot influence the state written for that frame.Source: §C.3 · p. 16
  • Repeated identical evidence increases correction strength toward 1, while keeping it below 1 for any finite repetition count.Unlike token-count averaging, the effective strength increases with the number of repetitions.Source: §C.3 · p. 16

C.4 NUMERICAL REALIZATION: MASKING, PRECISION, AND GRADIENT FLOW

  • Padded tokens have zero write weight, and fully padded frames leave the previous state untouched; gradients pass through the solve by implicit differentiation.The backward pass reuses the stored factorization.Source: §C.4 · pp. 16–17
  • The gate cap and anchor bound the update system’s conditioning independently of the data, with an upper bound linear in write-token count.The cap bounds accumulated weights, while the anchor supplies a data-independent lower eigenvalue bound.Source: §C.4 · p. 16
  • The implementation solves the update with a per-frame Cholesky factorization and reuses that factorization across value columns.The Gram matrix and solve use FP32; the stated per-frame cost is independent of episode length.Source: §C.4 · p. 16

D DETAILS OF REAL-WORLD EXPERIMENTS

The real-world evaluation uses an I7 Pro mobile manipulation platform on two CNC-workspace tasks, with 20 trials for each task and method.

D DETAILS OF REAL-WORLD EXPERIMENTS

E LIMITATIONS

The evaluation is limited to episodic manipulation, with memory reset at episode boundaries; memory management across successive tasks remains unexplored.

E LIMITATIONS

  • Evaluation covers episodic manipulation, with memory reset at episode boundaries.Source: E LIMITATIONS · p. 17
  • Memory management during continuous execution across successive tasks remains unexplored.Source: E LIMITATIONS · p. 17
  • Future work could investigate when to retain or reset memory as task instructions change.Source: E LIMITATIONS · p. 17

Related papers

Loading 2609.32453v2…