PaperlayerView exact revision on arXiv

Source-linked AI summary

MemDuet: Learning Dual-Process Memory for Long-Context Reasoning

Jiangcheng Song, Haodong Jing, Yulin Mu, Lesong Tao, Jixin Wang, Jin Cui, Tianjing Ren, Runxuan Xie, Yongqiang Ma, Nanning Zheng

arXiv:2610.19928v1cs.CL

TL;DR

As context grows, reasoning performance and inference efficiency remain tightly coupled, while existing recurrent-memory control can be sequential or operate on compressed evidence. MemDuet combines parallel selection of raw chunks by a small model with large-model verification and memory updating, and reports a favorable accuracy–efficiency trade-off across long-context benchmarks.

  • Problem

    As context grows, recurrent-memory control can remain sequential or use compressed evidence, leaving reasoning performance and inference efficiency tightly coupled.

    Source: §1 · pp. 1–2

  • Method

    MemDuet trains a small model to select raw context chunks in parallel and a large model to verify memory updates and rewrite memory, using supervised, GRPO, and joint optimization stages.

    Source: §1 · p. 2

  • Results

    MemDuet improves the performance–efficiency trade-off over full-update recurrent memory agents across three benchmarks, with higher reasoning accuracy and substantially lower inference cost at contexts up to 3.5M tokens.

    Source: §1 · p. 2

  • Takeaways & Limitations

    MemDuet’s reported accuracy–efficiency trade-off becomes more favorable as context grows.

    Source: §1 · p. 2

    Evaluation covers three long-context benchmarks and mainly Qwen-family backbones, leaving generalization to other domains and model families uncertain.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Reasoning over infinitely long documents is becoming a reality with recurrent memory agents. However, increasing context length exacerbates the trade-off between reasoning performance and inference efficiency. Inspired by Kahneman's account of fast and slow thinking, we propose MemDuet, a dual-process memory framework that learns to coordinate a small language model for fast, parallel evidence selection from raw context chunks (System 1) with a large language model for memory-update verification and memory updates in document order (System 2). We train MemDuet in three stages: supervised fine-tuning of System 1 for evidence selection, Group Relative Policy Optimization (GRPO) of System 2 for verification and memory rewriting, and joint optimization of both policies to coordinate selection and updating for long-context reasoning. Experiments on diverse long-context reasoning benchmarks (RULER-HQA, LongBench v2, and 2WikiMultiHopQA) demonstrate a favorable accuracy-efficiency trade-off as context length grows. Averaged across evaluation settings on the three datasets, MemDuet achieves an end-to-end inference speedup of 9.74x and an output-token reduction of 73.91% relative to RL-MemAgent.

1 Introduction

MemDuet addresses the coupling of reasoning performance and inference efficiency as context grows by combining parallel selection over raw chunks with memory-update verification and rewriting. Its three-stage training coordinates the two systems, and experiments report a favorable performance–efficiency trade-off on long-context benchmarks.

1 Introduction

  • As context grows, existing recurrent-memory control decisions remain sequential or act on compressed evidence, while reasoning performance and inference efficiency remain tightly coupled.The introduction also notes that extending context windows, sparse or linear attention, and retrieval-augmented generation leave reasoning over unbounded documents unresolved.Source: §1 · pp. 1–2
  • A three-stage strategy trains System 1 with supervision, optimizes System 2 with GRPO, then jointly optimizes both systems under a shared reasoning objective.Joint task-level rewards allow evidence selection and memory updating to co-adapt.Source: §1 · p. 2
  • Across RULER-HQA, LongBench v2, and long-context 2WikiMultihopQA, MemDuet improves the performance–efficiency trade-off over full-update recurrent memory agents, scaling to 3.5M-token contexts with higher reasoning accuracy and substantially lower inference cost.Source: §1 · p. 2
  • MemDuet uses a small language model to select candidate chunks in parallel from raw context and a large language model to verify and rewrite memory.System 2 makes memory updates in document order under the current memory.Source: §1 · p. 2

2 Related Work

Prior recurrent-memory methods make memory management task-driven, but their control remains in a sequential loop or operates on compressed evidence. MemDuet instead brings cognitive dual-process specialization to recurrent memory through parallel selection of raw chunks and subsequent verification and consolidation.

2.1 Recurrent Memory for Long-Context Reasoning

  • MemAgent, InfMem, GRU-Mem, and LycheeMemory learn or optimize memory management, but their control remains sequential or uses already-compressed evidence.MemDuet instead scans raw chunks in parallel before recurrent updating and coordinates selection with consolidation.Source: §2.1 · p. 2

2.2 Cognitively Inspired Language Models and Memory

  • Earlier cognitively inspired systems apply cognitive principles to memory organization, retrieval, or reasoning-effort allocation; MemDuet specializes System 1 selection and System 2 consolidation within recurrent memory.System 1 selects raw candidate chunks in parallel, while System 2 verifies whether they warrant updates under current memory.Source: §2.2 · p. 3

3 Methodology

MemDuet processes query-conditioned documents as ordered chunks with bounded textual memory, alternating parallel candidate selection with document-order verification and memory updates. Its three training stages establish selection, train memory updating, and then jointly optimize both policies.

3.1 Preliminaries

  • The task is to answer a query using evidence from a document that may exceed the model’s context window, divided into ordered chunks with a fixed chunk-size budget.The query remains available throughout processing.Source: §3.1 · p. 3

3.2 MemDuet Workflow

  • MemDuet alternates parallel raw-chunk selection by System 1 with System 2 verification and memory updates in document order.System 1 evaluates windows using the shared query and current memory; the selected candidate chunks are passed to System 2.Source: §3.2 · p. 3, §3.2.1 · pp. 3–4

3.2.1 System 1: Memory-Conditioned Parallel Candidate Selection

  • System 1 independently evaluates a window of up to k consecutive chunks under a shared query and memory, selecting chunks as candidates for updating.The candidates retain their original text for forwarding to System 2.Source: §3.2.1 · pp. 3–4

3.2.2 System 2: Speculative Verification and Memory Consolidation

  • System 2 independently assesses candidate chunks, commits the first accepted chunk in document order, and generates replacement memory; later chunks are reconsidered under updated memory.If none is accepted, it retains current memory and advances past the window.Source: §3.2.2 · p. 4

3.3 Training of MemDuet

  • Training proceeds through System 1 warm-up, System 2 reinforcement learning with System 1 frozen, and joint optimization of both systems with task rewards.Stage II trains verification, memory rewriting, and answer generation; Stage III coordinates selection with verification and updates.Source: §3.3 · p. 4

3.3.1 Stage I: System 1 Supervised Warm-up

  • Stage I trains memory-conditioned evidence selection with weighted action classification and a paired-memory hinge loss.Evidence-containing chunks are labeled Update and other chunks Skip; the paired-memory objective contrasts the original example with a post-update memory negative.Source: Figure 3 · p. 4, §3.3.1 · p. 4

3.3.2 Stage II: System 2 Reinforcement Learning

  • Training samples G complete trajectories from behavior-policy snapshot θold, computes group-relative advantages from final-answer task rewards, and applies clipped GRPO with generated-token KL; its event set includes verification commitments and generated tokens.Source: §3.3.2 · p. 5

3.3.3 Stage III: Joint Policy Optimization

  • Stage III jointly optimizes the systems’ LoRA adapters using rollout groups and task-level advantages, adding selection events while retaining commitment and generation ratios and generation-only KL regularization.Selection events record both Skip and Update actions.Source: §3.3.3 · p. 5

4 Experiments

MemDuet pairs small-model evidence selection with large-model verification and document-order memory updates. Its accuracy–efficiency advantages grow with context length, while analyses identify training, verification, and ordered consolidation as important design choices.

4.2 Main Results

  • At 3.5M tokens, MemDuet beats RL-MemAgent by 3.90 points, with a 49.61× speedup and 99.14% fewer output tokens on RULER-HQA.The fixed System 1 scanning and coordination cost is less easily offset at 7K; at longer lengths, overhead is amortized and unnecessary System 2 processing can be avoided.Source: §4.2 · p. 6, Table 1 · p. 6
  • MemDuet has the highest option accuracy on Long and Ultra LongBench v2 groups, with 3.00× and 8.03× speedups, and leads on 2WikiMultihopQA at 224K and 896K tokens.It trails on LongBench v2 Short and Medium and on 2WikiMultihopQA at 14K and 56K; the reported advantage is concentrated in longer contexts.Source: §4.2 · pp. 6–7, Table 2 · p. 7

4.3 Analysis

  • Figure 4 compares latency and output-token scaling from 7K to 3.5M tokens; MemDuet’s latency and output tokens grow more slowly than RL-MemAgent’s.MemDuet has a small latency overhead at 7K, and latency savings are smaller than token savings because input processing and System 1 computation remain necessary.Source: §4.3 · p. 7, Figure 4 · p. 7
  • Removing draft–verify or state reevaluation reduces accuracy by up to 3.90 points; removing draft–verify also lowers speedup to 9.89×.The ablations retain System 1 selection without verification, or reuse decisions made with window-start memory; the k = 1 variant preserves accuracy, with speedup changing from 12.16× to 11.84×.Source: §4.3 · pp. 7–8, Table 3 · p. 7
  • Without Stage I, speedup falls from 12.16× to 2.54× and mean output-token ratio roughly triples; without Stage II, rewriting weakens, while Stage III recovers 2.34–6.25 accuracy points over omitting it.The reported curriculum learns when to update, how to update, and how to coordinate the two decisions.Source: Table 3 · p. 7, §4.3 · p. 8
  • Across Qwen2.5 and Qwen3.5 backbones, MemDuet reaches 80.08–83.36% downstream accuracy with mean speedups of 5.69–14.26×; Qwen3.5-2B has the highest final accuracy and mean speedup.Qwen3.5-0.8B instead has the highest selection F1 and lowest output-token ratio, so selection quality is considered alongside answer quality and inference cost.Source: §4.3 · p. 8, Table 4 · p. 8
  • The best parallelism speedup is 1.111× at k = 8 and 1.75M tokens, while k = 16 and k = 32 slow inference at every evaluated length.System 2 updates memory in document order and reevaluates later chunks, so speculative work can be discarded and synchronization can outweigh concurrency.Source: §4.3 · p. 8, Table 5 · p. 8
  • Without additional training, the proprietary-API setup reduces latency by 70.7–71.8% and cost by 58.2–59.9%, with option accuracy changing by at most 0.60 points.A smaller model proposes chunks and a stronger model accepts or rejects each proposal before memory rewriting.Source: §4.3 · pp. 8–9, Table 6 · p. 8

5 Conclusion

MemDuet combines small-model parallel evidence selection with large-model verification and document-order memory updates. Three long-context benchmarks show a favorable accuracy–efficiency trade-off, with greater efficiency gains at longer contexts.

5 Conclusion

  • Across three long-context reasoning benchmarks, MemDuet shows a favorable accuracy–efficiency trade-off, with greater efficiency gains at longer contexts.Its framework combines memory-conditioned parallel selection by a small language model with large-model verification and document-order updates.Source: §5 · p. 9

Limitations

The evaluation and method have scope boundaries: evidence comes from three long-context benchmarks and mainly Qwen-family backbones, and short inputs and parallel execution present specific challenges.

Limitations

  • Evaluation covers three long-context benchmarks and mainly Qwen-family backbones, leaving generalization to other domains and model families uncertain.Selective processing may discard useful evidence on shorter inputs, System 2 cannot recover evidence missed by System 1, and document-order updates limit parallelism’s practical benefits.Source: Limitations · p. 9

Ethical Considerations

The ethical considerations address risks in the resources and processing, privacy and misuse concerns, and the limits of what efficiency and benchmark results establish.

Ethical Considerations

  • Existing benchmarks and pretrained models may contain inaccurate, biased, or sensitive material, and benchmark performance alone does not establish fairness, privacy protection, or deployment suitability.The section calls for respecting applicable licenses and access conditions.Source: Ethical Considerations · p. 9
  • Selective processing may omit relevant evidence or minority viewpoints, while repeated updates may preserve unsupported claims or discard qualifications.The verifier is not an independent guarantee of factual correctness; consequential uses call for access to original documents, evidence inspection, and human review.Source: Ethical Considerations · p. 9
  • Long documents, intermediate memories, and generated answers may expose personal or confidential information.The section recommends minimizing sensitive data, restricting access to stored memories and logs, and setting retention and deletion procedures.Source: Ethical Considerations · p. 9
  • Lower inference cost may improve access to document analysis while also facilitating large-scale surveillance or unauthorized profiling.Reduced latency and output-token counts do not directly measure energy use or carbon emissions; environmental benefits require dedicated measurement that includes training and total usage.Source: Ethical Considerations · p. 9

A.1 Dataset Construction

This block describes benchmark construction, evaluation protocols, and the data and procedures used to train and run MemDuet, from supervised selection through joint optimization.

A.1 Dataset Construction

  • RULER-HQA tests 128 fixed questions across ten context lengths from 7K to 3.5M tokens, while 2WikiMultihopQA uses four document-count settings across the same 128 questions.Both constructions add distractors while retaining supporting documents; LongBench v2 is grouped by context length across all 503 questions.Source: §A.1 · p. 11

A.2 Models and Baselines

  • The comparison includes direct-context base models, retrieval-augmented systems, and recurrent memory agents, with Qwen2.5-7B-Instruct as System 2 and several Qwen2.5/Qwen3.5 small models as System 1.The listed agent baselines incrementally maintain and update memory; direct-context baselines answer from the supplied context without an external retrieval or memory-update workflow.Source: §A.2 · p. 12

A.3 Evaluation Metrics and Protocol

  • Open-ended QA is scored with normalized substring exact match, LongBench v2 with option accuracy, and inference efficiency with latency speedup and output-token ratio relative to MemAgent.Latency includes initialization, selection, verification, rewrites, and answering, while output-token counts include both systems’ outputs.Source: §A.3 · p. 12

B.1 Stage I: Memory-Contrastive Supervised Warm-up

  • Stage I trains System 1 to UPDATE on evidence chunks and SKIP on other chunks, including paired SKIP examples using the teacher’s post-update memory.The training set has 195,111 examples: 41,180 UPDATE, 112,751 natural SKIP, and 41,180 post-update SKIP examples.Source: §B.1 · p. 13

B.2.1 Dataset Construction; B.2.2 Reward Function; B.2.3 Training Configuration

  • Stage II uses question-context-answer examples with center-truncated contexts of at most eight ordered 5,000-token chunks, then trains System 2 using binary final-answer rewards and group-relative advantages.For each question, 16 trajectories are sampled; System 1 is fixed while System 2 handles candidate scoring, acceptance, and committed memory rewrites.Source: §B.2.1 · p. 13, §B.2.2 · p. 13, §B.2.3 · pp. 13–14

B.2.4 Training Pseudocode; B.3.4 Training Pseudocode

  • The Stage II and Stage III pseudocode describes rollout collection, event-loss replay, policy updates, and validation or checkpointing.Stage III collects fresh complete rollouts and updates both adapters after backpropagating their gradients.Source: §B.2.4 · p. 14, §B.3.4 · pp. 15–16

B.3 Stage III: Joint Optimization of Selection and Consolidation

  • Stage III reuses the Stage II data and answer reward, generates fresh online trajectories from both policies, and assigns each trajectory’s advantage to selection, verification, memory-generation, and answer events.Both backbones remain frozen while the systems’ adapters are optimized; selection and verification events receive no KL term.Source: §B.3.1 · p. 15, §B.3.2 · p. 15, §B.3.3 · p. 15

B.4 Complete MemDuet Inference

  • At inference, System 1 scores chunks in parallel within a window, System 2 scores selected candidates, and the earliest accepted candidate triggers a memory rewrite before processing continues.If no candidate is accepted, memory stays unchanged and the cursor advances; the final answer is generated from the question and final memory.Source: §B.4 · p. 16

B.5 Training-Free Transfer to Proprietary APIs

The API adaptation transfers MemDuet’s roles and controller workflow to fixed proprietary models without additional training. It replaces probability-based acceptance with text-label verification.

B.5 Training-Free Transfer to Proprietary APIs

  • The transfer uses off-the-shelf DeepSeek or OpenAI models as Systems 1 and 2, adapting their roles through prompts and an external controller rather than SFT or GRPO.The models’ parameters remain fixed.Source: §B.5 · pp. 16–17
  • System 1’s generated k/u labels skip or forward chunks, while System 2’s labels reject or accept candidates instead of using action log-probabilities and probability-ratio acceptance.The API workflow does not compute acceptance ratios or draw separate acceptance random variables.Source: §B.5 · p. 17
  • API requests use 5,000-token chunks, a four-chunk lookahead, and a 1,024-token memory budget; the listed DeepSeek prices are off-peak rates, with peak rates twice as high.The passage identifies Table A3 as listing USD prices per million tokens but supplies no price values here.Source: §B.5 · p. 17, Table A3 · p. 17

C.1 RULER-HQA Results

The supplied results passages identify the scope and metrics of the RULER-HQA comparison and describe the acceptance rule’s distinction from distribution-preserving speculative decoding.

C.1 RULER-HQA Results

  • Table A4 reports RULER-HQA answer accuracy, time speedup, and relative output-token counts across context lengths, using MemAgent-7B as the baseline for both efficiency metrics.Source: §C.1 · p. 17

C.2 System 1 Backbone Comparison

  • Table A5 equally weights the ten context lengths from 7K to 3.5M for Accuracy, Speed, and Tokens; Speed and Tokens are ratios of MemDuet to RL-MemAgent latency and output-token counts, respectively.Source: §C.2 · p. 17

D.1 Difference from Distribution-Preserving Speculative Decoding

  • MemDuet’s sampled proposal-and-acceptance rule does not preserve System 2’s standalone action or answer distribution; with UPDATE probabilities 0.4 and 0.6, selection and acceptance probability is 0.4.Unlike standard speculative decoding, rejection does not trigger residual resampling; distributional equivalence is not a guarantee of answer correctness.Source: §D.1 · p. 17

D.2 Quantified Coverage under High Recall

This appendix states conditional bounds on reference-positive action coverage, specifies the prompts used across MemDuet’s stages, and traces memory evolution in case studies.

D.2 Quantified Coverage under High Recall

  • 0.99 selector recall and 0.99 mean acceptance yield 0.9801 reference-positive action coverage, not answer accuracy.The verifier cannot recover a candidate omitted by System 1; a sufficient acceptance condition is v_t,i ≥ 1−η on selected reference-positive states.Source: §D.2 · p. 17
  • A union bound limits the chance of skipping or rejecting any of at most L positive decisions, but aggregate held-out recall alone does not establish the needed conditional bounds.With acceptance probability one on positives, the bound reduces to p_miss ≤ min{1, Lε}; live decisions exclude stale results discarded after an earlier rewrite.Source: §D.2 · pp. 17, 19

E.1 Candidate Selection

  • The prompt templates define shared placeholders and make System 1’s chunk decision a single k/u token, with parallel calls using the same committed memory.For supervised before/after pairs, the memory placeholder and target action change; the task text need not be duplicated.Source: E Prompt Templates · p. 19, §E.1 · p. 19

E.2 Candidate Verification

  • System 2 verifies selected chunks using a first-position k/u decision and action scores for open-weight inference; the API adaptation parses labels without scores.The verification interface uses no separate natural-language accept/reject prompt or small-model memory draft.Source: §E.2 · pp. 19–20

E.3 Memory Initialization and Updating

  • Memory initialization and updates produce a complete replacement memory, retaining relevant prior details while adding useful information from the new chunk.The local MemAgent full-update baseline shares this template.Source: §E.3 · p. 20

E.4 Final Answer Generation

  • Final answering uses the question and final committed memory, while LongBench v2 supplies the question, four options, and letter-answer instruction across stages.The final LongBench answer is an option letter; the answer prompt does not include raw documents or discarded candidates.Source: §E.4 · p. 20, §E.5 · p. 20

F. Case Studies

  • The case studies include three successful trajectories and one failure; one successful path has the answer evidence at initialization and no later memory generation.That trajectory’s c9 candidate is rejected, while the final answer matches the reference; its generation count is not a latency measurement or general efficiency estimate.Source: F Case Studies · p. 20, §F.1 · p. 22

F.1 Successful Reasoning and Memory Evolution

  • A two-hop trajectory joins the film–novelist fact in c3 with the birth-date fact in c5, with System 2 generating the displayed memories.Provisional acceptance and commitment in document order are separate events.Source: §F.1 · p. 21

F.2 Failure Analysis

The failure trace ends with the director’s birth date as the answer, although the reference is the novelist’s birth date.

F.2 Failure Analysis

  • The c6 rewrite binds director Clive Donner’s birth date to the novelist; later evidence identifies Hunter Davies but does not repair the date.The trace does not establish which gate decision caused Davies’s birth date to be omitted.Source: §F.2 · p. 22
  • 21 January 1926 is the predicted answer, versus the reference 7 January 1936; four memory generations were used, including initialization.The final memory identifies Davies but lacks his birth date, retains Donner’s date, and adds an unsupported age ordering.Source: §F.2 · p. 22
  • This trace establishes neither a causal false SKIP nor failure prevalence, and its shorter successful comparison is uncontrolled for evidence order and distractors.Acceptance does not certify truth.Source: §F.2 · p. 22

Related papers

Loading 2610.19928v1…