PaperlayerView exact revision on arXiv

Source-linked AI summary

MemReread: Enhancing Agentic Long-Context Reasoning via Memory-Guided Rereading

Baibei Ji, Xiaoyang Weng, Juntao Li, Zecheng Tang, Yihang Lou, Min Zhang

arXiv:2605.10268v1cs.CLcs.AI

TL;DR

Long-context reasoning suffers from attention dilution, costly quadratic attention, and failures to recover latent evidence during memory-based processing. MemReread uses streaming reading with memory-triggered question decomposition and targeted rereading, optimized by reinforcement learning for adaptive pass selection. Experiments report superior performance over baselines while the method retains linear time complexity with respect to context length.

  • Problem

    Long-context reasoning is limited by attention dilution, quadratic attention costs, permanent latent-evidence loss, and interference from invalid retrieval queries.

  • Method

    MemReread preserves streaming reading, triggers question decomposition and targeted rereading when terminal memory is insufficient, and uses rereading-count-based reinforcement learning to adapt pass counts.

  • Results

    MemReread surpasses baseline frameworks across model scales, with the 4B model achieving up to 12.1% higher accuracy over ReMemR1.

  • Takeaways & Limitations

    MemReread recovers overlooked critical facts while adaptively modulating rereading passes according to task complexity.

  • Takeaways & Limitations

    Retrieval-augmented processing incurs heavy storage, fragmented-information recovery difficulties, and high peak computation costs when multiple chunks are processed together.

Abstract

from arXiv · show

To tackle long-context reasoning tasks without the quadratic complexity of standard attention mechanisms, approaches based on agent memory have emerged, which typically maintain a dynamically updated memory when linearly processing document chunks. To mitigate the potential loss of latent evidence in this memorize-while-reading paradigm, recent works have integrated retrieval modules that allow agents to recall information previously discarded during memory overwriting. However, retrieval-based recall suffers from both evidence loss during memory formation and interference induced by invalid queries. To overcome these limitations, we propose MemReread. Built upon streaming reading, MemReread circumvents intermediate retrieval. It triggers question decomposition and rereading when the final memory is insufficient, enabling the recovery of indirect facts that were prematurely discarded. This design supports non-linear reasoning while preserving the inherent logical flow of document comprehension. To further enhance practicality, we introduce a reinforcement learning framework that enhances length extrapolation capability while dynamically determining the number of rereading passes based on task complexity, thereby flexibly controlling computational overhead. Extensive experiments demonstrate that MemReread consistently outperforms baseline frameworks on long-context reasoning tasks, while maintaining linear time complexity with respect to context length.

1 Introduction

Long-context reasoning is hindered by attention dilution, quadratic attention costs, and information loss in chunk-level memory and retrieval. MemReread addresses these failures by using memory-triggered decomposition and targeted rereading, while adapting rereading effort to task complexity.

  • Long-context tasks degrade when attention dilution obscures critical facts and quadratic attention makes context-window extension costly.
  • Retrieval-based agents can permanently lose latent evidence discarded before later context reveals its relevance.Historical memory cannot recover evidence that was never archived.
  • Invalid retrieval queries arise when agents cannot distinguish overwritten information from facts still residing in unread chunks, introducing interference.Extraneous retrieval content pollutes memory and dilutes critical signals.
  • MemReread decomposes the question when terminal memory is insufficient, rereads with sub-questions, and integrates resulting answers into root memory.Reading acquires information, while reasoning identifies missing information and decides whether another pass is needed.
  • The framework uses a rereading-count advantage calculation to retain information while minimizing rereading overhead, and experiments report performance gains over specialized memory agents.The authors also analyze computational overhead to assess engineering feasibility.

2 Preliminary

The paper formalizes retrieval-augmented memory agents and diagnoses their failure modes, then evaluates rereading as a way to recover overlooked information. Rereading improves both tested memory architectures, while the framework avoids some retrieval-related storage and interference costs.

  • Retrieval-Augmented Memory Agents: Retrieval-augmented memory agents process bounded document chunks sequentially, updating memory through a policy that may use retrieved content.Their process is formulated as a Markov Decision Process.
  • Retrieval-Augmented Memory Agents: Retrieval cannot recover chunk facts discarded before future context establishes their relevance, and ambiguous missing-information states trigger noisy queries.Chunk retrieval also requires retaining raw text and handling fragmented or long inputs.
  • Experimental Setup: The analysis selects baselines that keep context windows below 8K tokens without heavy chunk-level storage.
  • Retrieval Failure Analysis: On the Global Reasoning Task, ReMemR1 shows a negative retrieval effect and severe accuracy oscillations, whereas MemAgent rises more steadily as reading progresses.The analysis attributes this pattern to premature latent-evidence discarding and ineffective-retrieval interference.
  • Memory Agents with Rereading: Rereading detects missing information after a pass, generates a sub-question, rereads the context, answers from sub-memory, and updates the terminal memory.
  • Memory Agents with Rereading: Rereading improves both MemAgent and ReMemR1, with progressively larger gains as the maximum number of rereading passes increases.MemAgent’s streaming paradigm is used as MemReread’s foundation to avoid retrieval-induced interference.

3 Methodology

MemReread extends streaming reading with memory-guided decomposition, targeted rereading, and integration when terminal memory lacks sufficient evidence. Its reinforcement-learning objective combines process and outcome supervision while encouraging efficient rereading.

  • MemReread Framework: MemReread operates through Read, Decompose, Integrate, and Answer phases using one bounded memory throughout execution.The reading stage processes chunks sequentially, while later operations use the terminal memory.
  • MemReread Framework: When terminal memory lacks complete evidence, the agent generates a specific sub-question, rereads the context, and integrates the resulting question-answer pair.Intermediate sub-memory is discarded, while the question-answer pair is retained in terminal memory and decomposition history.
  • Rereading-Adaptive GRPO: The training strategy separates process and outcome advantages to improve length extrapolation and reduce training inefficiency from sparse rewards.Process supervision uses memory-based state rewards, while outcome supervision uses a rereading-adaptive advantage.
  • Rereading-Adaptive GRPO: Rereading-Adaptive outcome rewards favor fewer passes for correct trajectories and more passes for incorrect trajectories within uniform-outcome groups.For mixed-outcome groups, correct trajectories receive positive advantages and incorrect trajectories negative advantages, with rereading count further modulating them.
  • Rereading-Adaptive GRPO: The overall advantage combines process and outcome components, with α controlling the importance of the outcome advantage.The process advantage is indexed by rereading-pass number and chunk index.

4 Experiments

Experiments evaluate MemReread against streaming-memory and retrieval-augmented baselines across long-context datasets, then analyze its accuracy, computational costs, rereading limits, and adaptive training strategy.

  • Experimental Setup: Evaluations use HotpotQA and 2WikiMultiHopQA, with test sequences spanning 8K to 1M tokens and comparisons against MemAgent and ReMemR1.HotpotQA is used in-distribution, while 2WikiMultiHopQA evaluates out-of-distribution length extrapolation.
  • Accuracy: MemReread surpasses both baseline frameworks across model scales, with the 4B model achieving up to 12.1% higher accuracy than ReMemR1.The largest improvements are reported on out-of-distribution datasets.
  • Efficiency: MemReread requires 3–4× MemAgent’s average test time on 2WikiMultiHopQA but adaptively selects rereading passes according to task complexity.The experiment compares sample-wise average runtime across context lengths for MemAgent, ReMemR1, and MemReread.
  • Efficiency: MemReread maintains constant memory-space complexity comparable to MemAgent, whereas ReMemR1’s stored memory scales linearly with context length.The rereading mechanism avoids storing historical memory at every step.
  • Rereading Limits: Increasing the rereading limit improves performance but yields diminishing marginal returns at pc = 4, so the authors select pc = 3 for inference.Each additional limit unit also raises the upper bound of inference time by one full streaming-reading pass.
  • Adaptive Training: Rereading-Adaptive GRPO outperforms standard GRPO while using fewer average reading passes and modulating passes across benchmarks of increasing difficulty.The evaluation covers RULER-QA, LongBench-E-QA, and LongBench-v2 with pc = 3 for both methods.

5 Conclusion

MemReread extends streaming reading with adaptive rereading to recover overlooked critical facts, while Rereading-Adaptive GRPO adjusts rereading to task complexity. The paper reports superior baseline performance and task-adaptive rereading, addressing challenges in efficient long-context reasoning and learnable memory management.

  • Conclusion: MemReread recovers overlooked critical facts through an adaptive rereading mechanism built on the streaming reading paradigm.The framework is proposed to improve long-context reasoning without abandoning streaming reading.
  • Conclusion: Rereading-Adaptive GRPO adaptively modulates rereading passes based on task complexity.It is introduced as the reinforcement learning strategy used to optimize MemReread.
  • Motivation: The work targets the computational bottlenecks of quadratic self-attention and the trade-off between contextual fidelity and overhead in unbounded documents.The conclusion situates MemReread within research on attention-efficient sequence modeling.
  • Conclusion: The paper frames memory operations such as storage, retrieval, and eviction as learnable sequential decisions rather than solely heuristic policies.This positions reinforcement learning as a mechanism for context handling and structured memory management.

A.3 Evaluation of Long-Context Reasoning

Long-context reasoning evaluation has progressed from simple linear tasks to complex multi-step benchmarks. These evaluations expose accuracy limitations when critical evidence appears in intermediate positions.

  • Benchmark Evolution: Long-context reasoning benchmarks have expanded from basic linear tasks to complex, multi-step reasoning tasks.The evaluation literature includes both synthetic and real-world settings.
  • Known Limitations: The lost in the middle phenomenon shows that model accuracy can degrade sharply when critical evidence lies in intermediate sequence positions.Evidence at sequence boundaries is contrasted with evidence located in the middle.
  • Evaluation Scope: Synthetic and real-world evaluations are used to systematically probe long-context reasoning limitations.These evaluations build on observed failures in long-context scenarios.

B.1 Global Reasoning Task

The Global Reasoning Task targets retrieval failure modes by embedding indirect and direct facts non-linearly in long contexts. Its templates and algorithm evaluate question-guided rereading, memory integration, and reinforcement-learning supervision.

  • Task Design: The dataset targets two retrieval failure patterns using Statistics and Variable Tracking tasks derived from RULER-QA.These tasks assess global latent information aggregation and long-range latent dependency tracking.
  • Facts Construction: Indirect facts are crucial for answering but not directly related to the question, whereas direct facts connect the question to those latent facts.The construction explicitly distinguishes direct, indirect, and distractor facts.
  • Statistics: Statistics tasks require revisiting earlier event-location facts after later context identifies the relevant event type and location.The design tests distinguishing relevant events from distractors and counting them.
  • Variable Tracking: Variable Tracking tasks require revisiting preceding system-log entries after later aliases reveal which entity determines the final value.The final value is selected from the entry with the largest sequence number.
  • Context Construction: Contexts use implicit entity substitution, randomly sample 3 to 10 facts, and place direct facts in positions 0.5 to 0.9.Indirect facts are inserted relatively uniformly across padded background contexts.
  • MemReread Workflow: MemReread decomposes the question, rereads all chunks with a targeted sub-question, answers from sub-memory, and integrates the resulting question-answer pair.The process repeats up to a rereading-pass limit or until no further sub-question is generated.

C.4 Evaluation Details

The evaluation compares MemReread with memory, retrieval, and post-training baselines under costly long-context settings. Results report preserved or improved performance, statistical significance, and adaptive rereading efficiency.

  • Evaluation Setup: Evaluations subsample long-context datasets because main experiments incur extraordinary computational cost, while using 4 × NVIDIA A800 GPUs for non-API evaluations.LongBench-v2 is evaluated up to 1M tokens, with excess context truncated.
  • Adaptive Rereading: ReA-GRPO matches GRPO validation performance while modestly reducing average reading passes.Extra readings are triggered selectively for challenging samples.
  • Baseline Comparisons: MemReread consistently outperforms LLM baselines at the 4B scale.The comparison is reported in Table 12.
  • Baseline Comparisons: CDT gains on 2WikiMultiHopQA at 8K and 16K but degrades significantly beyond 32K under the reported setup.The authors attribute this degradation to SFT compromising inherent reasoning capabilities.
  • Baseline Comparisons: MemReread demonstrates better overall performance than InfMem on out-of-distribution datasets at the 4B scale.The comparison is conducted because only 4B InfMem checkpoints are publicly available.
  • Statistical Significance: MemReread significantly outperforms ReMemR1 and MemAgent on 2WikiMultihopQA under paired-sample t-tests with p < 0.05.The tests use sample-level correct and incorrect predictions encoded as 1 and 0.

D.5 Comparison on Additional Benchmarks

Additional benchmarks show that MemReread’s advantage is strongest on ultra-long contexts and narrows on shorter tasks. The pattern is consistent with its emphasis on preserving long-range dependencies and reducing information fragmentation.

  • Benchmark Results: MemReread consistently outperforms all baselines across the evaluated additional long-context benchmarks.The reported benchmarks include Tables 14, 15, and 16.
  • Context-Length Effects: The performance advantage is most pronounced on RULER-QA contexts above 256K and LongBench-v2-1M(Long).The margin narrows on LongBench-QA and LongBench-E-QA, capped at approximately 20K tokens.
  • Context-Length Effects: Shorter tasks have at most four context chunks, reducing the probability of losing critical cross-chunk dependencies.Performance in these settings is predominantly governed by backbone reasoning capability.
  • Interpretation: Because all baselines share the same backbone architecture, short-context performance alignment is expected.The authors connect MemReread’s core advantage to preserving long-range dependencies and mitigating information fragmentation.

E Further Analysis

Further analysis evaluates MemReread across model scales, architectures, framework initializations, and case studies. The findings show broad zero-shot performance, while examples illustrate recovery of indirect facts through targeted rereading and memory integration.

  • Scalability: MemReread consistently outperforms MemAgent and ReMemR1 across Qwen models from 4B and 8B to over 200B and 1000B parameters.The evaluation is conducted in zero-shot, training-free settings on 2WikiMultiHopQA.
  • Portability: MemReread achieves superior zero-shot performance across diverse backbone architectures, indicating cross-architecture generalization in the evaluated settings.The comparison is reported in Table 18.
  • Cross-Framework Transfer: Initializing MemReread with MemAgent weights yields substantial gains even without Rereading-Adaptive RL training.The analysis examines MemReread initialized from MemAgent and ReMemR1 checkpoints.
  • Interpretation: The authors suggest memory-guided rereading may act as an intrinsic reasoning mechanism, but leave rigorous characterization of this dynamic for future work.The claim is presented as a suggestion rather than a settled characterization.
  • Baseline Failures: MemAgent can preserve indirect facts that retrieval-augmented ReMemR1 discards, while ineffective retrieval may introduce irrelevant memories and wrong answers.The cases attribute ReMemR1 failures to premature latent-evidence loss and ineffective or noisy queries.
  • Case Analysis: MemReread resolves indirect facts by decomposing questions, rereading for targeted evidence, and integrating question-answer pairs into root memory.In one case, it first resolves the alias Sector-X to sys_timeout, then extracts the value 4117 during rereading.

F.2 Cases of Main Experiment

Case analyses compare ReMemR1 and MemReread on multi-hop reasoning examples. MemReread progressively resolves bridge entities and retrieves later-hop facts, whereas ReMemR1 can query the wrong entity or fail after discarding needed evidence.

  • Multi-Hop Rereading: ReMemR1 can issue a correct birthplace query after the relevant fact was discarded, but retrieval fails because the birthplace was never stored in historical memory.The case illustrates irreversible evidence loss during earlier memory formation.
  • Multi-Hop Rereading: MemReread identifies Peter Paul Brauer as the director before issuing a targeted birthplace query, recovering Elberfeld, Germany.The intermediate director identity is integrated into memory before the second rereading step.
  • Entity Resolution: ReMemR1 targets Queen Victoria instead of Princess Alexandra’s actual mother, retrieving unrelated facts and producing an incorrect result.The error occurs before full context reading has completed.
  • Entity Resolution: MemReread first resolves an intermediate entity and then rereads for the dependent fact, producing the correct answer in the Princess Alexandra example.It identifies Grand Duchess Maria Alexandrovna of Russia as the mother, then recovers her death date, 24 October 1920.
  • Failure Modes: MemReread errors arise predominantly from inherent reasoning deficits and ineffective integration of newly recovered information.The failure analysis reports misalignment during memory updates and fragility when reconciling conflicting facts.

G Limitation and Future Work

The study identifies limits in task coverage, inference efficiency, and dependence on the model’s intrinsic capabilities. It leaves broader evaluation and more efficient, robust memory mechanisms for future work.

  • Task Generalization: The evaluations primarily target long-context reasoning, so generalization to code understanding, summarization, and long-form generation remains empirically unvalidated.The limitation concerns task generalization beyond the evaluated scope.
  • Inference Latency: MemReread adds a rereading phase and therefore incurs higher inference latency than single-pass streaming approaches.The authors identify latency as an inherent cost of the mechanism.
  • Intrinsic Capabilities: MemReread’s efficacy is bounded by the model’s intrinsic reasoning and memory capabilities, as shown by its observed failure cases.The study reports occasional failures during task execution and ineffective conflict integration.
  • Future Work: Future work includes broader task evaluations, more efficient memory mechanisms, and more robust memory representations.These directions directly address the study’s stated limitations.
Loading 2605.10268v1…