PaperlayerView exact revision on arXiv

Source-linked AI summary

CoEM: Empowering Long-Context Reasoning with Commit-on-Evidence Memory

Jingguang Li, Yebo Wu, Zuyi Guo, Kailang Ma, Xianjie Dai, Han Zheng, Benwang Chen, Li Li, Can Rong, Heye Huang

arXiv:2609.36935v2cs.AIcs.CL

TL;DR

Long-context reasoning is challenged by distant, sparse evidence, and recurrent memory can discard details before their relevance is clear. COEM retains unresolved excerpts for later decisions and verifies proposed facts against their sources; across benchmarks and backbones it outperforms budget-matched memory agents, including by 10.4–11.4 F1 points at 6,400 documents with Qwen3.5-9B.

  • Problem

    As context grows, relevant evidence becomes sparse among distractors and long-distance dependencies are difficult to identify and integrate, while immediate memory updates can discard details later needed for reasoning.

    Source: §1 · pp. 1–2

  • Method

    COEM keeps unresolved source excerpts verbatim in a bounded pending set, revisits them as context arrives, and uses a frozen verifier to check proposed facts before commitment.

    Source: §1 · pp. 2–3, §3.3 · p. 5

  • Results

    10.4–11.4 F1 points over the strongest memory baseline at 6,400 documents with Qwen3.5-9B; COEM consistently outperforms budget-matched memory agents across three benchmarks and two backbones.

    Source: §5 · p. 9

  • Takeaways & Limitations

    COEM’s results suggest that long-context memory can benefit from preserving unresolved evidence until later context clarifies its relevance.

    Source: §5 · p. 9

    The frozen verifier establishes support by supplied passages, not truth beyond them or correctness of the final answer; verifier errors and evidence loss under finite memory remain possible.

    Source: ETHICS STATEMENT · p. 10

Abstract

from arXiv · show

Long-context reasoning is essential for complex and long-horizon tasks, yet the performance of large language models (LLMs) degrades as context length increases. Recent approaches address this by processing input chunk by chunk while maintaining a bounded textual memory in model context. However, premature information compression can discard critical details essential for subsequent reasoning. In this paper, we introduce Commit-on-Evidence Memory (CoEM), which learns when to convert source evidence into compact memory facts. Specifically, under a fixed context-memory budget, CoEM preserves potentially useful source excerpts verbatim in a pending set, allowing subsequent context to clarify their relevance before irreversible compression. As new context arrives, a learned policy revisits each pending excerpt and decides whether to promote it to the committed memory, retain it for further consideration, or discard it. A frozen verifier ensures proposed facts are accepted only if supported by retained excerpts and current context. To further guide effective memory management, we train this policy using reinforcement learning by combining fine-grained, step-level evidence rewards with final answer rewards. Extensive experiments demonstrate that CoEM consistently improves long-context reasoning. When evaluated on 6,400 documents long-context input, CoEM outperforms the strongest memory baseline by 10.4-11.4 F1 points on Qwen3.5-9B. Code repository: https://github.com/benmagnifico/CoEM.

1 INTRODUCTION

Long-context reasoning is difficult because relevant evidence is sparse among distractors and distant dependencies are hard to integrate, while recurrent memory can irreversibly discard details before their relevance is clear. COEM delays compression with pending source excerpts and source verification, and improves answer F1 over the strongest memory baseline by 10.4–11.4 points on 6,400-document inputs with Qwen3.5-9B.

1 INTRODUCTION

  • Long-context tasks require integrating evidence distributed across distant passages, yet relevant evidence grows sparser among distractors as input length increases.Processing full texts also increases training and inference costs and may exceed a model’s context window.Source: §1 · p. 1
  • 32.03%–44.53% of randomly sampled questions across three multi-hop QA benchmarks exhibit delayed evidence relevance.The paper also reports that MemAgent occupies substantially more memory than GRU-Mem while achieving lower question-answering performance.Source: §1 · p. 2, Figure 2 · p. 2
  • 10.4–11.4 answer F1 points over the strongest memory baseline on 6,400-document inputs with Qwen3.5-9B.Across Qwen3.5-4B and Qwen3.5-9B, COEM consistently outperforms prior recurrent-memory agents across three benchmarks and tested context lengths.Source: §1 · p. 2
  • Immediate memory updates can irreversibly discard details that seem irrelevant at first but later become essential for reasoning.COEM’s Figure 1 contrasts its delayed decision on unclear evidence with two baselines.Source: §1 · pp. 2–3, Figure 1 · p. 2
  • COEM divides its fixed context-memory budget between compact committed facts and verbatim pending excerpts, which a policy can promote, keep, or drop as later context clarifies relevance.A frozen verifier checks whether proposed facts are supported by cited excerpts before commitment.Source: §1 · pp. 2–3

2 RELATED WORK

Prior memory approaches extend working context through external stores or bounded recurrent memory, but recurrent methods cannot recover source details discarded before their relevance becomes clear. COEM retains unresolved evidence and combines step-level evidence feedback with final-answer rewards.

Memory Mechanisms for LLMs

  • External-memory systems use stores beyond the active context, whereas COEM carries retained information within strictly bounded recurrent textual memory.Examples discussed include retrieval-augmented generation, MemoryBank, MemGPT, ReadAgent, and LightMem.Source: §2 · p. 3

Recurrent Memory for Long-Context Reasoning

  • Recurrent-memory methods improve memory reuse or efficiency, but cannot recover source details discarded before their relevance becomes clear; COEM instead retains unresolved evidence in a bounded pending set.MemAgent uses an RL-based overwrite policy, ReMemR1 adds callbacks over historical snapshots, and GRU-Mem uses update and exit gates.Source: §2 · p. 3

Reinforcement Learning for Memory Management

  • COEM combines final-answer rewards with step-level evidence rewards for source-supported proposals and valid operations, using a frozen verifier and later feedback to guide earlier decisions.The related methods discussed include ReMemR1, LongRLVR, and InfoMem.Source: §2 · p. 3

3 COEM: MEMORY CONTROL UNDER PARTIAL OBSERVABILITY

COEM treats long-context reading as sequential memory control under partial observability, retaining unresolved source excerpts alongside committed facts. Its policy revisits pending evidence, verifies proposed facts against source text, and learns memory decisions from answer and step-level evidence rewards.

3.1 PROBLEM SETTING AND CONTEXT MEMORY

  • COEM maintains a bounded state of committed facts and pending verbatim excerpts, the only information from earlier chunks carried forward to the next reading step.The policy observes the question, current chunk, and preceding state; a deterministic controller executes its actions with verification and budget checks.Source: §3.1 · pp. 3–4

3.2 UNRESOLVED EVIDENCE MANAGEMENT

  • At intermediate steps, pending excerpts can be promoted, kept, or dropped; when new evidence exceeds pending capacity, older items are resolved until it fits.At the final step, all remaining pending evidence must be promoted or dropped before the model answers from committed memory.Source: §3.2 · pp. 4–5

3.3 SOURCE-VERIFIED COMMITMENT AND ATOMIC MEMORY UPDATES

  • A frozen NLI verifier accepts a proposed fact only when its cited source spans remain available, match the original, and entail the fact.Memory replacement is atomic: all inserted facts must pass verification and the resulting memory must satisfy capacity constraints, or the transaction is rejected.Source: §3.3 · p. 5

3.4 STEP-LEVEL EVIDENCE REWARD FOR LEARNING DELAYED MEMORY DECISIONS

  • Training combines a trajectory-level answer reward with step-level rewards for invalid operations and verifier-rejected facts, carrying later verification feedback to earlier memory decisions.Group-centered advantages are applied through a clipped policy objective; accepted facts receive no per-write bonus, while final-answer rewards guide task-relevant fact selection.Source: §3.4 · pp. 5–6

4 EXPERIMENTS

Across the tested context lengths and model scales, COEM outperforms recurrent-memory baselines, with a larger advantage at longer inputs. Experiments also test out-of-distribution generalization, capacity allocation, delayed evidence, verification, efficiency, and component ablations.

4.1 EXPERIMENTAL SETUP

  • Evaluation uses three multi-hop QA benchmarks, two Qwen3.5 backbones, and eight context lengths from 50 to 6,400 documents.HotpotQA supplies training data and in-distribution evaluation; 2WikiMultiHopQA and MuSiQue are out-of-distribution evaluations without additional training. Memory agents share a 1,024-token budget; COEM allocates 256 tokens to pending evidence and 768 to committed memory.Source: §4.1 · p. 7

4.2 MAIN RESULTS

  • 10.4–11.4 F1 points: COEM with Qwen3.5-9B leads the strongest same-budget memory baseline at 6,400 documents.Across 50–6,400 documents, COEM’s answer F1 declines 1.0–2.1 points, versus 9.6–11.3 points for ReMemR1.Source: §4.2 · p. 8
  • At 1,600 documents, Qwen3.5-9B COEM beats general-purpose LLMs by 17.5 F1 points on HotpotQA and 21.0 on both 2WikiMultiHopQA and MuSiQue.Without additional training, it performs best at every input length on 2WikiMultiHopQA and MuSiQue; the same trend holds with Qwen3.5-4B.Source: §4.2 · p. 8

4.3 MORE ANALYSIS

  • In a 6,400-document stress test, forced drops per admitted pending record rise from 0.5% under ordinary ordering to 1.8% with a 128-chunk gap and 7.6% under source-dense stress.At 1,600 documents, pending-set occupancy stays within budget in the reported episode, with three capacity events and a terminal decrease to zero.Source: §4.3 · p. 8
  • At 800 documents, answer F1 peaks at 85.5 with 256 pending-set tokens; at a 128-chunk evidence gap, COEM reaches 74.5 F1, 25.3 points above the strongest baseline.The capacity result reflects a trade-off: smaller pending allocations limit retention, while larger ones leave less room for committed facts.Source: §4.3 · p. 9
  • Calibrated DeBERTaV3-small reaches 97.2% faithfulness at 4.3 ms per pair, versus Qwen3.5-9B at 97.5% and 295.0 ms per pair.At 6,400 documents, COEM takes 548 seconds per question—24–42% less than MemAgent and both ReMemR1 variants, but 15.6% more than GRU-Mem while gaining 8.8 F1 points.Source: §4.3 · p. 9

4.4 ABLATION ANALYSIS

  • At 800 documents, removing RL causes an 18.0/21.0 F1 drop, while removing step-level evidence rewards causes a 5.1/6.2-point drop.Disabling pending retention and verification loses 12.1/14.3 points; adding verification recovers 3.8/4.2 points, while removing verification from the pending variant reduces faithfulness by 10.5 percentage points. A fixed two-chunk delay loses 3.2/3.9 points.Source: §4.4 · p. 9

5 CONCLUSION

The conclusion presents COEM as a memory agent for delayed evidence relevance, retaining unresolved excerpts until later context clarifies their importance. It reports widening gains over budget-matched agents as context grows and identifies extension to code and scientific documents as a next step.

5 CONCLUSION

  • COEM addresses delayed evidence relevance by keeping unresolved excerpts verbatim in a bounded pending set until later context clarifies them, while a frozen NLI verifier admits only source-supported facts.The conclusion distinguishes whether evidence is supported from whether it is relevant.Source: §5 · p. 9
  • Across three benchmarks and two backbones, COEM consistently outperforms budget-matched memory agents, leading by 10.4–11.4 F1 points at 6,400 documents with Qwen3.5-9B.At a 128-chunk evidence gap with Qwen3.5-4B, the reported lead is 25.3 points.Source: §5 · p. 9
  • The conclusion suggests that long-context memory should decide not only what to remember, but also when to compress it.Extending this principle to code and scientific documents is described as a natural next step.Source: §5 · p. 9

AI USE STATEMENT

The authors report AI assistance across manuscript work, literature discovery, and research execution, and describe review and verification steps they undertook.

AI USE STATEMENT

  • AI tools assisted with manuscript revision, language polishing, and translation.The authors report reviewing the AI-assisted content.Source: AI USE STATEMENT · p. 10
  • AI tools supported retrieval and discovery, including literature searches and identification of related work.The authors report checking cited sources against original publications.Source: AI USE STATEMENT · p. 10
  • AI tools assisted with research ideation and execution, including method and experiment design, mathematical analysis, code preparation, and figure generation.The authors report verifying derivations, testing code, and cross-checking experimental results and figures; they take responsibility for the final manuscript and artifacts.Source: AI USE STATEMENT · p. 10

ETHICS STATEMENT

The ethics statement scopes the study to public-benchmark long-context question answering with published pretrained components and notes limits of passage-based verification and finite memory.

ETHICS STATEMENT

  • The study evaluates long-context question answering on HotpotQA, 2WikiMultiHopQA, and MuSiQue using published pretrained model components.The authors note that the system may inherit biases and errors from datasets, source passages, and models.Source: ETHICS STATEMENT · p. 10
  • The frozen verifier assesses support by supplied passages, which does not establish truth beyond those passages or guarantee a correct final answer.Verifier errors and evidence loss under finite memory remain possible.Source: ETHICS STATEMENT · p. 10
  • The stated evaluation scope is benchmark-based long-context question answering, rather than broader task coverage.Source: ETHICS STATEMENT · p. 10

REPRODUCIBILITY STATEMENT

The reproducibility statement points to the paper’s implementation materials and supplementary experimental documentation. The appendix covers dataset construction, implementation settings, verifier calibration, and additional evaluations.

REPRODUCIBILITY STATEMENT

  • The source code includes controller, training and evaluation scripts, data-construction utilities, verifier tools, dependency specifications, and unit tests.The authors state that trained model weights and evaluation manifests will be released upon acceptance.Source: REPRODUCIBILITY STATEMENT · p. 10

A EXPERIMENTAL DETAILS AND ADDITIONAL RESULTS

A.1 DATASET CONSTRUCTION AND EVALUATION

Evaluation uses filtered question cohorts and nested document-length contexts, with fixed question IDs enabling paired comparisons across lengths. The section also specifies chunking, input availability, and answer-scoring procedures.

A.1 DATASET CONSTRUCTION AND EVALUATION

  • 128 evaluation questions per official development split are sampled with seed 4 and reused across lengths; training uses 32,768 HotpotQA questions and a disjoint 512-question development set.Questions are eligible only when supporting text fits 4,096 tokens under both policy tokenizers and the question fits 256 tokens.Source: §A.1 · p. 14
  • 50–3,200-document contexts use stable ordering so shorter inputs are subsequences of longer ones, preserving supporting text and document order.Distractors come from training-split title–paragraph pairs, with duplicates and near-matches to supporting passages excluded.Source: §A.1 · p. 14
  • Documents are serialized into non-overlapping chunks of at most 5,000 tokens, while stable source-span coordinates support auditing across chunk boundaries.The manifest maps chunk-relative offsets to absolute document offsets but does not expose discarded history to the policy.Source: §A.1 · p. 14
  • 614,000–819,000 distractor-prefix tokens occur at 6,400 documents; inputs exceeding the full-context baseline’s 262,144-token window are unavailable, not truncated.Recurrent methods read all chunks under the same context-memory budget.Source: §A.1 · p. 14
  • F1 is 100 times mean token-overlap score, with invalid or empty predictions scoring zero and exact match reported separately.For yes, no, or noanswer, credit requires an exact normalized-string match; the highest score is used across acceptable references.Source: §A.1 · p. 14
  • Eight-length averages weight lengths equally, dataset macro-averages weight datasets equally, and confidence intervals use 10,000 paired-question bootstrap resamples.The full-context baseline is excluded from all-length averages when some lengths are unavailable.Source: §A.1 · p. 14

A.2 IMPLEMENTATION AND MATCHED COMPARISONS

This section describes matched memory comparisons, action and verification procedures, diagnostics of delayed evidence and memory pressure, and the protocol and scope for measuring end-to-end latency. It also reports a verifier audit and natural-order samples examining faithfulness, latency, and delayed relevance.

A.2 IMPLEMENTATION AND MATCHED COMPARISONS

  • The matched comparison fixes each input at 8,192 tokens and gives the memory component 1,024 tokens; ReMemR1’s budget-matched adaptation allocates 480 tokens each to current memory and one historical snapshot, plus 64 to its query.MemAgent receives all 1,024 retained tokens, while GRU-Mem keeps its learned update action with early exit disabled; replacing ReMemR1’s archived snapshot permanently discards the older one.Source: §A.2 · pp. 15–16

A.3 PROMPT TEMPLATES AND ACTION FORMAT

  • The controller selects verbatim source spans, then promotes, keeps, or drops excerpts; promotions must cite visible source evidence, pass verification, and fit memory atomically.Final resolution requires all pending records to be committed or dropped before answering; the final answer uses committed facts only.Source: §A.3 · pp. 17–19

A.4 VERIFIER CALIBRATION AND INDEPENDENT AUDIT

  • The frozen DeBERTaV3-small verifier uses complete cited source spans as premises, and the selected calibrated entailment threshold is η = 0.90.Threshold selection maximizes supported-claim recall while limiting false acceptance to 5% on the selection split.Source: §A.4 · p. 19
  • 4.3 ms/pair versus 295 ms/pair, the small and Qwen3.5-9B verifiers achieve 97.2% and 97.5% faithfulness, respectively; calibration shifts false acceptance from 9.2% to 4.9% and false rejection from 3.9% to 6.4%.The audit uses the same blinded set of 2,000 preverification promotion attempts for all verifiers.Source: §A.4 · p. 20, Table 6 · p. 20

A.5 EVIDENCE ORDER, CONTEXT-MEMORY OCCUPANCY, AND CAPACITY PRESSURE

  • Figure 6 tracks committed and pending memory during 6,400-document reading, with pending excerpts included in total retained context memory and resolved before the final answer.The plotted final reading-state sample precedes final resolution, which leaves the pending set empty.Source: Figure 6 · p. 21, §A.5 · p. 21
  • 32.03%–44.53% of sampled HotpotQA, 2WikiMultiHopQA, and MuSiQue questions contain evidence whose relevance becomes apparent only later.Each benchmark cohort contains 128 questions with 800 documents per question; these frequencies measure delayed relevance under natural document order.Source: §A.5 · p. 21
  • Under pending-set capacity pressure, reconsideration occurs only when an eligible source does not fit; the separate stress study reports no token-budget violations and notes verification cannot recover discarded evidence.The example trace shows three capacity events and final resolution, with pending occupancy at or below 256 tokens.Source: §A.5 · p. 21, Table 7 · p. 22

A.6 END-TO-END EFFICIENCY

  • The end-to-end timing study uses one H800 per recurrent inference stream, batch size one, bfloat16 weights, identical decoding limits, and disabled cross-request prefix caching.Both policy scales use tensor-parallel size one in a dedicated timing deployment to hold hardware constant.Source: §A.6 · p. 21
  • Table 8 reports Qwen3.5-4B end-to-end latency in seconds per question on one H800, including controller operations, verification, memory callbacks, and final-answer generation in full-scan timing.Its numerical values are plotted in Figure 5(f); the verifier comparison is shown in Figure 5(e), with calibration and audit details in Appendix A.4.Source: Table 8 · p. 22, §A.6 · p. 22
  • Latency is measured on the same 128 questions after ten warm-ups, from the first recurrent prompt through final-answer completion, with early stopping disabled.The timing includes serialization, policy calls, rejected proposals, verifier calls, capacity handling, final resolution, and callback or archive operations; token and call counts are reported alongside mean latency.Source: §A.6 · p. 22

A.7 TRAINING CURVES AND MEMORY DECISIONS

CoEM’s training diagnostics compare answer quality and memory decisions with Outcome-only RL, while later analyses examine supporting-evidence retention and reward variants. The results report higher answer F1, better evidence retention, and gains from evidence rewards, with uncertainty estimates conditioned on three policy seeds.

A.7 TRAINING CURVES AND MEMORY DECISIONS

  • At update 500, COEM had a 5.2% verifier-rejection rate and a 0.8% invalid-operation rate, versus 13.6% and 1.3% for Outcome-only RL.The rejection rate denominator is proposals passing non-entailment checks; invalid operations use all proposal attempts.Source: §A.7 · p. 22
  • At update 500, mean training F1 was 87.3 for COEM and 82.7 for Outcome-only RL; peak development F1 was 86.7 and 81.9, respectively.The development maxima occurred at updates 450 and 425, so the best checkpoint need not be the final update.Source: §A.7 · p. 22, Figure 7 · p. 23
  • At update 500, COEM made 2.55 eligible proposals per chunk and about 2.42 passed verification, versus 2.15 and 1.86 for Outcome-only RL.Mean evidence rewards were −0.0275 and −0.0599 per chunk, while voluntary DROP rates were 0.24 and 0.42; forced drops are excluded.Source: §A.7 · p. 23, Figure 8 · p. 24

A.8 VARIATION ACROSS POLICY SEEDS

  • Table 9 reports seed means and standard deviations alongside paired question-bootstrap intervals based on 10,000 resamples, conditional on the available three seeds.Each row represents one dataset and context length; repeated lengths are not treated as independent questions.Source: §A.8 · p. 23, Table 9 · p. 24

A.9 SUPPORTING-EVIDENCE RETENTION AND COMMITMENT TIMING

  • 86.2% coverage for COEM versus 74.8% for Outcome-only RL immediately before final resolution; at a 128-chunk gap, source survival was 89.8% versus 73.4%.Source survival and relation coverage measure different properties because promoted excerpts leave the pending set even when their relation remains in committed memory.Source: Figure 9 · p. 25, §A.9 · p. 25
  • At a 32-chunk gap, 81.3% of COEM records were committed at the bridge step or the following step, compared with 68.2% for Outcome-only RL.The comparison concerns initially admitted records containing annotated supporting evidence.Source: Figure 9 · p. 25, §A.9 · p. 25

A.10 REWARD ABLATIONS WITH MATCHED ANSWER WEIGHT

  • At 800 documents, COEM gained 4.5 F1 points on HotpotQA and 5.6 on 2WikiMultiHopQA over the matched no-evidence control.The return-to-go variant exceeded the immediate evidence-reward variant by 2.4 and 2.9 points, respectively; these are aggregate training comparisons.Source: Table 10 · p. 26, §A.10 · p. 26

B COMPUTATIONAL RESOURCES

The resource accounting distinguishes training costs from a single inference run and tracks serialized memory separately from execution memory. The case studies show how pending excerpts can preserve details until an alias clarifies relevance, while atomic verified commitments and bounded capacity shape what survives.

B COMPUTATIONAL RESOURCES

  • The 1,024-token context-memory budget counts serialized state carried between chunks, separately from execution memory and any external baseline archive.Training accounting covers full 500-update COEM runs across three seeds; inference accounting describes one Qwen3.5-4B run on one H800, not cohort averages.Source: B COMPUTATIONAL RESOURCES · pp. 26–27

C CASE STUDIES

  • The two HotpotQA training examples are excluded from evaluation and tuning, and their source quotations are shown verbatim.Both examples use Qwen3.5-4B with BP = 256, BM = 768, H = 8, K = 1, J = 2, and η = 0.90.Source: C CASE STUDIES · p. 28, §C.1 · p. 28

C.1 DELAYED COMMITMENT ACROSS AN ALIAS RELATION

  • The pending set preserves the biography’s nationality and spousal-relation sentences until the question’s alternate name is linked to the spouse.Keeping only the spouse’s name would lose the requested nationality; the relevant source text supports later relevance assessment and verification.Source: §C.1 · p. 28
  • Promoting the two biography claims as one atomic transaction checks their premises and resulting memory before consuming the source, subject to J = 2.The verifier checks the biography for its two factual claims and the current passage for the alias claim, rather than certifying the whole multi-hop answer at once.Source: §C.1 · p. 29
  • A memory that retains only the marriage relation lacks the nationality when the alias arrives, though a supported summary preserving both relations could answer this example.The case illustrates the consequence of losing the attribute, not that every immediate summary must fail.Source: §C.1 · p. 29

C.2 CAPACITY PRESSURE BEFORE A LINKING PASSAGE

The capacity-pressure case follows a biography containing the requested date until the controller removes it before a later passage links the person to the relevant film. It illustrates how bounded capacity interacts with admission and replacement decisions, rather than showing that fixed capacity alone inevitably causes the error.

C.2 CAPACITY PRESSURE BEFORE A LINKING PASSAGE

  • Admission order selects the oldest remaining pending record, not the record judged least relevant; under unresolved relevance, an older source may be more useful than a later one.The bounded reconsideration pass leaves both stores unchanged in the displayed case, so the controller proceeds to forced resolution.Source: §C.2 · p. 30
  • The biography’s date is removed at step 7 before NLI evaluation when the proposed replacement does not fit, leaving the later cast passage unable to supply it.The source stated the date, but the later passage resolved the entity connection without repeating it.Source: §C.2 · pp. 30–31, Figure 13 · p. 31
  • The case shows a failure contingent on specific retention and replacement choices, not that the fixed budget inevitably causes the error.Declining a less useful admission or deleting suitable committed entries could have preserved the supported date.Source: §C.2 · p. 31
  • After a source is dropped, proposals citing its old offsets fail the source-availability check before NLI scoring, so verification cannot recover that evidence.The passage also notes that guessing from parametric knowledge would not restore the discarded evidence.Source: §C.2 · p. 31

D FORMAL ANALYSIS OF MEMORY CONTROL UNDER PARTIAL

This section formalizes CoEM’s bounded-memory controller, its operational constraints, and conditional guarantees for verification, delayed commitment, and evidence-based learning. It also describes how token-level evidence and answer rewards shape memory decisions, including distinctions among rejection histories with identical answer scores.

D.1 THE POMDP AND ITS FINITE-MEMORY CONTROLLER

  • The finite-memory controller is Markov on full environment states, but its bounded visible memory need not preserve a sufficient statistic of observation history.Distinct document prefixes can yield the same memory after a detail relevant to the eventual answer is discarded.Source: §D.1 · p. 32

D.2 MEMORY BUDGETS, TERMINATION, AND SOURCE VERIFICATION

  • O(T) inference cost and O(B) context-memory storage are guaranteed under fixed execution limits, with every reading step terminating.A T-chunk episode requires O(T) policy and verifier calls, including reconsideration and verification overhead.Source: §D.2 · p. 33
  • Committed facts pass source-availability, source-matching, and NLI checks, while failed updates preserve existing entries and their source identifiers.A bound on unsupported accepted facts additionally requires a uniform conditional verifier-error assumption; freezing or calibrating a classifier alone does not establish it.Source: §D.2 · pp. 33–34

D.3 THE VALUE OF RETAINING UNRESOLVED EVIDENCE

  • Delayed commitment cannot increase conditional task risk before waiting costs, but is beneficial only when its full cost is smaller than the risk reduction.A sufficient retention condition ensures an excerpt remains available through a later step when pending-set capacity permits all intervening admissions to fit; the condition excludes final-step resolution.Source: §D.3 · pp. 34–35

D.4 STEP-LEVEL EVIDENCE REWARDS AND FINAL ANSWER REWARDS

  • Step-level verification can provide a positive source-support learning signal without requiring success on every other required fact, under the stated verifier-discrimination assumptions.Group centering preserves the signal’s direction for an unclipped score-function objective without KL regularization, but the analysis does not establish that result for the practical modified updates.Source: §D.4 · p. 36

E.1 NOTATION AND ACTION REPRESENTATION

  • The controller copies source excerpts from original chunk tokens, and provenance identifiers select available premises rather than retrieving historical source text.Earlier cited spans are usable only while the complete span remains in a pending excerpt.Source: §E.1 · p. 37

E.2 MEMORY DECISIONS AND ATOMIC MEMORY UPDATES

  • Atomic promotion updates committed memory only after checks succeed; failed transactions preserve memory, and capacity resolution removes pending items when promotion fails or attempts are exhausted.Promotion attempts are shared across ordinary calls and capacity or final-step resolution, with each proposed fact consuming an attempt for every cited source record.Source: §E.2 · pp. 37–38
  • After the final chunk, pending excerpts are resolved in admission order with KEEP disabled, and invalid actions during resolution drop the selected item.Promotions follow the same verification and transaction rules, and the controller finishes with no pending items before answering from the question and committed memory.Source: §E.2 · p. 38
  • Capacity checks serialize the full memory state, treat committed-entry deletion and insertion as one transaction, and enforce limits on calls, inputs, and outputs.Rejected transactions leave memory unchanged; when a pending item has no promotion attempts left, it is dropped, and additional calls use bounded observations without accumulating earlier transcripts.Source: §E.2 · p. 38

E.3 TOKEN-LEVEL OBJECTIVE AND CREDIT ASSIGNMENT

  • Advantages compare complete memory transitions rather than assigning separate causal scores to individual source excerpts.The token objective indexes policy-generated tokens across decisions, proposed facts, removals, and answers; bounded calls within a chunk share the same time index.Source: §E.3 · p. 38
  • The optimization batch fixes the reference and rollout policies, uses the current policy for probability ratios and KL, and does not differentiate through completed-rollout advantages.Source text and verifier feedback are fixed inputs during differentiation.Source: §E.3 · p. 38
  • Unsupported-promotion penalties extend to earlier admission and retention steps, while supported but irrelevant facts earn no extra evidence reward.The answer reward evaluates task utility, so adding supported writes alone cannot increase the evidence reward.Source: §E.3 · p. 38
  • When answer scores are identical, centered answer advantages vanish, but different rejection histories can still produce nonzero evidence advantages.If both components are constant, the combined advantage is zero, although KL regularization can still contribute to the update; answer generation has zero evidence return.Source: §E.3 · p. 38

Related papers

Loading 2609.36935v2…