PaperlayerView exact revision on arXiv

Source-linked AI summary

A Benchmark and Diagnostic Study of Epistemic Admission in Shared Agent Memory

Xiaoyang Li, Yiqi Wang, Chencheng Zhu, KE XU, Wencheng Yang, Zequn Sun, Pingan Song, Yiqun Duan, Taotao Cai

arXiv:2609.30813v1cs.AI

TL;DR

Shared agent memory can make copied beliefs look like independent evidence, while a false admission exposes later readers. CPB evaluates admission in a frozen benchmark and running multi-agent stores; its results show a trade-off between rejecting copied claims and preserving answer coverage, with source-type gating reducing false adoption.

  • Problem

    Shared memory can make copied or paraphrased beliefs appear to have independent support, and a false admission exposes later readers to the claim.

    Source: §1 · p. 1

  • Method

    CPB combines a frozen public-source benchmark with running teams whose shared store logs writes and retrievals and whose consumer answers from the store.

    Source: §1 · p. 2

  • Results

    Across four agent families, policies rejecting copies also reject many true claims, while answer-preserving policies admit nearly as many false claims as share-all; uncontested false beliefs are asserted in 0.97 to 0.99 of probes.

    Source: §1 · p. 2, §5.3 · p. 8

  • Takeaways & Limitations

    No non-oracle policy consistently rejects false claims across verbatim copies, paraphrases, and paraphrases declared authoritative.

    Source: §6 · p. 11

    The protective effect of a competing truth is measured only in concurrent-conflict scenarios and fades when three or more false beliefs appear beside one true belief.

    Source: §5.3 · p. 8

Abstract

from arXiv · show

Evaluating claim admission in shared agent memory is challenging because repeated claims may be mistaken for independent evidence. An agent may copy or paraphrase a retrieved belief, while admitting a false claim exposes subsequent agents to it. To study this problem, we introduce the Correlated Promotion Benchmark (CPB), which evaluates whether candidate claims should be admitted to shared memory.CPB-Static constructs a frozen test split from publicly annotated sources with fixed gold actions. CPB-Live runs multi-agent teams over a shared store, records all writes and retrievals, and tracks source lineage defined by each scenario. A separate consumer answers from the store alone. We evaluate eight admission policies across four agent families. Our results show that policies which deduplicate sources reject many true claims alongside false ones, whereas policies preserving answer coverage admit nearly as many false claims as unrestricted sharing. Gating on declared source type reduces false adoption to 0.06--0.09, compared with 0.22--0.47 for other answering policies. Once an uncontested false belief enters memory, the consumer asserts it in 0.97--0.99 of probes across all families. No non-oracle policy consistently rejects false claims across verbatim copies, paraphrases, and paraphrases declared authoritative. These findings reveal the limitations of admission policies without access to source lineage.

1 Introduction

Shared agent memory can turn copied or paraphrased beliefs into apparent agreement, while a false admission exposes later readers. CPB measures admission policies and downstream consequences in static and running-team settings.

1 Introduction

  • Existing benchmarks rarely include conflicting or unequally reliable evidence, and fixed corpora cannot measure feedback among admission, retrieval, and later writes.Shared memory also lets restated beliefs create apparent agreement without independent evidence, potentially reinforcing false beliefs and spurious majorities.Source: §1 · p. 1
  • CPB-Static scores against fixed gold actions, while CPB-Live runs teams over a shared store and measures exposure and adoption after admission.The Live setting logs writes and retrievals and fixes source lineage by scenario rather than inferring it from policy behavior.Source: §1 · p. 2
  • Across four agent families, only source-type gating keeps false adoption low while still answering; policies that reject copies also reject many true claims.Policies preserving answer coverage admit nearly as many false claims as unrestricted sharing, while judges that filter halve damage at best.Source: §1 · p. 2
  • The benchmark measures consumer adoption of false shared beliefs, competing-truth effects, and how rewording and arrival order affect admission.Its diagnosis tests admission mechanisms under rewording and authoritative source typing.Source: §1 · p. 2

2 Related Work

Prior work studies correlated-source reasoning, shared-memory governance, propagation, and related benchmarks, but does not execute admission in a running team while logging shared-memory access and measuring adoption.

Related Work

  • Copy-aware truth-discovery methods infer copying from statistical agreement, while GovMem and MemTX use synthetic or writer-declared dependencies; CPB instead fixes lineage in the scenario.Earlier methods and extensions include DEPEN and later models for update histories, complex copying, and correlated fusion (Dong et al., 2009a;b; 2010; Pochampally et al., 2014).Source: §2 · p. 2
  • Shared-memory systems address provenance, governance, or poisoning, but their approaches do not jointly model a shared tier and its readers or execute some specified harm experiments.GovMem specifies a harm experiment without running it, while CAMA names but does not mitigate the write-back loop (Qi et al., 2026; Lin et al., 2026).Source: §2 · p. 2
  • Memory benchmarks usually test one-writer retention and retrieval, while conflict benchmarks test contradiction resolution; none runs admission in a team with logged access and measured adoption.Related work also finds collective false-belief adoption, propagation under interaction, correction by majorities, and correlated errors in judge panels (Xu et al., 2026; Rahman et al., 2026; Papadopoulos et al., 2026; Becker et al., 2026; Kohli, 2026).Source: §2 · p. 3

3 Method

CPB formalizes shared-store admission, uses scenario-fixed source lineage and logged downstream behavior to measure dependence and exposure, and pairs a fixed-gold Static benchmark with a running Live benchmark.

3.1 Problem formulation

  • An admission policy maps a candidate, its evidence, and the current store to one of four actions; only promotion changes the store for later rounds.Candidates run in a fixed agent order, but later candidates can differ as agents retrieve different admitted beliefs; policies see neither claim truth nor future readers.Source: §3.1 · p. 3

3.2 Measurement requirements

  • CPB fixes source lineage independently of the tested policy, rather than relying on similarity or inferred copying, and Live logs retrievals and probes consumer answers.The log measures reach and exposure within the simulation, while the consumer probe measures adoption; agent-declared derivation can be incomplete.Source: §3.2 · p. 4

3.3 The instrument

  • CPB-Static scores public-source verdicts against fixed gold actions after lineage collapse, while CPB-Live measures logged reach and exposure and consumer adoption.Static simulates propagation over authored consumers; Live observes writes and retrievals, and truth grading is separate from provenance declarations.Source: §3.3 · p. 4

4 Experimental Setup

The experiments combine a frozen public-source benchmark with fictional multi-agent scenarios, compare admission policies across agent families, and report defined metrics with specified inferential limits.

4.1 CPB-Static

  • CPB-Static contains 1,200 six-agent episodes, including 600 training, 200 validation, and 400 test episodes; its frozen test split uses a gold rule requiring two independent sources.Episodes draw on public claim-verification and editing corpora and Wikipedia citation lineage groups; all scenario content is authored fiction.Source: §4.1 · p. 5

4.2 CPB-Live

  • CPB-Live tests four scenario families: correlated agreement, concurrent conflict, mixed-context feeds, and dual-source true or false roots with restatements or copies.Each backbone runs 100 configurations, with additional truth-and-form cross configurations on Qwen and the closed family; the consumer is probed at round six.Source: §4.2 · p. 5

4.3 Policies and backbones

  • The policy suite compares private-only and share-all endpoints with voting, lineage-collapse, confidence, source-type, judge, and trained-baseline approaches.Live rules use cited evidence; Static-only policies include confidence gating, unanimity, and trained baselines. Qwen3.5-9B is primary, with Gemma-3-12B, Llama-3.1-8B, and Claude Opus 5 also used.Source: §4.3 · pp. 5–6

4.4 Metrics and statistics

  • Static false-promotion rate is promotions among candidates with non-promote gold actions; Live adoption is false-value assertions, and correctness is true-value assertions in scenarios containing one.Static intervals use source-item cluster bootstraps, with six prespecified comparisons Holm-corrected; other analyses, including Live, are descriptive and untested.Source: §4.4 · p. 6

5 Results

Across four agent families, governance is the only tested policy that keeps damage low while still answering, but its source-type gate admits authoritative rewordings. Consumers almost always repeat an uncontested false memory, and no tested admission approach handles every copy form.

5.1 Admission, exposure and downstream utility

  • Only governance keeps damage low while still answering in every family; judges halve damage at best, with 0.06–0.09 adoption and 0.58–0.65 correctness.Governance correctness is 0.90 on concurrent-conflict and correlated-agreement scenarios but 0.00 on mixed scenarios.Source: §5.1 · pp. 6–7, Table 1 · p. 7

5.2 Two published components inside the instrument

  • Mem0 absorbs 0.67 of Gemma’s and 0.86 of Llama’s admissions, mostly folding false candidates into existing false beliefs; A-MemGuard passes falsehoods supported by their cited sources.A-MemGuard rejects 0.00, 0.01, 0.18 and 0.00 of false candidates across Qwen, Gemma, Llama and the closed family.Source: §5.2 · pp. 7–8

5.3 Adoption with and without a competing truth

  • Without a competing truth, consumers assert an uncontested false belief in 0.97–0.99 of probes; with one, false values are voiced in 0.16–0.60, but protection is measured only in concurrent-conflict scenarios.With three or more false beliefs beside one true belief, false-only adoption returns to 0.42.Source: §5.3 · p. 8

5.4 Refusal on each policy’s own candidate stream

  • Governance refuses 0.84–0.88 of false candidates and varies by 0.04 across families, versus majority vote’s 0.00–0.89 refusal range, which tracks available evidence.The independence class refuses 0.67–0.88 of false candidates.Source: §5.4 · p. 9

5.5 Sensitivity to rewording and source type

  • The independence class rejects verbatim copies but admits reworded copies at 0.65–0.72 adoption; governance adoption rises from 0.00 on web-typed copies to 0.70 when reworded copies are register-typed.False admissions summed across policies rise from 12–30 on verbatim copies to 38–54 on reworded and 43–70 on authoritatively typed copies.Source: §5.5 · p. 9

5.6 Independence against truth

  • Admission tracks form more than truth: the lineage oracle blocks verbatim and reworded copies regardless of truth, but admits independent claims even when false.Semantic collapse admits 0.10 of reworded false claims; the lineage oracle admits independent claims at 0.95 when false and 0.90 when true.Source: §5.6 · p. 10

5.7 The write-back loop

  • Turning off agent retrieval cuts closed-family candidates per configuration from 8.3 to 2.2 and declared dependencies from 189 to zero, while false writes remain 28 versus 27.Whether the write-back loop increases false admission under policies that count support remains untested.Source: §5.7 · p. 11

6 Conclusion

CPB fixes source lineage and logs admitted beliefs’ reach; the results show that agreement, consistency, and source authority can still pass falsehoods to consumers.

6 Conclusion

  • Agreement, consistency, and source authority each leave routes for falsehoods to reach consumers; admission follows second-source form more than claim truth.The stated limits are that parent independence is presumed from distinct citations, collapse is shown to refuse copies rather than admit corroboration, and competing-truth protection is measured in one family.Source: §6 · p. 11

Organisation of the appendix

  • Appendix materials cover construction and policy details, supplementary results, grader validation, judge materials, reproduction materials, and limitations after the main-text sequence.Source: Organisation of the appendix · p. 16

A Coverage

  • Coverage pairs the 400-episode frozen Static split with 100 Live configurations per family; confidence, unanimity, and trained baselines are Static-only, while Mem0 and A-MemGuard are Live-only.Source: A Coverage · p. 16, Table 2 · p. 16

B Corpus construction, audit and provenance ledger

  • Adapters map public verdicts and citation lineages into scenarios; an audit corrected two assembly defects in 120 frozen episodes before the partition was refrozen.Live configurations were committed before execution and released with the store logs used for scoring.Source: B Corpus construction, audit and… · pp. 16–17

C The policy view and the policies

  • Policies use evidence and visible beliefs rather than claim truth or authored lineage; independence-rule variants decide identically on open families but differ on the closed family.The Live view includes candidate text, cited sources, retrievable beliefs, and declared provenance; the rules promote fresh authoritative support without contradiction, abstain under unresolved contradiction, or defer.Source: C The policy view and the policies · p. 17

D Two published systems inside the instrument

  • Mem0 runs as the shared store, while A-MemGuard checks candidates against cited sources and visible beliefs before promoting, keeping private, or requesting evidence.The comparison reports refusal, false-memory hold, false-answer adoption, true-answer correctness, and Mem0’s proportion of admissions merged into existing memories.Source: D Two published systems inside the… · p. 18, Table 4 · p. 18

E Downstream adoption per policy and per family

The consumer is evaluated on answers from shared memory, while adoption and refusal analyses reveal robustness to answer matching and the limits of comparisons on policy-specific streams.

E Downstream adoption per policy and per family

  • The consumer answers a targeted final-read question from shared memory alone; adoption measures false assertions and correctness measures true assertions when a true value exists.Source: E Downstream adoption per policy and… · p. 19, Table 5 · p. 19
  • Three hundredths: robustness matching moves adoption by no more than this and correctness by at most five hundredths, while leaving policy ordering unchanged.The matcher reaches agreement 0.975 with the first author and recall 1.000 on the false label in the human sample.Source: E Downstream adoption per policy and… · p. 19

F Adoption as a function of what shared memory held

  • A competing true belief is strongly protective against false answers, but that protection weakens as false beliefs accumulate.Source: Table 7 · p. 20

G Refusal conditioned on a false candidate arriving

H Per-class breakdown of the Live results

The Live breakdowns organize writes and damage by scenario class, and the truth-against-form design compares admission across different second-source relationships without supplied cell outcomes.

H Per-class breakdown of the Live results

I Truth against form

  • Table 13 crosses claim truth with verbatim, same-lineage reworded, or independent second-source form; pooled denominators differ because only two families ran stage 6.Each cell is the share of that class’s episodes in which the target proposition was written; the table caption says lower false and higher true shares are better.Source: I Truth against form · pp. 24–25, Table 13 · p. 25

J The write-back ablation

The write-back ablation compares share-all with and without agent retrieval while the policy still sees the store and the consumer still reads it. The accompanying diagnostics report no correction of admitted false beliefs and show that declared-dependency measures vary with agents’ reporting.

J The write-back ablation

  • The ablation compares share-all with agent retrieval on versus off, while the policy still sees the store and the consumer still reads it.Table 14 defines damage as consumer reads touching false beliefs and adoption as the false answer rate at the probe.Source: J The write-back ablation · p. 26, Table 14 · p. 26

K Correction, arrival order and declared provenance

  • Declared-dependency amplification is a reporting-shaped lower bound: Qwen declared none across 78 candidates, while false writes with later declared dependencies ranged from 0.01 to 0.82 across families.Agents were instructed to declare any shared belief their claim relied on.Source: K Correction, arrival order and… · p. 26

L How a live candidate is judged false

The paper distinguishes sentence-level assertion grading from provenance tracing and validates its assertion and consumer-answer graders against human labels. The comparisons reveal both limits in the graders and substantial differences between assertion and provenance scores.

L How a live candidate is judged false

  • The provenance trace can miss uncited restatements and count hedges citing false sources, while content matching can misclassify negations or mentions within other propositions.The reported results use assertion grading; provenance-based scores are archived.Source: L How a live candidate is judged false · pp. 26–27
  • The assertion grader labels whether each candidate asserts the false proposition, the true proposition, or neither; reported false-write and damage results use that grade.Adoption retains a separate grading rule for short consumer answers.Source: L How a live candidate is judged false · p. 27

M Human validation of the two graders

L How a live candidate is judged false

  • The assertion grader’s false-label precision was 0.958 and recall 0.852; its sampled errors mean reported false counts lean low rather than high.The passage cautions that this sample indicates error direction, not its rate over the full set.Source: L How a live candidate is judged false · p. 27

M Human validation of the two graders

  • For consumer answers, the rule’s false-label precision was 1.000 and recall 0.706; number and article normalization recovered every sampled rephrased-value miss.The reported adoption and correctness measures therefore undercount when consumers rephrase a value.Source: M Human validation of the two graders · p. 28

N Assertion and provenance measure different properties

  • On Llama, provenance tracing misses 1,524 false assertions; on the closed family, it counts 839 candidates asserting nothing false alongside 405 it gets right.Dropping the closed-family false positives lowers share-all damage from 275 to 214.Source: N Assertion and provenance measure… · p. 29, Table 17 · p. 29
  • Assertion grading evaluates whether a sentence asserts a proposition, while provenance traces where a claim came from; repetition may omit sources, and citations need not endorse claims.The table compares false-candidate counts and damage under the historical and assertion-based rules.Source: N Assertion and provenance measure… · p. 29, Table 17 · p. 29

O The judges on both halves

The judge comparisons show that performance varies by judge and agent family, and that judges do not match the explicit rule’s high-stakes abstention. The section also contrasts error directions on annotation judgments and admission decisions.

O The judges on both halves

  • The Llama judge and frontier judge over Qwen halve damage, reaching 72 against 145 and 78 against 147; the frontier judge over its own agents records 94 against 115.The first comparisons are on 60 base configurations; over all 100, the frontier judge’s damage is under half the ceiling.Source: O The judges on both halves · p. 30, Table 18 · p. 30
  • No judge matches the explicit rule’s high-stakes abstention, and the local judge was not run over frontier agents, leaving judge and agent effects partly entangled.Table 18 covers five judge-agent combinations on the 60 base configurations.Source: O The judges on both halves · p. 30, Table 18 · p. 30

P Over-assertion of the judgment layer

  • Error directions agree across models on annotation but differ on admission; human-against-human κ is 0.77 across all 100 pairs.The table distinguishes false same-judgments from false different-judgments on annotation and false promotions from false withholdings on Static.Source: P Over-assertion of the judgment layer · p. 31, Table 20 · p. 31

Q The pre-registered comparisons

The pre-registered comparisons find that the judge can over-promote relative to majority vote, while the governance rule performs better on high-stakes abstention. The section also reports accuracy comparisons for review and learned baselines and identifies the released reproduction materials.

Q The pre-registered comparisons

  • On the majority-wrong subset, the judge’s false-promotion rate exceeds the vote’s by 0.228; its false-promotion-to-false-withholding ratio is 1.60.The intervals are 0.169 to 0.288 for the difference and 1.02 to 2.52 for the ratio; the six-test Holm-corrected p-value is < 0.001.Source: Q The pre-registered comparisons · pp. 31–32
  • The governance rule beats the judge on high-stakes abstention recall by 0.10, while review all correlated beats both collapse rules on accuracy by 0.11.Review all correlated requests evidence less often.Source: Q The pre-registered comparisons · p. 32
  • A logistic regression and gradient-boosted tree reach 0.645 and 0.653 accuracy against the best rule’s 0.368, but both require a whole episode’s evidence graph and run only on Static.The reported gap is about 0.28, with intervals excluding zero.Source: Q The pre-registered comparisons · p. 32
  • The Qwen results are confirmatory with Holm adjustment across six tests; results on three other judge backbones are descriptive replications.Table 21 covers the three pre-registered judge comparisons.Source: Q The pre-registered comparisons · p. 32, Table 21 · p. 32

R Reproduction materials

S Limitations

The study’s Live conclusions are bounded by authored scenarios and mechanism-level evaluation; several assumptions, effects, and correction behaviors remain untested or narrowly measured.

S Limitations

  • Authored-fiction Live scenarios reveal mechanism behavior on this instrument, not how often these failures arise in deployment.Fiction allows lineage to be fixed and truth to be known.Source: S Limitations · p. 34
  • Parent independence in the correlated-agreement family is presumed from distinct citations rather than independently established.Source: S Limitations · p. 34
  • Dual-source sets show that collapse refuses copies, not that it admits genuine corroboration.Source: S Limitations · p. 34
  • The protective effect of a competing truth is measured only in the concurrent-conflict family.Source: S Limitations · p. 34
  • Although the store offers contest, demote, and supersede, no compared policy uses them, leaving correction untested.Source: S Limitations · p. 34
  • Assertion graders and judges are language models, validated against two human annotators on 120 items each; Live measurements are descriptive, without hypothesis tests.Source: S Limitations · p. 34
  • Beyond Mem0 and A-MemGuard, comparisons cover reimplemented mechanism classes rather than additional named systems.Source: S Limitations · p. 34
Loading 2609.30813v1…