PaperlayerView exact revision on arXiv

Source-linked AI summary

AECG: Asymmetric Experience Consolidation and Governance In Multi-Agent Systems

Ao Tian, Jialong Liu, Daqi Zheng, Xin Sun, Mengting Li, Zhizhao Xiao, Zijian Huang, Honglei Wang, Zijian Hei, Yukun Yan

arXiv:2610.05176v1cs.AI

TL;DR

Repeated reuse can make unreliable procedures persistent, while multi-agent memory research has comparatively underexplored reliability decay and cross-level reuse. AECG governs memory through scope preservation, confidence-aware reliability, exposure-prioritized review, and replay-gated updates. Across three frameworks and four benchmarks, it achieves the best score in 11 of 12 settings.

  • Problem

    Memory methods emphasize forming and recalling reusable knowledge, while reliability decay and safe reuse across collaboration levels remain comparatively underexplored.

    Source: §1 · p. 1

  • Method

    AECG preserves coordination scope, admits evidence asymmetrically, tracks reliability at two timescales, prioritizes review by downstream exposure, and gates revised skills through paired replay.

    Source: §1 · p. 2

  • Results

    AECG achieves the best score in 11 of 12 settings across three frameworks and four benchmarks, outperforming the strongest memory baseline by up to 10.23 percentage points.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The results support lifecycle governance of multi-agent memory, including reuse boundaries, reliability, and intervention, rather than passive accumulation alone.

    Source: §5 · p. 9

    Failure attribution is conservative evidence admission, not ground-truth causal identification.

    Source: §3.2.2 · p. 5

Abstract

from arXiv · show

Large language model (LLM)-based multi-agent systems increasingly rely on memory to transform execution trajectories into reusable procedural knowledge. Yet repeated retrieval also makes memory errors persistent: memory pollution arises when outdated, weakly supported, or spuriously successful procedures become recurring components of future reasoning. Multi-agent execution introduces an additional structural risk. Scope collapse occurs when procedural knowledge escapes the coordination scope in which it was shown effective and is repeatedly reused at incompatible decision levels, allowing local errors to influence cascades of downstream decisions. Meanwhile, task-level failures provide ambiguous supervision because they rarely reveal which recalled knowledge was responsible. We introduce AECG, a framework for asymmetric experience consolidation and governance for multi-agent systems. AECG turns memory from static experience storage into a dynamic reliability-governance loop, preserving coordination scope and using multi-scale, confidence-aware reliability to detect degradation. It then combines degradation with downstream impact to prioritize high-risk knowledge under a bounded review budget, applies targeted interventions, and reactivates revised skills only after paired replay. Across three multi-agent frameworks and four benchmarks, AECG achieves the best score in 11 of 12 framework--benchmark settings and improves over the strongest competing memory method by as much as 10.23 percentage points; removing scope preservation reduces accuracy by up to 16.89 points. AECG thereby reframes multi-agent memory from passive accumulation into auditable reliability governance. Code is available at https://github.com/fenhg297/AECG

1 INTRODUCTION

AECG addresses reliability risks in reused multi-agent procedural memory by preserving coordination scope and governing degradation and intervention. It outperforms competing memory methods across most evaluated settings.

Motivation

  • Prior memory research has advanced knowledge formation and retrieval, while reliability decay and safe generalization across collaboration levels remain comparatively underexplored.Correct retrieval alone does not establish that reused knowledge remains reliable or generalizes safely.Source: §1 · p. 1
  • Repeated retrieval can make outdated, weakly supported, or spuriously successful procedures persistent errors that contribute to negative transfer across tasks.Multi-agent decisions span interdependent planning, coordination, and execution levels, creating a risk when local procedures escape their supported scope.Source: §1 · pp. 1–2

Evaluation

  • AECG achieves the best score in 11 of 12 settings across three frameworks and four benchmarks, outperforming the strongest memory baseline by up to 10.23 percentage points.Removing scope preservation reduces accuracy by up to 16.89 points.Source: §1 · p. 2

Contributions

  • AECG preserves coordination boundaries, admits negative evidence conservatively, and uses dual-timescale reliability and topology-aware governance to guide skill updates.Revised candidates are activated only after replay mitigates implicated failures while preserving prior supported behavior.Source: §1 · p. 2

2 RELATED WORK

Prior work progresses from retrieving interaction histories to forming reusable skills and organizing multi-agent memory. AECG complements these approaches by focusing on the reliability and propagation of reused procedures.

Memory and experience reuse in LLM agents

  • Agent memory has progressed from retrieving histories and reflections to distilling reusable workflows, making generated abstractions persistent inputs to later reasoning.Generative Agents, Reflexion, ExpeL, and later systems represent successive forms of experience abstraction and reuse.Source: §2 · pp. 2–3

Memory in multi-agent systems

  • Multi-agent memory research studies where experience resides, which agents consume it, and how collaborative traces are represented, but primarily emphasizes construction and retrieval.The cited work includes hierarchical graphs, separate orchestration and subtask memories, and local, shared, or hybrid topologies.Source: §2 · p. 3

Procedural consolidation and skill evolution

  • Procedural-memory methods refine skills and retain evidence or reliability information, while repeated LLM-based consolidation can still discard evidence or introduce spurious rules.AECG complements this work by combining scope preservation, conservative failure attribution, two-timescale reliability, and downstream exposure.Source: §2 · p. 3

3 METHOD

AECG governs procedural memory through scope-constrained evidence formation, dual-timescale reliability assessment, and risk-aware review. Proposed skill changes are routed by evidence and activated only after replay checks.

3.1 PROBLEM FORMULATION AND METHOD OVERVIEW

  • AECG frames persistent knowledge as a trade-off between propagating unreliable skills and withholding useful guidance, addressed through a closed governance loop.The loop combines confidence-weighted evidence, historical and recent reliability beliefs, exposure-aware review, targeted interventions, and paired replay.Source: §3.1 · p. 3, Figure 2 · p. 4

3.2.1 SCOPE-PRESERVING MEMORY OBJECTS

  • Scope-tagged memories and skills retain their coordination level, and evidence is not pooled across team-, event-, and agent-level objects.Promotion preserves the seed memory's scope, constraining recall and comparison.Source: §3.2 · p. 3, §3.2.1 · p. 4

3.2.2 ASYMMETRIC EVIDENCE ADMISSION

  • Successful trajectories provide confidence-weighted positive evidence, whereas failures count against a skill only when qualified attribution identifies it; otherwise outcomes remain unresolved.Unresolved failed trajectories remain available as BadCases, and attribution is conservative evidence admission rather than ground-truth causal identification.Source: §3.2.2 · pp. 4–5

3.2.3 SKILL FORMATION AND EVIDENCE INHERITANCE

  • Repeatedly supported same-scope memories can seed reusable skills, with only the seed supplying initial evidence and nearby memories clarifying applicability.A fraction of seed evidence is inherited; the slow state retains discounted evidence while the fast state caps inherited mass at window W.Source: §3.2.3 · p. 5

3.3 DUAL-TIMESCALE RELIABILITY ESTIMATION

  • Each active skill version maintains slow cumulative and fast discounted Beta-style beliefs to distinguish historical support from recent effectiveness.These distributions are used for scoring, not as posteriors under a full generative Bayesian model; their comparison helps identify recent decline relative to a skill's history.Source: §3.3 · pp. 5–6

3.4 RISK- AND EXPOSURE-AWARE MEMORY GOVERNANCE

  • AECG combines reliability degradation with structural exposure because degradation alone does not determine how limited review capacity should be allocated.Exposure measures potential downstream reach and is applied only after reliability evidence qualifies a skill for review.Source: §3.4 · p. 6, §3.4.1 · p. 6

3.4.1 EXPOSURE-AWARE BUDGETED TRIAGE

  • Exposure-aware triage ranks reliability-qualified skills by potential propagation exposure for review under a bounded budget.Exposure discounts remote descendants and is not a causal effect estimate.Source: §3.4.1 · p. 6

3.4.2 EVIDENCE-ROUTED AND REPLAY-GATED INTERVENTION

  • Qualified failures route reviewed skills to context narrowing or procedural repair, and versioned candidates activate only after paired replay verifies mitigation and regression tolerance.Failure fixtures test the implicated behavior; disjoint historical same-scope success fixtures test preservation of supported behavior.Source: §3.4.2 · p. 6

4 EXPERIMENTS

The experiments test whether AECG improves performance across frameworks and domains, whether its governance components are necessary, and whether its reliability and triage mechanisms work as intended. The evaluation spans three multi-agent frameworks and four benchmarks, with mechanism tests under matched conditions.

4.1 EXPERIMENTAL SETUP

  • The evaluation covers three frameworks and four benchmarks, with shared tasks, prompts, tools, execution limits, and evaluators across methods.Results average three independently seeded runs; mechanism tests use AgentNet–ScienceWorld with a shared post-warm-up memory snapshot and matched conditions.Source: §4.1 · p. 7

4.2 RQ1: DOES AECG IMPROVE END-TASK PERFORMANCE?

  • 11 of 12 framework–benchmark settings have the best score with AECG, which also ranks first in average performance for all three frameworks.The sole exception is MacNet–PDDL, where ConMem exceeds AECG by 2.56 points.Source: §4.2 · pp. 7–8
  • The framework-wise margins over the strongest memory method are 5.92 on AutoGen, 4.13 on MacNet, and 5.64 on AgentNet.AECG also improves over No-Memory by 6.36 points on AutoGen, 10.75 on MacNet, and 11.29 on AgentNet in average score.Source: §4.2 · p. 7

4.3 RQ2: ARE THE GOVERNANCE COMPONENTS NECESSARY, AND DOES SCOPE PRESERVATION PREVENT SCOPE COLLAPSE?

  • 16.89 points on ScienceWorld and 8.34 on MBPP-Plus are lost when scope preservation is removed, the largest component-ablation degradation.Removing the fast belief drops performance by 4.76 on ScienceWorld and 6.26 on MBPP-Plus; removing exposure from review priority also degrades both domains.Source: §4.3 · p. 8, Table 2 · p. 8
  • 36.5% of selected uses cross supported scope when scope is flattened, alongside success of 51.29 and exposure of 14.8, versus 68.18 success and zero cross-scope use with hard scope.The Flat-scope variant falls below the No-Memory ScienceWorld baseline of 55.68.Source: Table 3 · p. 8, §4.3 · p. 8

4.4 RQ3: CAN DUAL-TIMESCALE RELIABILITY OVERCOME HISTORICAL INERTIA?

  • 3.8 uses on average are needed for AECG to review a corrupted skill, versus 9.7 for the cumulative-only variant, reducing detection delay by 60.8%.Earlier review reduces polluted-skill failed uses from 17.9 to 6.4 and raises post-corruption success from 58.2% to 64.5%; No-Governance reaches 49.3% success and 31.2 failed uses.Source: Figure 3 · p. 8, §4.4 · p. 9

4.5 RQ4: DOES DOWNSTREAM EXPOSURE IMPROVE TRIAGE UNDER A FIXED BUDGET?

  • 81.5% of polluted-skill predicted exposure is covered in the first two reviews by exposure-aware ranking, versus 53.2% for risk-only and 49.8% for random review.Under the same confidence gate and review budget, exposure-aware triage reduces failed-task exposure to 22.4, versus 48.6 for risk-only and 61.2 for random review.Source: Figure 4 · p. 9, §4.5 · p. 9

5 CONCLUSION

AECG governs procedural memory across reuse boundaries, reliability, and potential impact. Its evaluations favor AECG in most settings, while its exposure measure and localized feedback retain stated limits.

5 CONCLUSION

  • AECG achieves the best performance in 11 of 12 settings, while mechanism studies report benefits from scope preservation, dual-timescale beliefs, and downstream-exposure review allocation.The conclusion notes that graph-based exposure is a structural proxy, not a true causal estimate.Source: §5 · p. 9
  • AECG combines scope-preserving reuse, asymmetric evidence and dual-timescale reliability monitoring, exposure-based review prioritization, and paired-replay-gated skill updates.Source: §5 · p. 9
  • AECG depends on calibrated localized feedback, and its graph-based exposure measure is a structural proxy rather than a true causal estimate.The authors identify joint modeling of responsibility and reliability, and intervention-grounded causal influence, as future work.Source: §5 · p. 9

AI USE STATEMENT

The authors disclose AI assistance in translation and polishing, software implementation, and preparation of selected figures. They state that they reviewed the text and checked the code and figures.

AI USE STATEMENT

  • Generative AI tools assisted with English translation and language polishing, software implementation, and preparation of selected figures.Source: AI USE STATEMENT · p. 10
  • The authors reviewed and revised AI-assisted text, inspected and tested AI-assisted code, and checked AI-assisted figures against the methods and results.Source: AI USE STATEMENT · p. 10
  • The authors take full responsibility for the final content, including text, code, and figures produced with AI assistance.Source: AI USE STATEMENT · p. 10

APPENDICES

The appendices collect implementation details, experimental materials, and supplementary analyses. They also present an algorithm spanning scope-preserving execution, evidence consolidation, exposure-aware review, and replay-gated governance.

APPENDICES

  • The supplementary material lists additional main-result analysis, mechanism-test protocols, prompt templates, and method and experimental details.Source: APPENDICES · p. 13
  • The method appendices cover skill abstraction, conservative failure attribution, context narrowing, procedural repair, replay contracts, implementation, and hyperparameter selection.Source: APPENDICES · p. 13

A ALGORITHM

  • Algorithm 1 executes with scope-matched recall, retains failed trajectories as BadCases, and updates memory from admitted scope-local evidence.Source: A ALGORITHM · p. 14
  • The algorithm updates skill beliefs and promotes eligible skills while linking equivalents without duplicate evidence transfer.Source: A ALGORITHM · p. 14
  • Periodic or budgeted reviews propose narrowing, repair, or no change, then evaluate feasible candidates through a replay gate before updating skill state.Source: A ALGORITHM · p. 14

B METHOD IMPLEMENTATION DETAILS

AECG assigns evidence conservatively, estimates reliability drift and exposure, and uses bounded review and replay to govern skill changes. Its hyperparameters are selected off the evaluation benchmarks, and failed-task attribution requires agreement between two judges.

B.1 EVIDENCE ASSIGNMENT AND SKILL PROMOTION

  • Task-level success credit is divided across distinct provenance families, unresolved failures add no negative evidence, and skill promotion depends on evidence from the same scope.Equivalent skills are linked without migrating the same evidence again.Source: §B.1 · pp. 14–15

B.2 BETA-STYLE BELIEF DRIFT AND UNCERTAINTY

  • The drift score estimates uncertainty-adjusted clipped degradation from paired Beta-style belief samples; it is a ranking score, not a calibrated test statistic.The score is bounded by belief-mean degradation and attenuates it when samples do not consistently support degradation.Source: §B.2 · pp. 15–16

B.3 TRIAGE AND ACTION-SPECIFIC VALIDATION

  • Exposure discounts distant downstream decisions and deduplicates converging paths; fixed-budget review selects eligible high-severity skills, while targeted narrowing or repair must pass paired replay.The exposure measure bounds potential propagation but does not claim every reachable node was causally changed.Source: §B.3 · pp. 16–17

B.4 HYPERPARAMETER SELECTION

  • Reliability and promotion parameters use an independent AgentBoard development hold-out, while governance values stay fixed and review budget is held equal across methods.The authors do not tune hyperparameters on evaluation benchmarks.Source: §B.4 · p. 17

B.5 LLM-BASED FAILURE ATTRIBUTION

  • Two LLM judges inspect each BadCase, and negative evidence is admitted only when both identify the same applied skill and scope without abstaining and their mean confidence reaches 0.70.Otherwise, the observation remains unresolved and contributes no negative evidence.Source: §B.5 · pp. 17–18

C.1 BENCHMARKS, BASELINES, FRAMEWORKS, AND MODELS

The evaluation spans four benchmarks, five memory baselines, and three multi-agent frameworks under shared controls. Figure 5 presents performance across these dimensions, with its outlined AECG row corresponding to Table 1.

C.1 BENCHMARKS, BASELINES, FRAMEWORKS, AND MODELS

  • PDDL, FEVER, ScienceWorld, and MBPP-Plus cover planning, fact verification, scientific reasoning and action, and Python synthesis, respectively.Their reported metrics include task success rate for PDDL, classification accuracy for FEVER, and augmented hidden tests for MBPP-Plus.Source: §C.1 · p. 18
  • The comparison includes No-Memory, MacNet-M, MemoryBank, ConMem, and G-Memory, which represent distinct persistent-memory approaches.No-Memory removes cross-task persistent memory while retaining native within-task context.Source: §C.1 · p. 18
  • AutoGen, MacNet, and AgentNet represent conversation-centric, directed-acyclic-graph, and decentralized-routing frameworks, with methods sharing tasks and execution controls.Agent execution uses gpt-4.1-mini; memory-side processes use gpt-4o-mini, and results average three independently seeded runs.Source: §C.1 · p. 18
  • Figure 5 displays performance across frameworks, benchmarks, and memory methods, and its outlined AECG row corresponds to Table 1.The caption identifies the figure’s scope but does not state a performance winner.Source: Figure 5 · p. 19

C.2 ADDITIONAL MAIN-RESULT ANALYSIS

AECG’s benchmark-wise gains are concentrated on ScienceWorld and MBPP-Plus, while PDDL remains more competitive. The pattern spans frameworks rather than relying on a single framework or task family.

C.2 ADDITIONAL MAIN-RESULT ANALYSIS

  • Under MacNet, AECG raises ScienceWorld accuracy from 47.73 for No-Memory to 71.59 and MBPP-Plus from 75.41 to 79.51.These are the largest improvements highlighted across the benchmarks.Source: §C.2 · p. 19
  • Under AgentNet, AECG raises ScienceWorld accuracy from 55.68 to 68.18 and MBPP-Plus from 68.85 to 77.87.The passage identifies these tasks as repeatedly reusing intermediate procedures and being especially sensitive to stale or over-generalized skills.Source: §C.2 · p. 19
  • On PDDL, ConMem slightly outperforms AECG on MacNet, while AECG remains best on AutoGen and AgentNet.Thus the overall advantage is not attributed to just one framework or task family.Source: §C.2 · p. 19

C.3 MECHANISM-TEST PROTOCOLS

The mechanism-test protocols control task streams and isolate corruptions, scope eligibility, and review-budget choices. The prompt templates specify how skills are formed, attributed, revised, and gated for reactivation.

C.3 MECHANISM-TEST PROTOCOLS

  • AgentNet–ScienceWorld tests use a shared post-warm-up snapshot; controlled corruption validates altered skills, while scope and triage protocols define eligibility and review comparisons.The task stream is split 60% for memory warm-up and 40% for evaluation; triage permits at most one review per round, with more qualified candidates than budget in the first two rounds.Source: §C.3 · p. 19

D PROMPT TEMPLATES

  • The prompt templates retain constraints governing skill formation, failure attribution, evidence-routed intervention, and replay-gated revision.Formatting instructions, examples, and implementation-specific boilerplate are omitted.Source: D PROMPT TEMPLATES · p. 20

SKILL ABSTRACTION PROMPT

  • Skill abstraction uses the seed as primary evidence, treats auxiliary memories as supplementary, and preserves coordination scope and target granularity.The resulting skill removes task-specific details and specifies applicability, triggers, procedure, avoidance, and evidence summary.Source: SKILL ABSTRACTION PROMPT · p. 20

CONSERVATIVE FAILURE ATTRIBUTION PROMPT

  • Failure attribution links a skill to failure only through a trace-supported clause and observed behavior, tests alternatives, and admits negative evidence only after judge agreement.Judges may abstain when evidence is incomplete, candidates are indistinguishable, or a stronger non-skill explanation remains.Source: CONSERVATIVE FAILURE ATTRIBUTION PROMPT · pp. 20–21

CONTEXT NARROWING PROMPT

  • Context narrowing proposes a minimal, pre-execution predicate supported by bad cases, then revises only applicability and trigger conditions while keeping the candidate inactive pending replay.The prompt requires abstention if a safe predicate cannot represent the failure region or would over-exclude valid uses.Source: CONTEXT NARROWING PROMPT · p. 21

PROCEDURAL REPAIR PROMPT

  • Procedural repair changes only evidence-supported procedure or avoidance defects, preserves useful steps and scope, and leaves the revision as a candidate until replay succeeds.The prompt disallows changing applicability or trigger conditions and hard-coding one bad case.Source: PROCEDURAL REPAIR PROMPT · pp. 21–22

REPLAY CONTRACT PROMPT

  • Replay activates a candidate only if it mitigates implicated behavior on failure fixtures and preserves supported behavior on disjoint historical-success fixtures within tolerance.Otherwise, the parent remains active or the candidate is isolated or monitored under the governance policy.Source: REPLAY CONTRACT PROMPT · p. 22
Loading 2610.05176v1…