PaperlayerView exact revision on arXiv

Source-linked AI summary

Remembering Is Not Enough: Refining Memories and Learning When to Use Them in Multi-Agent Systems

Yipeng Hao, Yuxuan Cai, Wenkai Ji, Jie Zhou, Liyang Yu, Qin Chen, Zhiguo Yan, Shulei Zheng, Liang He

arXiv:2610.36652v1cs.AIcs.MA

TL;DR

Multi-agent systems need to distinguish flawed memory rules from useful rules applied in the wrong context, since team outcomes alone cannot reliably decide whether to repair a memory. DISCERNMEM combines replay-validated rule refinement with role-specific applicability learning; across three architectures it reports the highest average scores and consistent strengths in code generation and planning.

  • Problem

    Team-level success or failure cannot reliably show whether a memory’s behavioral rule is defective or whether the rule was applied in the wrong context.

    Source: §1 · p. 1

  • Method

    DISCERNMEM uses paired replays to diagnose and validate rule repairs, while per-memory meta-memory learns role-specific use and avoidance conditions from reuse history.

    Source: §1 · p. 2, §3.2 · p. 4, §3.3 · p. 6

  • Results

    Across four benchmarks and three hosts, DISCERNMEM achieves the highest average score for each host and leads in eight of twelve host–benchmark settings, with consistent strengths in code generation and symbolic planning.

    Source: §4.2 · p. 6, §5 · p. 9

  • Takeaways & Limitations

    The findings support treating memory-content refinement and applicability learning as complementary aspects of memory adaptation in multi-agent systems.

    Source: §5 · p. 9

    Its evidence assumes tasks can be replayed under matched configurations and scored automatically, so open-ended, human-judged, and irreversible-action settings do not provide such evidence directly.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Memory helps LLM-based multi-agent systems reuse experience, but stored advice can be flawed or applied in the wrong situation. Team-level success or failure alone cannot reliably show whether a memory needs correction or should not have been used. We propose DiscernMem, a training-free framework that improves both memory quality and memory use. To improve quality, Defect-Aware Memory Refinement (DMR) compares team executions with a single memory and with no memory, diagnoses rule defects, and tests repairs on current and earlier tasks to guard against losing previously useful behavior. To guide appropriate use, Boundary-Learning Meta-Memory (BLM) records each memory's past uses and effects, learning when its owner role should apply or avoid it based on task context and incoming messages. Together, these components help teams correct flawed experience while retaining useful rules that were applied incorrectly. Experiments with Qwen3-4B-Instruct-2507 on four benchmarks show that DiscernMem achieves the highest average score on AutoGen, CAMEL, and MacNet, exceeding the strongest reported baselines by 0.51-4.32 points. It leads on code generation and symbolic planning across all three systems, including a PDDL score of 37.71 versus 28.48 on CAMEL.

1 Introduction

Reliable multi-agent memory must distinguish defective behavioral rules from useful rules applied in the wrong context. DiscernMem addresses this distinction by refining rule content and learning when roles should use memories.

1 Introduction

  • Persistent memory lets LLM agents reuse experience without changing model parameters, but a previously useful rule can become harmful under different conditions.Examples include Generative Agents retaining observations and reflections and Reflexion storing feedback-derived lessons.Source: §1 · p. 1
  • Prior memory methods organize, refine, or select experience, but semantic relevance and aggregate utility alone do not distinguish rule defects from contextual misuse.This motivates deciding whether to repair a role’s behavioral rule or revise its conditions of use.Source: §1 · p. 1
  • Team-level outcomes cannot isolate each memory’s value: agents may compensate for a defective rule, or a helpful rule may be used in a failing task.Proposed repairs also need checking beyond the latest failure to avoid damaging previously useful behavior.Source: §1 · p. 2
  • DISCERNMEM pairs execution-grounded rule diagnosis and repair with meta-memory that learns role-specific use and avoidance conditions from reuse history.Repairs are tested on the triggering and earlier tasks, while applicability conditions use context available to the owner before acting, including upstream messages.Source: §1 · p. 2
  • DISCERNMEM achieves the highest average score on AutoGen, CAMEL, and MacNet, with its clearest strengths in code generation and planning rather than uniformly better QA.The introduction reports average-score gains of 0.70, 0.51, and 4.32 points over the strongest reported baseline on AutoGen, CAMEL, and MacNet, respectively.Source: §1 · p. 2

2 Related Work

Prior work studies multi-agent collaboration structures and ways to store, organize, and adapt agent memories. DISCERNMEM complements these approaches with training-free adaptation of role-owned memories.

2.1 LLM-Based Multi-Agent Systems

  • DISCERNMEM complements collaboration frameworks with training-free adaptation of individual role-owned memories while leaving the backbone and collaboration structure unchanged.The related work discusses programmable conversations in AutoGen, roleplaying in CAMEL, structured development workflows in ChatDev and MetaGPT, and directed acyclic graph collaboration in MacNet.Source: §2.1 · p. 2

2.2 Memory for LLM Agents

  • Memory systems retain reflections, feedback-derived lessons, executable skills, linked notes, or compressed interaction histories for future decisions.Examples include Generative Agents, Reflexion, Voyager, A-Mem, and SimpleMem.Source: §2.2 · p. 2
  • Recent methods adapt memory through experience distillation, utility learning, architectural evolution, or task-family-scoped edits; DISCERNMEM distinguishes rule defects from contextual misuse.It combines replay-validated content repairs with per-memory meta-memory that learns role-specific applicability from accumulated use evidence.Source: §2.2 · p. 3

3 Method

DISCERNMEM separates memory-content refinement from applicability learning in a fixed multi-agent system. It uses paired execution evidence to diagnose and validate repairs, while meta-memory learns role-specific use boundaries.

3.1 Overview

  • The framework keeps the LLM, roles, and communication structure fixed, scores tasks before updating memories, and stores each memory’s owner, rule, scope, effects, and repair history.Its task cycle retrieves memories, executes and scores the team, then uses post-task evidence to update later tasks.Source: §3.1 · pp. 3–4, Figure 1 · p. 3
  • New experience is proposed only when no supplied memory is attributed to a content defect; proposals are matched to existing memories or made provisional, with new memories calibrated on similar earlier tasks.An LLM proposes at most one rule per role or abstains; calibration excludes the source task.Source: §3.1 · p. 4

3.2 Defect-Aware Memory Refinement

  • DMR evaluates a memory’s team effect with matched replays that supply only that memory to its owner versus no memories, then diagnoses whether harmful reuse was out of scope or reflected a supported rule defect.A harmful outcome alone does not justify rewriting: content diagnosis follows only for negative, in-scope use and compares paired trajectories for concrete defects.Source: §3.2 · p. 4
  • Candidate repairs are checked on the triggering task and sampled historical task sets; they must remove trigger harm within tolerance and avoid more complete failures on tested tasks.Accepted candidates that improve the trigger score by more than ϵ rank first; failed repair rounds retain the original rule but deactivate the memory.Source: §3.2 · p. 5

3.3 Boundary-Learning Meta-Memory

  • BLM learns per-memory use and avoidance conditions from reuse history, adding identified exceptions and expanding use conditions only after positive effects on enough distinct tasks.Lifecycle status affects eligibility; role-specific retrieval supplies selected rules with both condition sets so the owner can judge applicability before acting.Source: §3.3 · pp. 5–6

4 Experiments

The experiments compare DISCERNMEM across three multi-agent hosts and four benchmarks, then examine component contributions, memory use and refinement, and sensitivity to update and reuse budgets.

4.1 Experimental Setup

  • Evaluation covers four benchmarks across question answering, code generation, and symbolic planning, using a fixed-order prequential task stream and a shared frozen Qwen3-4B-Instruct-2507 backbone.Tasks are scored before their experience becomes available for memory updates; post-task replay scores are excluded.Source: §4.1 · p. 6

4.2 Main Results

  • DISCERNMEM leads KodCode and PDDL across all three hosts, with KodCode scores of 81.35–81.60 and PDDL scores of 29.15, 37.71, and 36.99.Its cross-host strengths are in procedural execution rather than uniformly better factual answering.Source: §4.2 · p. 6
  • DISCERNMEM achieves the highest average scores on AutoGen (59.03), CAMEL (56.93), and MacNet (59.67), ranking first in eight of twelve host–benchmark settings.The evaluation uses a shared reference configuration without host- or benchmark-specific tuning.Source: §4.2 · p. 6, §4.1 · p. 6, Table 1 · p. 7

4.3 Ablation Studies

  • The full system scores 29.15 on AutoGen/PDDL and 81.35 on CAMEL/KodCode; disabling both tested functions lowers scores to 25.36 and 78.80.Keeping only meta-memory-guided reuse yields 26.31 and 79.50, while keeping repair validation yields 27.40 and 80.20.Source: Table 2 · p. 7, §4.3 · p. 7

4.4 Further Analysis

  • Content-revised memories make up 1.2–4.3 times their share of injections, while 79.9–94.2% of memories retain their original bodies.On MacNet PopQA, revised memories are 5.8% of the store but receive 25.3% of injections; unchanged bodies may still receive BLM boundary updates.Source: Figure 2 · p. 8, §4.4 · p. 8
  • Final stores range from 44 to 5,632 memories; KodCode stores have 90.5–96.9% injected at least once, versus 39.9–62.8% for PopQA stores.Utilization measures deployment, not whether injected rules helped.Source: Figure 3 · p. 8, §4.4 · p. 8
  • On AutoGen/PDDL, increasing DMR repair rounds from one to three raises the score from 23.68 to 29.15, while further rounds do not improve the peak.The figure marks within-sweep maxima, not replacement settings for the main results.Source: Figure 4 · p. 9, §4.4 · p. 9
  • On AutoGen/PDDL, identity matching with five or more candidates yields 27.99–29.62, compared with 23.82 when every proposal becomes a new memory.The sensitivity sweep favors identity-aware admission over unconditional memory accumulation without requiring the largest tested budgets.Source: Figure 4 · p. 9, §4.4 · p. 9

5 Conclusion and Future Work

The conclusion presents DISCERNMEM as a training-free approach to distinguishing defective rules from contextually misapplied experience, and reports gains across hosts and benchmarks.

5 Conclusion and Future Work

  • DISCERNMEM combines paired-replay, retention-aware memory refinement with role-specific applicability learning from each memory’s reuse history.The authors frame memory-content refinement and applicability learning as complementary aspects of experience-driven adaptation.Source: §5 · p. 9
  • Across four benchmarks and three hosts, DISCERNMEM has the highest average score in each host’s comparison and leads in eight of twelve host–benchmark settings.The reported strengths are consistent in code generation and symbolic planning.Source: §5 · p. 9
  • Future work will investigate adaptive replay allocation, attribution to jointly supplied memories, and applicability learning under changing task distributions and team configurations.Source: §5 · p. 9

Limitations

The authors identify limits tied to replay requirements, isolated memory evaluation, applicability-boundary updates, and the scope of behavioral rules.

Limitations

  • The method relies on matched-configuration paired replays and automatic scoring, evidence not directly available for human-judged tasks or environments with irreversible actions.Replays add executions beyond each scored task, with their number growing as more memories are supplied and repairs attempted.Source: Limitations · p. 9
  • Each memory is evaluated and repaired in isolation, so rules that help only in combination and failures arising from interactions among roles’ rules may not be corrected.Source: Limitations · p. 9
  • Applicability boundaries are not self-correcting, and behavioral rules cannot supply factual knowledge missing from the backbone.An overly broad avoidance condition cannot be overturned by later positive uses; boundary quality is bounded by the backbone LLM’s scope judgments.Source: Limitations · pp. 9–10

Ethical Considerations

The work uses publicly available research artifacts and reports no human participants, annotators, or newly collected data.

Ethical Considerations

  • The evaluation uses publicly available benchmarks, models, and software, with all artifacts used for research evaluation.Source: Ethical Considerations · p. 10
  • The work involves no human annotators, human participants, or newly collected data.Source: Ethical Considerations · p. 10
  • The listed artifacts carry licenses including CC BY-NC 4.0, Apache 2.0, GPL-2.0, and MIT.The passage identifies artifact-specific licenses for datasets, code, models, and software.Source: Ethical Considerations · p. 10

A Additional Method Details

The appendix specifies memory lifecycle and boundary updates, then details repair validation, calibration, and the per-task update cycle. Repairs are checked on trigger and earlier tasks, while calibration initializes newly admitted memories before regular reuse.

A.1 Memory Lifecycle

  • Provisional memories activate after at least two positive effects and no negative effects; sustained harm deactivates them after at least three effects, including two negative and none positive.Unrepairable diagnosed defects also deactivate memories, except failed calibration repairs; persistent neutral ineffectiveness can deactivate provisional memories, and positive reevaluation can revive inactive memories.Source: §A.1 · p. 12
  • BLM adds use and avoidance conditions only after enough qualifying positive uses; avoidance takes precedence, and attachment checks use constructed context.Extraction-time conditions remain in scope, and conditions are not labeled by origin.Source: §A.1 · p. 12

A.2 Repair Validation and Calibration

  • H1 and H2 contain fixed earlier tasks, excluding trigger and source tasks; repair checks require non-regression, not positive transfer.H1 samples prior positive uses or correction-validation tasks; H2 retrieves related tasks, skipping cases where parent and no-memory scores are already maximal.Source: §A.2 · p. 12
  • Accepted repair candidates must pass trigger-task checks; they are ranked by improvement type, trigger gain, worst historical margin, then edit size.Non-improving candidates must cite trajectory evidence, state the expected correction, and change the memory body.Source: §A.2 · p. 12

A Additional Method Details

  • Each task cycle scores the original execution before recording reuse effects, updating BLM, repairing diagnosed content defects, and admitting or attaching new experience.New proposals are identity-matched to existing memories where possible; otherwise they are added provisionally and calibrated.Source: §A.2 · p. 13

A.2 Repair Validation and Calibration

  • Initial calibration replays newly admitted memories on earlier tasks; negative effects trigger repair, and the first passing candidate becomes a new provisional version.Calibration candidates must satisfy the score check on tasks with complete candidate, parent, and no-memory comparisons; if none passes, the original memory remains provisional.Source: §A.2 · p. 13

A.3 Reference Configuration

The reference configuration uses shared defaults without host- or benchmark-specific tuning. Retrieval is role-specific, while paired replays use matched configurations and a shared seed.

A.3 Reference Configuration

  • The main-result runs use shared defaults without host- or benchmark-specific hyperparameter tuning.Table 3 describes candidate, injection, repair-round, and validation-task budgets as upper bounds, with n+ as a minimum evidence threshold; calibration shares n2 and R.Source: §A.3 · p. 14, Table 3 · p. 14
  • Retrieval runs once per task and role for eligible memories owned by the receiving role; measured effects affect eligibility through lifecycle states, not similarity-ranking weights.Similarity compares observed context with memory body and use conditions; proposal matching compares the proposal with existing memories.Source: §A.3 · p. 14
  • Paired replays use matched configurations and one shared seed, and their scores inform updates without replacing the original task score.Assessing supplied memories requires one single-memory replay per memory and one shared no-memory replay; repair validation, attachment, and calibration may require additional replays.Source: §A.3 · p. 14

B Case Studies

The case studies show that an identical paired-score drop can reflect either a defective rule or contextual misuse. DISCERNMEM edits the former but preserves a sound rule used outside its scope.

B Case Studies

  • For the Gripper memory, an unverifiable drop condition caused repeated actions; deleting that clause restored the trigger task to the no-memory score and improved a related task.The revised version began provisional and was promoted after positive effects on subsequent reuses.Source: B Case Studies · p. 14
  • For the composer question, a director-credit rule was applied outside its scope; DISCERNMEM left the rule unchanged, preserving its benefit on in-scope director questions.The scope check found no use condition covering the composer question, so the harm was attributed to scope rather than rule content.Source: B Case Studies · p. 14
  • Both harmful reuses lowered paired scores, but their diagnoses assigned different causes and prompted different responses.Table 4 reports paired no-memory and single-memory scores for a PDDL and a PopQA task.Source: B Case Studies · p. 14, Table 4 · p. 15
  • Rewriting the out-of-scope rule could weaken a memory that remains useful within its scope, whereas narrowing the defective rule's scope would leave its bad clause intact.The cases therefore require distinct content-repair and applicability responses despite sharing a paired-score drop.Source: B Case Studies · p. 15

C Experimental Details

The experiments use fixed-order benchmark streams and a frozen model, with task scores recorded before experience updates memory. The appendix also specifies task composition, infrastructure, ablations, and sweeps.

C Experimental Details

  • The evaluation streams contain 6,993 TriviaQA questions, 7,267 PopQA questions, 2,000 KodCode problems, and 60 PDDL problems, scored before each task's experience updates memory.All tasks contribute to evaluation and are eligible for post-task learning; replay scores do not replace original task scores.Source: C Experimental Details · p. 15
  • The 60 PDDL problems comprise 20 Gripper, 20 Barman, 10 Blocksworld, and 10 Tyreworld tasks; progress rate is the highest fraction of goals satisfied at any episode step.A progress rate of 100 requires all goal conditions to hold.Source: C Experimental Details · p. 15
  • Reported settings use one run each and approximately 900 GPU hours, including retrieval and code-execution GPUs.Experiments used RTX 4090 GPUs; QA retrieval used two additional GPUs, and KodCode execution used one more.Source: C Experimental Details · p. 15
  • The ablations remove BLM role-and-scope filtering, DMR repair validation, or both, while retaining paired assessment and lifecycle management.Sweeps vary repair rounds on PDDL and injection and identity-matching budgets on PDDL and KodCode; sweep optima do not replace the shared main-result configuration.Source: C Experimental Details · pp. 15–16

D Additional Results

Additional results characterize memory lifecycles, reuse, revisions, and injection sensitivity; the prompts specify operational rules for extracting, matching, selecting, checking, diagnosing, generalizing, and repairing memories.

D Additional Results

  • Additional runs report lifecycle, reuse, revision, and sensitivity patterns, with injection-budget peaks differing between PDDL and KodCode.PDDL peaks at two injected memories per task and role (31.28), while KodCode peaks at one (81.90); lifecycle and revision counts vary across settings, and all nonzero injection budgets beat zero injection within the reported sweeps.Source: D Additional Results · pp. 16–19, Figure 9 · p. 18

E.1 Memory extraction

  • The extractor stores at most one transferable role-specific rule when a trajectory links an identifiable decision to progress or prevention of an evidenced error.It may abstain, and extraction is enabled only when no supplied memory is attributed to a content defect.Source: §E.1 · p. 19

E.2 Identity judgment

  • Identity judgment treats memories as duplicates only when they express interchangeable executable decisions without losing a required state-changing step.It distinguishes duplicate admission from memory-quality assessment.Source: §E.2 · p. 20

E.3 Retrieval selection

  • Retrieval selection ranks current-role memories by direct usefulness for the task’s observable decision point.The selected memories retain the similarity order established by the system.Source: §E.3 · p. 21

E.4 Scope check

  • Scope checking tests whether pre-action contexts satisfy a memory’s use or avoidance conditions.A concrete observable mismatch can be returned as a deployment avoidance boundary; later outcomes do not determine the scope judgment.Source: §E.4 · pp. 21–22

E.5 Content diagnosis

  • Content diagnosis compares a harmful isolated-memory replay with a no-memory baseline to identify a supported rule defect.Allowed diagnoses include missing steps, wrong conditions or order, unexecutable actions, missing progress checks, and wrong stop conditions.Source: §E.5 · p. 22

E.6 Use-condition generalization

  • Use-condition generalization derives use conditions from supported positive states and avoidance conditions only from qualified evidence of a missing prerequisite.The prompt directs summarizing across supplied states rather than copying a single task instance.Source: §E.6 · p. 23

E.7 Repair generation

  • Repair generation proposes the smallest evidence-supported local edit to a confirmed memory counterexample, or returns no update when none is justified.Validation comparisons identify capabilities the edit must preserve, while trigger-task comparisons define the repair target.Source: §E.7 · pp. 23–24

Related papers

Loading 2610.36652v1…