PaperlayerView exact revision on arXiv

Source-linked AI summary

EvolveMAS: A Unified Self-Evolution Framework for Decentralized Multi-Agent Systems

Yalun Wu, Qiwei Ma, Bingbing Xu, Dan Zhang, Long Bai, Boyang Wang, Jiatong Li, Zheqi Lv, Shaozu Yuan, Peiying Wang, Jiawei Wang, An Zhang, Tat-Seng Chua

arXiv:2610.34450v1cs.MAcs.AI

TL;DR

Decentralized MAS lack a unified approach to test-time adaptation and reliable experience reuse across cooperative, mixed-motive, and adversarial interactions. EvolveMAS combines a dedicated MemoryAgent and within-episode memory updates with cross-episode knowledge evolution. Across five benchmarks, it achieves a normalized average score of 77.2, exceeding the strongest setting-specific baseline by 9.7 points.

  • Problem

    How can decentralized MAS support test-time adaptation and reliable experience reuse across cooperative, mixed-motive, and adversarial settings despite context bloat and memory pollution risks?

    Source: §1 · p. 2

  • Method

    EvolveMAS pairs each main agent with an external MemoryAgent for governed Structured Memory and combines within-episode adaptation with cross-episode evolution of reusable knowledge.

    Source: §1 · p. 2, §3.1 · pp. 5–6

  • Results

    77.2 normalized average score, 9.7 points above the strongest setting-specific baseline, is EvolveMAS’s highest result across five benchmarks and six evaluation configurations.

    Source: §1 · p. 2, §4.2 · p. 8

  • Takeaways & Limitations

    The results support experience-driven adaptation across three interaction settings without updating model parameters.

    Source: §5 · p. 11

Abstract

from arXiv · show

Decentralized multi-agent systems (MAS) enable large language model agents to interact without centralized control. Agents need to adapt to other agents at test time and reuse accumulated experience to refine their interaction strategies for future interactions across cooperative, mixed-motive, and adversarial settings. However, existing approaches often develop adaptation mechanisms for particular interaction settings. This raises a central question: how can decentralized MAS support test-time adaptation and reliable experience reuse across interaction settings? We propose EvolveMAS, a unified self-evolution framework that supports experience-driven adaptation within and across episodes. EvolveMAS pairs each main agent with a dedicated MemoryAgent for test-time memory governance, which maintains isolated Structured Memory outside the main agent's reasoning context. EvolveMAS organizes adaptation into two complementary processes: Within-Episode Online Adaptation and Cross-Episode Knowledge Evolution. During Within-Episode Online Adaptation, MemoryAgent updates episodic, semantic, and working memory from interaction evidence to inform subsequent actions. During Cross-Episode Knowledge Evolution, the main agent uses memory and episode outcomes to distill reusable interaction knowledge comprising partner profiles and interaction strategies for subsequent episodes. Across five benchmarks and six evaluation configurations spanning the three interaction settings, EvolveMAS achieves the highest normalized average score of 77.2 among the compared methods. Ablations show that test-time memory governance and cross-episode knowledge reuse both improve performance. Interaction traces further illustrate how updated memory and distilled knowledge guide subsequent interactions. These results support experience-driven adaptation in decentralized MAS without updating model parameters.

1 Introduction

Decentralized MAS need a unified way to adapt to unfamiliar partners and reuse experience across cooperative, mixed-motive, and adversarial settings. EvolveMAS combines governed memory, within-episode updates, and cross-episode knowledge reuse; evaluations report the highest normalized average score among compared methods.

1 Introduction

  • The framework targets adaptation to unfamiliar partners because agents must learn from interaction experience to adjust responses and refine future strategies.The motivating settings span cooperative, mixed-motive, and adversarial interactions.Source: §1 · pp. 1–2
  • EvolveMAS pairs each main agent with a dedicated MemoryAgent that maintains Structured Memory outside the agent’s reasoning context.The framework organizes memory as episodic, semantic, and working memory and updates it during interaction to inform subsequent actions.Source: §1 · p. 2
  • Cross-Episode Knowledge Evolution uses interaction outcomes to refine reusable Partner Profiles and Interaction Strategies for future episodes.Within-Episode Online Adaptation updates memory as interactions unfold; Cross-Episode Knowledge Evolution distills knowledge for subsequent interactions.Source: §1 · p. 2
  • 77.2 normalized average score, 9.7 points above the strongest setting-specific baseline, is EvolveMAS’s highest result across five benchmarks spanning three interaction settings.The introduction reports this result alongside Figure 2’s comparison of seven methods across six configurations.Source: §1 · p. 2, Figure 2 · p. 3
  • Ablations show that test-time memory governance improves interaction performance, with additional gains from cross-episode Knowledge reuse.Additional analyses report that online memory updates improve question-answering accuracy over offline updates and that governance reduces active-context growth.Source: §1 · pp. 2–3

2 Related Work

Prior work studies decentralized coordination and adaptation, test-time self-evolution, and persistent agent memory. EvolveMAS is distinguished by applying a shared evolution framework across interaction regimes while keeping memory outside the main agent’s reasoning context.

Decentralized Multi-Agent Systems

  • Many MAS adaptation approaches modify system structures or policies for specific objectives and are typically tailored to particular interaction settings, structures, or training procedures.EvolveMAS instead maintains the same decentralized organization and evolution framework across interaction regimes.Source: §2 · p. 4

Self-Evolution

  • Existing self-evolution methods are primarily designed for specific tasks or interaction settings, while EvolveMAS couples within-episode adaptation with cross-episode Knowledge evolution.Prior work includes single-agent experience reuse and iterative adaptation in multi-agent systems.Source: §2 · p. 4

Agent Memory

  • In long-horizon decentralized interactions, retaining accumulated memory in the main agent’s context can lead to context expansion, attention dilution, and memory pollution.EvolveMAS addresses these challenges with an external MemoryAgent that maintains isolated Structured Memory and provides governed interaction state.Source: §2 · p. 4

3 EvolveMAS: Memory-Governed Self-Evolution

EvolveMAS pairs each decentralized main agent with an isolated MemoryAgent and combines online memory updates with cross-episode knowledge evolution, without updating model parameters.

3 EvolveMAS: Memory-Governed Self-Evolution

  • EvolveMAS pairs each main agent one-to-one with a dedicated MemoryAgent and isolated memory, preserving decentralized organization while supporting self-evolution without parameter updates.MemoryAgents follow a shared schema and governance protocol, but their contents evolve independently from each main agent’s local interaction history.Source: §3 · p. 4, §3.1 · p. 5

3.1 Framework Overview

  • At the episode level, Execute uses prior Interaction Knowledge to guide behavior, Evaluate derives a task-native result signal, and Evolve distills updated knowledge from governed memory and outcomes.Interaction Knowledge comprises reusable Interaction Strategies and Partner Profiles.Source: §3.1 · pp. 5–6

3.2 Within-Episode Online Adaptation

  • Within an episode, MemoryAgents update Structured Memory from newly observed evidence, which conditions the main agent’s subsequent actions.Episodic Memory records concrete events and outcomes, Semantic Memory stores reusable facts and regularities, and Working Memory maintains a compact state for the ongoing interaction; the latter is refreshed as context changes.Source: §3.2 · pp. 6–7

3.3 Cross-Episode Knowledge Evolution

  • Across episodes, the main agent uses the task-native result signal, governed Structured Memory, and prior Interaction Knowledge to distill strategies and partner profiles for future interactions.Evolved Knowledge conditions Execute in subsequent episodes, coupling cross-episode reuse with online memory updates without parameter updates.Source: §3.3 · p. 7

4 Experiments

Across five benchmarks and six evaluation configurations, EvolveMAS leads the overall comparison, while results, ablations, and examples examine how performance evolves and how its two adaptation mechanisms contribute. The experiments also report lower context occupancy and per-episode cost than retaining raw histories as experience accumulates.

4.1 Experiment Setting

  • The five benchmarks span cooperative BBH and HumanEval, mixed-motive LLM-Deliberation and SOTOPIA, and adversarial One Night Ultimate Werewolf, using heterogeneous teams and an isolated GPT-4o-mini MemoryAgent.The evaluation covers five source method families under six configurations, including benchmark-native and cross-setting comparisons.Source: §4.1 · p. 7

4.2 Main Results across Interaction Settings

  • 77.2 normalized average score; EvolveMAS ranks first in four of six configurations and exceeds the strongest setting-specific baseline by 9.7 points and Self-Memory by 7.7.Table 1 reports cross-setting results across six percentage-scaled metrics.Source: Table 1 · p. 8, §4.2 · p. 8

4.3 Cross-Setting Evolution

  • Accumulated experience supports continual evolution across cooperative, mixed-motive, and adversarial settings under the same protocol, although trajectory shapes vary by interaction structure.BBH ends 4 percentage points above native and HumanEval approximately 1.2 points above; LLM-Deliberation’s advantage persists, while SOTOPIA narrows toward parity and one-sided Werewolf gains persist over later games.Source: §4.3 · pp. 8–9, Figure 4 · p. 9

4.4 Ablating the Two Evolution Mechanisms

  • In LLM-Deliberation, external MemoryAgent governance raises unanimity from 24.0% to 74.0% without cross-episode Knowledge and from 46.0% to 86.0% with it; adding Knowledge also raises unanimity under either memory setup.Average agreement follows the same ordering across the four conditions: 4.90, 5.24, 5.36, and 5.40 parties.Source: §4.4 · p. 10, Figure 5 · p. 10

4.5 Case Study

  • In the SOTOPIA example, the seller’s $58.60 asking price is added to memory while a meet-halfway regularity is retained, informing the buyer’s next-turn $44 counteroffer.Across episodes, outcome-conditioned distillation yields a partner profile of responsiveness to concrete terms and reusable negotiation regularities.Source: §4.5 · p. 10, Figure 6 · p. 11

4.6 Additional Analyses

  • System-level analyses report stable performance across evaluated team sizes and lower per-episode cost and acting-context occupancy than retaining raw interaction histories as experience accumulates.Other analyses report higher question-answering accuracy with online than offline MemoryAgent updates, and stronger models distilling more transferable Interaction Knowledge.Source: §4.6 · p. 11

5 Conclusion

EvolveMAS applies a shared self-evolution framework across cooperative, mixed-motive, and adversarial settings, with evaluations on five benchmarks. The conclusion emphasizes memory governance and cross-episode Knowledge reuse without model parameter updates.

5 Conclusion

  • Evaluations on five benchmarks support EvolveMAS’s applicability across cooperative, mixed-motive, and adversarial settings.Ablations highlight the value of test-time memory governance and cross-episode Knowledge reuse.Source: §5 · p. 11
  • EvolveMAS supports experience-driven adaptation across the three interaction settings without model parameter updates.The conclusion frames this as a shared self-evolution framework.Source: §5 · p. 11
  • As a plug-in extension, EvolveMAS supports decentralized interaction while preserving each agent’s autonomy.The paper connects this design to integrating capabilities of agents deployed by different users without centralized control.Source: §5 · p. 11
  • Future work can vary agent capability, population size, and accumulated experience to study empirical scaling laws and design trade-offs.The stated trade-offs concern reasoning performance, computational cost, and safety.Source: §5 · p. 11

A Experimental Protocols

  • Appendix A documents transferred method-family instantiations and computation of the full Deliberation metric suite.The main text reports cross-setting results and their interpretation.Source: A Experimental Protocols · p. 13

A.1 Transfer Construction

The transfer construction preserves source methods’ native protocols while evaluating them through target benchmarks’ runners and evaluators. The appendix also explains evaluation precision and why the ablation reports unanimity on any proposal.

A.1 Transfer Construction

  • Each setting-native family retains its original role structure, team composition, and social mode, then runs through the target benchmark’s native runner and evaluator.Cross-episode memory enters through the Self-Memory row on the same team; BBH and HumanEval force cooperative mode, while other targets retain native social modes.Source: §A.1 · p. 13
  • Cooperative targets map source roles to benchmark teams, while SOTOPIA, LLM-Deliberation, and Werewolf use setting-specific team mappings.ChatDev maps to planner, developer, and tester; AgentNet or Werewolf to five heterogeneous agents; and SOTOPIA to three cooperating agents on cooperative targets. Werewolf’s fixed five-player count requires LLM-Deliberation’s native five-model configuration.Source: §A.1 · p. 13
  • For frozen-model evaluation, Werewolf/DPIns discussion tactics are encoded as a fixed prompt across all six configurations.The original method uses reinforcement learning to choose discussion tactics and belief modeling to infer other players’ roles; DPIns is the released implementation’s name for this agent.Source: §A.1 · p. 13
  • Team sizes follow benchmark-native protocols, and every main agent is paired with an isolated GPT-4o-mini MemoryAgent.The main setups include five agents on BBH, a three-role HumanEval pipeline, three SOTOPIA roles, six LLM-Deliberation parties, and five Werewolf players; BBH uses a weak lineup while the specified analyses use the strong lineup.Source: §A.1 · p. 13
  • The evaluation uses fixed benchmark sets, shared protocols within each comparison column, and paired comparisons where specified.The sets include 100 BBH questions, all 164 HumanEval tasks, 80 SOTOPIA interactions, 50 LLM-Deliberation sessions per row, and 100 games per Werewolf audit; BBH is paired item by item and SOTOPIA at the scenario level.Source: §A.1 · pp. 13–14
  • EvolveMAS’s LLM-Deliberation advantages over the native baseline and Self-Memory are statistically supported, while their 0.02 difference is not.Across 50 sessions, EvolveMAS averages 5.40 (SD 0.64), versus 4.92 (SD 0.90) for the native baseline and 4.90 (SD 0.71) for Self-Memory; the 0.48 and 0.50 advantages have 95% intervals excluding zero.Source: §A.1 · p. 14

A.2 Ablation Metric Choice

  • Unanimity on any proposal is reported because a Deliberation session is resolved when every party accepts a proposal, and it matches the average-agreement ordering carried into Table 1.The measure can depend on how many proposals a session makes, so final-proposal unanimity is also reported; MemoryAgent+Ke reaches 86.0% unanimity on any proposal but 46.0% on the final proposal.Source: §A.2 · p. 14

B Cross-Setting Transfer Profiles

The transfer profiles show setting-specific strengths, while EvolveMAS remains high across all six configurations. Cooperative benchmark results also show different sensitivity to structural mismatch.

B Cross-Setting Transfer Profiles

  • EvolveMAS is the only method that remains high across all six configurations, while cooperative, social, and adversarial methods concentrate strength in their respective regions.The profiles compare methods within each sector; annotated labels report percentage-scaled scores rounded to one decimal place.Source: B Cross-Setting Transfer Profiles · pp. 14–15
  • 72.0–74.0 on BBH is shared by ChatDev, SOTOPIA, Deliberation, and Werewolf transfers, while HumanEval scores cluster from 92.7–97.0.The passage attributes the contrast to independent tasks and a strong cooperative ensemble already performing near ceiling.Source: B Cross-Setting Transfer Profiles · p. 15

C MAS-Level Properties

The systems checks examine team-size scaling, structured cooperation versus static aggregation, and cost and context growth. Reported results show stable BBH accuracy across team sizes and bounded context occupancy with EvolveMAS in the short SOTOPIA check.

C MAS-Level Properties

  • EvolveMAS BBH accuracy stays in the 83–86 band across 1, 2, 4, 5, and 6 agents; the 4-agent score of 86.0 lies within n=100 sampling noise, indicating stable accuracy across tested sizes.Source: C MAS-Level Properties · p. 15
  • 84.0 for structured cooperation with memory exceeds 81.0 for static Mixture-of-Agents aggregation at five agents.The comparison is between structured interaction and pooling independent answers.Source: C MAS-Level Properties · p. 15, Figure 7 · p. 16
  • 0.0105 vs. 0.0341 cost and 0.62% vs. 6.54% context occupancy show lower final-episode values for EvolveMAS than No Memory.EvolveMAS pays fixed MemoryAgent overhead and then stays flat, while No Memory’s cost rises as raw interaction history accumulates in context.Source: C MAS-Level Properties · pp. 15–16, Figure 7 · p. 16
  • 94.0 on the short-horizon GSM8K check for EvolveMAS compares with 92.0 for No Memory.The passage describes this as indicating that MemoryAgent preserves performance on independent single-shot tasks.Source: C MAS-Level Properties · p. 16

D Capability-Stratified Sedimentation

The BBH lineup comparison shows EvolveMAS gains with a weaker team but not with a stronger one, while capability-stratified analysis finds that all models accumulate semantic entries but only stronger models distill reusable strategies. The accompanying breakdowns frame BBH as supplementary and Werewolf audits as the source of the WW-V and WW-W measures.

D Capability-Stratified Sedimentation

  • All five models accumulate Semantic Memory, but only the four strong/mid models fill the 40-item strategy store; Qwen3 produces many redundant, task-specific entries rather than generalizable rules.Qwen3 has 56 raw entries, deduplicated to 11 unique entries, and its unique count matches Gemini-2.5-Pro's.Source: D Capability-Stratified Sedimentation · pp. 16–17
  • The boolean_expressions example contrasts rule-level strategies from Gemini-2.5-Pro and GLM-5.2 with Qwen3's bare answers.GLM-5.2 also records a boundary case and a step-by-step evaluation meta-strategy.Source: D Capability-Stratified Sedimentation · p. 17

E Within-Setting Breakdowns

  • The breakdown section treats BBH results as supplementary and uses the Werewolf audits to define the WW-V and WW-W columns.The stronger-lineup BBH evaluation uses 100 items and shows less room for improvement as baseline performance approaches the benchmark ceiling.Source: E Within-Setting Breakdowns · p. 17, §E.1 · pp. 17–18

E.1 BBH Lineups

  • Under the weak lineup, EvolveMAS reaches 83.0%, four points above the 79.0% native baseline; under the strong lineup, it is two points below the 84.0% AgentNet baseline.Self-Memory reaches 82.0% under the weak lineup and matches AgentNet at 84.0% under the strong lineup.Source: §E.1 · p. 18, Table 5 · p. 18

E.2 Mechanism Ablation Beyond LLM-Deliberation

The factorial ablation favors the full configuration on LLM-Deliberation, BBH, and SOTOPIA, with limited resolution for some single-run comparisons. Werewolf audits show why symmetric outcomes can obscure which side benefits, while one-sided tests reveal the direction of EvolveMAS's effect.

E.2 Mechanism Ablation Beyond LLM-Deliberation

  • The full configuration attains the highest score on LLM-Deliberation, BBH, and SOTOPIA; HumanEval conditions differ by at most three tasks, and SOTOPIA differences are within episode variation.Each condition has one designated execution; BBH uses a strong lineup that already resolves most instances.Source: §E.2 · pp. 18–19, Table 6 · p. 19

E.3 Symmetric Werewolf Audit

  • In the symmetric Werewolf audit, EvolveMAS reaches 48.0% village win versus 31.0% for Self-Memory, but village win compresses opposing roles' gains into one zero-sum measure.EvolveMAS's rate declines from 0.56 in the first quartile to 0.40 in the final quartile as both sides accumulate memory.Source: §E.3 · p. 19, Table 7 · p. 19

E.4 Directional Werewolf Audit

  • On a single village-win scale, village win rises to 0.620 when only the village receives EvolveMAS and falls to 0.300 when only the wolf receives it.When both sides evolve, village win no longer tracks either side's gain, so the one-sided rows carry the directional claim.Source: §E.4 · pp. 19–20, Table 8 · p. 20
  • Under WW-W, EvolveMAS reaches 70.0 on the wolf's own objective, versus 45.0 for AgentNet, 49.0 for ChatDev, 57.0 for SOTOPIA, 46.0 for LLM-Deliberation, and 44.0 for DPIns.Under WW-V, LLM-Deliberation reaches 64.0 versus 62.0 for EvolveMAS, a two-point margin corresponding to two games at n=100.Source: §E.4 · p. 20

F Long-Horizon Memory Ablation

With starting memory held fixed, online MemoryAgent updates outperform offline batch updates overall on LoCoMo, especially on single-hop questions, while multi-hop results are tied. The comparison uses a shared judge and filter, so its arms are internally comparable but absolute accuracies are not directly comparable with published LoCoMo results.

F Long-Horizon Memory Ablation

  • Each condition answers the same 1,540 questions, scored for correctness by a gpt-4.1 judge after a deterministic filter rejects noncommittal answers when references state concrete facts.The benchmark covers ten conversations with 154 questions each.Source: F Long-Horizon Memory Ablation · p. 20
  • Absolute accuracies are not comparable with published LoCoMo results, which use the released protocol for answers produced outside a multi-agent loop.All arms in this ablation share the judge and filter, making their internal comparison consistent.Source: F Long-Horizon Memory Ablation · p. 20
  • Online updates (EvolveMAS) score 18.8 versus 16.4 for offline batch updates overall with gpt-4.1 memory; on single-hop questions, the scores are 30.9 versus 24.8.The two update strategies tie at 26.0 on multi-hop questions.Source: F Long-Horizon Memory Ablation · pp. 20–21, Figure 9 · p. 21
  • EvolveMAS scores 18.8 overall versus 16.4 for Self-Memory, and leads on single-hop fact recall (30.9 versus 17.0) and common-sense questions (19.1 versus 17.8).Self-Memory scores 42.7 versus 26.0 for EvolveMAS on multi-hop reasoning, while Self-Memory and Offline MemoryAgent tie overall.Source: F Long-Horizon Memory Ablation · p. 21, Figure 9 · p. 21

G Qualitative Cases

The qualitative cases show a shared episodic, semantic, and working-memory substrate whose stored content varies by setting. Examples range from reusable coordination and verification rules to negotiation anchors, stance changes, and adversarial cues.

G Qualitative Cases

  • Across the six benchmark columns, the same episodic, semantic, and working-memory regions are reused, but stored content adapts to each setting.Examples include role-specific coordination cues, partner signals and negotiation anchors, and deception-relevant cues that remain valid across games.Source: G Qualitative Cases · p. 21
  • The BBH collaboration skill stores a reusable verification rule: invite a collaborator to check deductions for logical consistency before finalizing.The strategy is intended to transfer across subtasks rather than cache a task-specific answer.Source: G Qualitative Cases · p. 22
  • In HumanEval, tester working memory specifies edge cases and invariants to verify next, making the reusable signal a role-specific test focus rather than a cached answer.The example checks that rolling_max handles an empty input and preserves monotone-prefix invariants.Source: G Qualitative Cases · p. 22
  • In Antique Chair bargaining, Semantic Memory records a $10 price floor and the regularity that expressing pricing flexibility can facilitate negotiation.Working memory directs the agent to maintain a friendly tone while aiming to sell for at least $10.Source: G Qualitative Cases · p. 22
  • During LLM-Deliberation, SportCo switches from its blocked opening deal to the coalition-preferred deal, and the session reaches agreement among five of six parties.The trace illustrates a stance update during deliberation.Source: G Qualitative Cases · p. 22
  • The Werewolf example retains night-action order and role-swap rules as cues for identifying suspect claims, while the same accumulation loop stores coordination and negotiation regularities in other settings.Under symmetric evolution, village win moves from 0.56 early to 0.40 late as both sides accumulate memory.Source: G Qualitative Cases · pp. 21, 23
Loading 2610.34450v1…