PaperlayerView exact revision on arXiv

Source-linked AI summary

EvoTeam: Multi-Agent Collaboration via Evolving Teams

Qinuo Cao, Xin He, Dianxuan Fu, Lizhi Zhang, Zhaoqing Xing, Jiapu Wang, Xin Wang, Na Zou, Yi Chang, Qinggang Zhang

arXiv:2610.39813v1cs.AI

TL;DR

Evolving individual agents does not ensure effective collaboration, and heterogeneous tasks can make shared team structures and reused interaction rules unsuitable. EvoTeam jointly evolves team organization, team-specific rules, and agent skills; across four benchmarks, it leads all six metrics in mixed-benchmark evolution.

  • Problem

    Existing approaches can constrain team reorganization and reuse interaction rules across incompatible collaboration contexts, complicating evolution across heterogeneous tasks.

    Source: §1 · p. 2

  • Method

    EVOTEAM maintains a library of teams, selects a compatible team or creates one, and jointly evolves team organization, team-specific interaction rules, and agent skills through execution experience.

    Source: §1 · p. 2

  • Results

    Across four benchmarks and six task categories, EVOTEAM led all six metrics under mixed-benchmark evolution, averaging 59.58 versus 55.55 for the strongest baseline.

    Source: §6 · p. 8

  • Takeaways & Limitations

    Cross-backbone evaluations and ablations support the transferability of evolved collaboration experience and the value of jointly evolving team organization and coordination practices.

    Source: §6 · p. 8

    The evaluation does not systematically examine collaborative evolution in multimodal interactive environments.

    Source: Limitations · p. 9

Abstract

from arXiv · show

LLM-based multi-agent systems (MAS) rely on collaboration among specialized agents to solve complex tasks. Existing methods improve MAS by refining the prompts, skills, and execution strategies of individual agents based on task feedback. However, improving individual agents does not necessarily lead to better team collaboration. Recent studies have therefore extended evolution to the collaboration level, using collective execution experience to adjust team organization, and inter-agent interactions. Nevertheless, evolving collaboration across heterogeneous tasks remains challenging, as existing approaches often constrain team reorganization within a shared structure and reuse interaction rules across incompatible team contexts. In this paper, we propose EvoTeam, a collaboration-centric evolution framework that jointly evolves team organization, interaction rules, and individual agent skills through execution experience. It maintains a library of reusable teams, selecting or creating agent teams and guiding collaboration through team-specific interaction rules. Collaboration experience accumulates within compatible team contexts, while individual skills remain reusable across teams. Experiments across four benchmarks demonstrate its effectiveness. Our code is available at https://github.com/DEEP-JLU/EvoTeam.

1 Introduction

Experience-driven improvements to individual agents do not ensure effective collaboration, while heterogeneous tasks can make shared team structures and reused interaction rules unsuitable. EVOTEAM addresses this by evolving teams, team-specific interaction rules, and individual skills through execution experience.

1 Introduction

  • Existing collaboration-evolution approaches can constrain team reorganization and reuse interaction rules across incompatible contexts, while heterogeneous-task evolution can underperform target-only evolution.The resulting mismatches may produce unsuitable handoffs or stopping conditions.Source: §1 · p. 2
  • EVOTEAM jointly evolves team organization, interaction rules, and individual agent skills through execution experience.Its stated components support team organization and coordination rules improving jointly through accumulated execution experience.Source: §1 · p. 2
  • EVOTEAM maintains reusable teams, selecting a compatible team or creating one for task requirements, and coordinates through task-specific responsibilities and team-specific interaction rules.Compatible tasks can reuse and refine rules; updates remain independent across teams, while individual skills remain reusable across teams.Source: §1 · p. 2
  • Experiments across four benchmarks and six task categories examine EVOTEAM under individual- and mixed-benchmark evolution, with ablations, cross-backbone evaluations, and case analyses.These analyses examine collaboration mechanisms and experience reuse.Source: §1 · p. 2

2 Preliminary Study

The preliminary study examines whether experience from previous tasks benefits continued multi-agent evolution across heterogeneous tasks and investigates how team organization and interaction-rule applicability may limit reuse.

2.1 Cross-Task Performance Degradation

  • Both Inter-CT and Intra-CT underperform target-only META-TEAM on all three metrics, including when Intra-CT retains a compatible team template.Inter-CT starts from another benchmark with an unsuitable template; Intra-CT starts from another task category in the same benchmark with a compatible template.Source: §2.1 · p. 3, Figure 2 · p. 3

2.2 Analysis of Potential Failure Modes

  • A source-oriented team may lack the target task’s required division of labor, while inherited rules can trigger inappropriate handoffs or premature stopping.The example rule directs a file agent to report findings and finalize, potentially ending an implementation task before coding and testing.Source: §2.2 · p. 3

2.3 Discussion

  • The study motivates task-appropriate team structures and sharing or refining interaction rules only among tasks with compatible collaboration requirements.Accumulated agent skills may remain useful across team contexts, unlike team templates and interaction rules reused without regard to new task requirements.Source: §2.3 · p. 3

3 The Framework of EVOTEAM

EvoTeam selects or creates task-appropriate teams, guides execution with team-specific rules and accumulated agent skills, then uses execution evidence to refine teams, rules, and skills.

3.1 Adaptive Agent Team Composition

  • The library stores reusable teams with collaboration requirements and candidate rosters; for each task, a compatible team is selected or a new provisional team is created.A Chairman recruits task-specific agents from the selected roster, and the same team can recruit different subsets for different tasks.Source: §3 · p. 3, §3.1 · p. 4

3.2 Team-Scoped Collaboration

  • Interaction rules are scoped to the selected team, while agents use their accumulated skills and consult applicable rules to coordinate and hand off information.Agents select rules according to their descriptions and current coordination needs; the Chairman can reassign subtasks when an allocation blocks progress, and the system records agent and handoff traces.Source: §3.2 · pp. 4–5

3.3 Multi-Level Experience Evolution

  • Team-level reflection uses the query and composite execution trace to assess whether the candidate roster supports the required division of work, revising the roster and team description when needed.New teams enter the library only when their roster supports the required division of work and a reusable collaboration structure.Source: §3.3 · p. 5
  • Rule reflection uses the handoff trace to identify missing information or mismatches with agent responsibilities, then creates, revises, or removes team-specific rules.Updated rule sets remain reusable within their associated teams.Source: §3.3 · p. 5
  • Agent reflection uses each agent’s local trace to add useful procedures, revise skills after observed errors, or retain the current skill set when evidence for change is unclear.Updated agent skills remain reusable across teams.Source: §3.3 · p. 5

4 Experiments

The experiments evaluate EVOTEAM across individual- and mixed-benchmark evolution, cross-backbone transfer, component ablations, and performance–cost trade-offs. EVOTEAM leads the mixed-benchmark metrics, while transfer and ablation results examine how collaboration experience performs across backbones and team configurations.

4.1 Experimental Setup

  • The evaluation uses four benchmarks, six task categories, held-out splits, and GPT-5.6-Luna by default.Individual-benchmark evolution optimizes each benchmark separately; mixed-benchmark evolution interleaves the four evolution splits.Source: §4 · p. 6, §4.1 · p. 6

4.2 Overall Effectiveness (Q1)

  • Under mixed-benchmark evolution, EVOTEAM ranks first on all six metrics, averaging 59.58 versus Meta-Team’s 55.55; under individual-benchmark evolution, it averages 58.85 and leads four of six metrics.Its individual-benchmark advantage over Meta-Team spans GAIA, both LoCoBench tasks, and BeyondSWE DepMig.Source: §4.2 · pp. 6–7

4.3 Cross-Backbone Transfer (Q2)

  • Across all six cross-backbone configurations, EVOTEAM beats unevolved MAS; transfer performance stays within 4.00 points of same-backbone results.On GAIA, GPT-5.6-Luna evaluation scores 77.00 with DeepSeek evolution versus 78.00 with same-backbone evolution.Source: §4.3 · p. 7, Table 2 · p. 7

4.4 Ablation Study (Q3)

  • Learned interaction rules improve full EVOTEAM over the team-only variant on all four tasks, with gains of 10.00 points on GAIA and 8.75 on LOCA.The smaller gains on coding tasks indicate that the contribution of interaction guidance varies by task.Source: §4.4 · p. 7, Figure 4 · p. 8
  • Task-conditioned team templates improve performance even without interaction rules, with gains of 6.00 points on GAIA and 2.50 on LOCA.LoCoBench Feat and BeyondSWE also improve when templates are retained.Source: §4.4 · p. 8

4.5 Cost Efficiency (Q4)

  • On GAIA and BeyondSWE DepMig, EVOTEAM scores highest while costing less per task than Meta-Team and ReCreate; Skill-MAS costs less but performs worse.The reported cost comparison is under mixed-benchmark evolution.Source: §4.5 · p. 8

5 Related Work

Prior experience-driven work improves individual agents or evolves collaboration, while EVOTEAM focuses on sustaining collaboration evolution across heterogeneous tasks.

Individual Agent Evolution

Collaborative Multi-Agent Evolution

  • Meta-Team, SkillMAS, and EMAS use collective execution experience to evolve agent behavior, coordination, or orchestration, while EVOTEAM maintains reusable teams with separate interaction rules.EVOTEAM accumulates coordination experience within compatible team contexts while keeping individual skills reusable across teams.Source: §5 · p. 8

6 Conclusion

EVOTEAM jointly evolves reusable team organization, team-specific interaction rules, and individual agent skills for collaboration across heterogeneous tasks. Across four benchmarks and six task categories, it leads all six mixed-benchmark metrics, while transfer evaluations and ablations support the transferability and value of its collaboration experience.

Conclusion

  • Under mixed-benchmark evolution, EVOTEAM leads all six metrics, averaging 59.58 versus 55.55 for the strongest baseline.The framework accumulates coordination experience within compatible team contexts while preserving skill reuse across teams.Source: §6 · p. 8

Limitations

The evaluation does not systematically examine collaborative evolution in multimodal interactive environments, and EVOTEAM keeps LLM parameters fixed.

Limitations

  • Multimodal interactive evolution remains unevaluated systematically; experience reuse also depends on the base model correctly interpreting and applying stored guidance.EVOTEAM accumulates experience through explicit updates while keeping LLM parameters fixed.Source: Limitations · p. 9

Ethical Considerations

EVOTEAM’s deployment requires oversight of inherited model risks and persistent experience, while its procedure preserves team-scoped rules and reusable agent skills. Evaluation uses disjoint evolution and holdout data across four benchmarks and mixed-task settings.

Ethical Considerations

  • Applications beyond evaluated benchmarks should address model bias, inaccurate outputs, unintended tool actions, sensitive information in traces, and suitability of accumulated knowledge.The paper recommends access controls, privacy protection, human oversight for consequential actions, and review of reused rules and skills.Source: Ethical Considerations · p. 9

A Algorithms

  • After each task, three reflectors update team organization, team-specific interaction rules, and individual skills, which remain with agents across team contexts.Existing teams retain their identities when revised; provisional teams are admitted only when their roster supports the required work division and reusable organization.Source: A Algorithms · p. 12

B.1 Benchmark Details

  • The evaluation covers GAIA, LoCoBench, LOCA-Bench, and BeyondSWE, using benchmark-specific metrics and disjoint evolution and holdout splits.BeyondSWE evaluates CrossRepo and DepMigrate tracks; the LOCA-Bench holdout spans five seeds and six environment-description lengths for eight task types.Source: §B.1 · pp. 12–13
  • Mixed-benchmark evolution processes 116 shuffled instances from the source benchmarks, then evaluates the frozen system separately on source-specific holdout groups.Each task retains its source tools, environment, output contract, and evaluator.Source: §B.1 · pp. 12–13

B.2 Evaluation Metrics

The experiments use source-native evaluation protocols and shared resource conditions to compare EVOTEAM with single-agent, fixed-team, and evolving MAS baselines. LoCoBench scores combine four normalized dimensions.

B.2 Evaluation Metrics

  • Scores use benchmark-native protocols: exact-answer accuracy on GAIA, execution-state success on LOCA-Bench, and resolved-instance tests on BeyondSWE.BeyondSWE resolution requires passing all required Fail-to-Pass tests while preserving Pass-to-Pass tests.Source: §B.2 · p. 13
  • The eight LOCA-Bench task types each contribute 30 distinct holdout cases spanning five seeds and six environment-description lengths.The benchmark task types follow Meta-Team (Hao et al., 2026c).Source: Table 3 · p. 14
  • LoCoBench Score combines software engineering excellence, functional correctness, code quality, and long-context utilization, with continuous scores averaged within each subset.Scores are reported on a [0, 100] scale, equivalent to multiplying native [0, 5] LCBS by 20.Source: §B.2 · p. 14

B.3 Baseline Details

  • The seven baselines include single-agent and fixed-team systems, architecture or scaffold search, meta-skill evolution, and experience-driven scaffold or collaboration evolution.ReCreate updates shared scaffolds from execution evidence, while Meta-Team evolves agent behavior, collaboration, and team organization.Source: §B.3 · pp. 14–15

B.4 Evolution and Evaluation

The study separates independent-benchmark evolution from mixed-benchmark evolution and freezes evolved systems before holdout evaluation. Additional analyses examine dimensions, difficulty levels, and component ablations.

B.4 Evolution and Evaluation

  • In individual-benchmark evaluation, each benchmark or track evolves independently, with no experience shared across settings and configurations frozen before disjoint holdout evaluation.SA, MAS, and OWL are evaluated without prior evolution; the other listed methods evolve on the corresponding evolution split.Source: §B.4 · p. 15
  • Mixed-benchmark evolution carries experience across a fixed shuffled stream of 116 instances, then evaluates each frozen system on source-specific holdout groups.ReCreate, Meta-Team, Skill-MAS, and EVOTEAM start from the same shared agent pool and retain each task’s source-specific tools and evaluator.Source: §B.4 · p. 15
  • Methods share benchmark-specific tools, partitions, and per-task resource limits; LOCA-Bench budgets remain fixed across environment-description lengths.The paper also analyzes GAIA difficulty levels, LoCoBench evaluation dimensions, and component ablations.Source: §B.4 · p. 15, C Additional Experiments · p. 15

C.1 Component Ablation

The ablations and case analyses examine how reusable teams and scoped interaction rules contribute to collaboration evolution, while additional results characterize cost, task difficulty, and benchmark-specific strengths.

C.1 Component Ablation

  • The complete framework performs best in five of six settings and beats the variant removing both teams and rules in all six.Removing both components lowers performance on GAIA, LOCA, LoCoBench Feature Implementation, and BeyondSWE CrossRepo; removing rules slightly improves LoCoBench Refactoring (66.40 vs. 65.65).Source: §C.1 · pp. 15–16, Figure 6 · p. 16

C.2 Cost Analysis

  • EVOTEAM achieves the highest score at the lowest estimated cost on GAIA and both LoCoBench tasks.On BeyondSWE DepMig, it outperforms Meta-Team and ReCreate at lower cost, while AgentSquare and OWL are cheaper but score lower; Figure 7 compares performance and estimated cost per task.Source: §C.2 · p. 16, Figure 7 · p. 16

C.3 GAIA Difficulty Levels

  • EVOTEAM has the highest GAIA pass rate at all three difficulty levels, with gains over Meta-Team of 2.63, 16.32, and 7.69 percentage points on Levels 1, 2, and 3.Level 3 scores remain much lower for all methods.Source: §C.3 · p. 16, Table 7 · p. 17

C.4 LoCoBench Dimensions

  • EVOTEAM leads in LoCoBench software engineering excellence and functional correctness on both subsets, but ReCreate leads code quality on both and Meta-Team leads Feature Implementation long-context utilization.On Refactoring, EVOTEAM improves functional correctness from 0.7148 to 0.7494 over Meta-Team and has the highest long-context utilization score.Source: §C.4 · p. 17, Table 6 · p. 17

D.1 Team Selection

  • Team selection reuses a compatible collaboration structure or creates a provisional team, whose candidate roster supports Chairman recruitment and task-specific assignments.The Chairman coordinates the recruited agents, reviews returned work, resolves conflicts, and may adjust assignments.Source: §D.1 · p. 17, §D.2 · p. 17, §D.3 · p. 18

D.4 Interaction-rule Selection and Use

  • Interaction rules guide handoffs only when their trigger and information-transfer need apply; post-task reflection can create, refine, retain, retire, or leave rules unchanged.Separate reflection prompts assess team membership and individual skills, with agent updates favoring small supported changes and excluding one-off answers and team-level failures.Source: §D.4 · p. 18, §D.6 · p. 19, §D.5 · p. 19

E.1 Scoped Experience Accumulation

  • On CourseAssistant, EVOTEAM’s scoped update retained prior rules and added a family-specific verification rule; evaluation returned 1.0 after checking 62 individualized sent messages.The comparison reports that Meta-Team’s shared profile-note replacement removed accumulated distinctions, but does not establish the cause of the score difference.Source: §E.1 · pp. 22–23, Figure 16 · p. 22

E.2 Cross-Benchmark Accumulation

  • Across four LoCoBench and BeyondSWE tasks, a retained team accumulated coordination rules and expanded its candidate roster, while the final task recruited only three agents.The team’s handoffs carried calling sites, behavioral constraints, change summaries, and compatibility checks under task-specific plans rather than a fixed execution sequence.Source: §E.2 · pp. 23–24, Figure 17 · p. 23, Figure 19 · p. 24

E.3 Cross-Benchmark Team Reuse

The section illustrates how task-matched teams and team-scoped handoff rules support collaboration, including reuse across benchmarks when tasks share a coordination pattern.

E.3 Cross-Benchmark Team Reuse

  • Across escrow, clearance-promotion, and compatibility-repair tasks, the team applies dependency analysis, bounded implementation, and independent verification.For escrow, the rules require an interface inventory before implementation and verification evidence for affected components.Source: §E.3 · pp. 24–25

E.4 Organization and Coordination

  • 62.54 versus 44.47 LCBS: EvoTeam outperforms Meta-Team on the Carnival refactoring task.The comparison is shown in Figure 20(c).Source: Figure 20 · p. 25, §E.4 · p. 26

E.3 Cross-Benchmark Team Reuse

  • All three holdout tasks succeed with one shared team because they follow analysis, contract-preserving implementation, and independent verification.The tasks span LoCoBench, LOCA-Bench, and BeyondSWE.Source: §E.3 · pp. 24–25

E.4 Organization and Coordination

  • EvoTeam assigns a verifier and uses team-scoped rules to connect responsibilities, while Meta-Team’s later unvalidated edits leave an invalid wrapper.The evaluator compiles every Python file under solution/; an invalid wrapper receives compilation score 0, and FC loses that component.Source: §E.4 · pp. 25–26, §E.3 · p. 25

F.1 Individual Agent Evolution

  • Prior individual-agent evolution retains feedback, reflections, skills, or execution knowledge, while collaborative performance also depends on alignment between coordination structures and task requirements.This motivates asking how accumulated capabilities should be organized when tasks require different responsibilities and information handoffs.Source: §F.1 · p. 26

F.2 Collaborative Multi-Agent Evolution

  • EvoTeam maintains reusable teams, refines team organization and associated interaction rules within compatible contexts, and keeps individual skills reusable across teams.Rule updates remain independent across teams.Source: §F.2 · p. 27

G Frequently Asked Questions

EvoTeam organizes teams around collaboration requirements and evolves team organization, interaction rules, and agent guidance using execution experience. In the reported mixed-benchmark comparisons, it achieves stronger performance at lower per-task cost than Meta-Team and ReCreate on GAIA and BeyondSWE DepMig.

Q2 Team organization

  • Teams are organized by collaboration requirements, not benchmark or task-category labels; the selector considers division of work, complementary roles, and verification needs.When no compatible team exists, the Composer constructs a provisional team whose admission is determined after execution.Source: G Frequently Asked Questions · pp. 27–28

Q3 Interaction rules

  • Interaction rules guide handoff information and evidence without prescribing a complete execution sequence, and assignments can change when progress is blocked.Rule applicability depends on the sender–recipient pair, trigger, and current information-transfer requirements.Source: G Frequently Asked Questions · p. 28

Q4 Rules and skills

  • Rules remain team-specific because they depend on team responsibilities and information dependencies, while individual skills remain reusable across teams.The FAQ says team-specific rules are isolated, whereas skills and recruitment profiles remain shared.Source: G Frequently Asked Questions · p. 28

Q5 Evolution and evaluation

  • Evolution updates team rosters and descriptions, team-specific rules, and individual guidance, then freezes persistent updates before disjoint holdout evaluation.Task-specific recruitment, planning, rule use, and reassignment still occur during execution.Source: G Frequently Asked Questions · p. 28

Q6–Q7 Reuse and evidence

  • Examples across LoCoBench and BeyondSWE show shared teams accumulating coordination experience across benchmarks when tasks have compatible requirements.Ablations assess learned interaction rules’ contribution beyond task-conditioned team organization, while a case analysis illustrates team-scoped updates.Source: G Frequently Asked Questions · p. 28

Q8 Performance–cost trade-off

  • EvoTeam achieves stronger performance at lower reported per-task cost than Meta-Team and ReCreate on GAIA and BeyondSWE DepMig under mixed-benchmark evolution.The FAQ attributes this comparison to Figures 5 and 7.Source: G Frequently Asked Questions · p. 28
Loading 2610.39813v1…