PaperlayerView exact revision on arXiv

Source-linked AI summary

Not Every Agent Needs the Strongest Model: Unified Prompt-Model Optimization for Heterogeneous LLM-based Multi-Agent Systems

Mingxiu Cai, Haoyang Xian, Xinyu Ma, Yutong Wang, Zhenxi Song, Jing Li, Xuebo Liu

arXiv:2610.31743v2cs.MA

TL;DR

Prompt optimization and model assignment are commonly searched separately, despite their coupled effects across agent roles and execution topologies. UniMASO jointly adapts prompts and model assignments using cost-aware, topology-aware search and cross-level trajectory memory. Across five benchmarks and five topologies it achieves favorable quality–cost trade-offs; centrally, it improves average score while substantially reducing deployment cost.

  • Problem

    Prompt optimization generally fixes model assignments and model-routing methods typically fix prompts, leaving role requirements and inference costs outside a joint search.

    Source: §1 · p. 1

  • Method

    UniMASO jointly searches prompts and model assignments using multi-granularity feedback, cost- and topology-aware coordination, and cross-level trajectory memory.

    Source: §1 · p. 2

  • Results

    Across five benchmarks and five execution topologies, UniMASO achieves favorable quality–cost trade-offs; centrally, average score rises from 69.79 to 70.40 as token-weighted deployment cost falls by 85.2% versus DeepSeek-V4-Pro with MASPO.

    Source: §4.2 · p. 6, §6 · p. 10

  • Takeaways & Limitations

    The findings favor joint prompt-model adaptation for efficient multi-agent collaboration.

    Source: §6 · p. 10

Abstract

from arXiv · show

Large language model (LLM)-based multi-agent systems (MAS) rely on role-specific prompts and execution models to solve complex tasks collaboratively. However, prompt optimization typically keeps model assignments fixed, leaving differences in role requirements and their associated inference costs outside the search. We present UniMASO, a Unified framework for heterogeneous Multi-Agent System Optimization through the joint adaptation of prompts and model assignments. UniMASO evaluates candidate updates through local, downstream, and global feedback, using cost-aware Pareto search to retain useful quality-cost trade-offs. Topology-aware co-evolution then coordinates these updates according to agent roles and dependencies. Cross-level trajectory memory further supplies historical successes and failures to guide subsequent optimization. Experiments on five benchmarks and five execution topologies show that UniMASO achieves competitive task performance at substantially lower deployment cost than the strongest homogeneous configurations. Under centralized execution, it improves the average score from 69.79 to 70.40 while reducing token-weighted deployment cost by 85.2% relative to DeepSeek-V4-Pro with MASPO. These results support joint prompt-model adaptation as an effective approach to efficient multi-agent collaboration. The code is available at https://github.com/Achilles008/UniMASO_main.

1 INTRODUCTION

Prior methods generally optimize prompts or assign models separately, although both choices shape role capabilities, downstream information, quality, and cost. UniMASO jointly adapts prompts and models with topology-aware, cost-conscious search and trajectory memory.

1 INTRODUCTION

  • Prompt optimizers generally fix execution models, while model-routing methods typically fix prompts, leaving role requirements and inference costs outside a joint search.Fixed assignments can waste strong models on routine roles or bottleneck demanding roles with lightweight models.Source: §1 · p. 1
  • Because prompt and model changes propagate through the agent graph, independent search can favor locally attractive configurations that perform poorly at system level.Changes can alter downstream context, end-to-end quality, and cost.Source: §1 · p. 1
  • Under centralized execution, UniMASO raises the average score from 69.79 to 70.40 while reducing token-weighted deployment cost by 85.2% versus DeepSeek-V4-Pro with MASPO.The evaluation spans five benchmarks and five execution topologies.Source: §1 · p. 2
  • UniMASO formulates heterogeneous MAS optimization over prompt–model configurations, making role–capability misalignment explicit.Each agent’s prompt shapes behavior and information passed downstream, while its execution model determines capabilities and inference cost.Source: §1 · pp. 1–2
  • Its cost- and topology-aware search combines multi-granularity evaluation with system-level co-evolution across dependent agents.Candidate updates receive local, downstream, and global feedback, and cost-aware Pareto search retains quality–cost trade-offs.Source: §1 · p. 2
  • Cross-level trajectory memory turns local feedback, global outcomes, and cost observations into experience for later optimization rounds.The memory guides subsequent search using historical successes and failures.Source: §1 · p. 2

2 RELATED WORK

Prior multi-agent and prompt-optimization work automates role workflows and prompt search, while model-assignment methods address heterogeneous capabilities. UniMASO connects model assignment to iterative prompt optimization and topology-level feedback.

2.1 MULTI-AGENT SYSTEMS AND PROMPT OPTIMIZATION

  • Prior systems establish role-based workflows and automate agent graphs or prompt and topology co-evolution, while prompt optimizers search instructions using scores, traces, or language feedback.These methods include nodewise coordination and topology-guided search, but keep node-level execution models fixed.Source: §2.1 · p. 2

2.2 HETEROGENEOUS MODEL ASSIGNMENT

  • Heterogeneous model-selection methods route models or configure roles and collaboration structures, but typically decouple model choice from evolving prompts and downstream effects.UniMASO instead makes model choices depend on candidate prompts, topological position, downstream influence, and realized inference cost.Source: §2.2 · p. 2
  • UniMASO’s overview links trace-based prompt-candidate evaluation, heterogeneous execution, topology-aware assignment updates, and cross-level trajectory-memory retrieval.Retrieved historical evidence guides subsequent optimization rounds.Source: Figure 2 · p. 3

3 METHODOLOGY

UniMASO jointly searches prompts and execution models, evaluating local changes in context and coordinating them across the agent topology. Cost-aware selection and trajectory memory guide this process toward useful quality–cost trade-offs.

3 METHODOLOGY

  • UniMASO alternates local prompt–model exploration with topology-aware system coordination, evaluating agents in topological order and storing accepted updates for later rounds.Local proposals use interaction traces and multi-granularity evaluation; system coordination reconciles them under the realized graph.Source: §3 · p. 3

3.1 PROBLEM FORMULATION

  • The system is modeled as a directed graph whose edges pass agent outputs as context, with each agent assigned a role prompt and execution model.The joint objective maximizes task quality while minimizing average token cost, so changes are evaluated at system level.Source: §3.1 · pp. 3–4

3.2 COST-AWARE MULTI-AGENT OPTIMIZATION

  • Candidate prompts are evaluated through local validity, successor-output utility, and global alignment, then filtered by cost-adjusted beam search that retains nondominated quality–cost options.Retained candidates are reevaluated after peer updates before the best local state is passed to system coordination.Source: §3.2 · p. 4

3.3 TOPOLOGY-AWARE HETEROGENEOUS MODEL ADAPTATION

  • UniMASO extends local search to prompt–model pairs and coordinates topology-conditioned proposals under model availability, role-sharing, routing, and capability–cost constraints.Proposal support is weighted by concentration, giving more weight to concentrated focal-view proposals in the configuration-level decision.Source: §3.3 · pp. 4–5

3.4 TRAJECTORY MEMORY-DRIVEN SELF-EVOLUTION

  • Trajectory memory records preceding configurations, proposed modifications, multi-granularity feedback, execution cost, and acceptance outcomes for later evolution.Records distinguish low-cost global successes, local successes with global failures, and high-cost global successes; retrieved experiences guide updates under current dependencies and models.Source: §3.4 · p. 5

4 EXPERIMENTS

The experiments evaluate UniMASO across five benchmarks and five execution topologies, comparing heterogeneous prompt–model optimization with prompt-only and homogeneous baselines. UniMASO improves the centralized average score while sharply reducing deployment cost, and its lower-cost trade-off extends across topologies.

4.1 EXPERIMENTAL SETUP

  • The evaluation covers AGIEval-MATH, GPQA-Diamond, BFCL-V3, LiveCodeBench v6, and HotpotQA across Independent, Centralized, Sequential, Decentralized, and Dynamic topologies.The model pool includes Qwen3.5-4B, Qwen3.5-9B, Qwen3.5-27B, DeepSeek-V4-Flash, and DeepSeek-V4-Pro.Source: §4.1 · p. 5
  • Comparisons include prompt optimizers and multi-agent baselines; prompt-only MAS methods optimize role prompts while keeping topology and model assignments fixed.Five homogeneous configurations are compared with UniMASO’s heterogeneous assignment using identical evaluation and deployment-cost accounting.Source: §4.1 · p. 5
  • Scores use accuracy, Pass@1, or exact match by benchmark, while average columns report mean benchmark scores and corresponding total deployment costs.Deployment cost sums agent input and output token counts weighted by fixed relative model-cost coefficients; reported costs cover each full evaluation set.Source: §4.1 · p. 6

4.2 MAIN RESULTS

  • 70.40 average score at 30.92M cost beats DeepSeek-V4-Pro + MASPO’s 69.79 at 208.28M, reducing token-weighted deployment cost by 85.2%.The gain is 0.61 points; UniMASO leads on AGIEval-MATH and HotpotQA, while homogeneous configurations score higher on the other three benchmarks.Source: §4.2 · p. 6, Table 2 · p. 6
  • Across five topologies, UniMASO lowers deployment cost in all and improves mean score in four; decentralized execution trades 66.96 for 66.58 while reducing cost from 239.80M to 60.37M.In Dynamic execution, UniMASO scores 70.37 at 20.56M versus 70.08 at 134.13M for the homogeneous reference.Source: §4.2 · p. 7, Figure 3 · p. 7

5 ANALYSIS

The analysis tests UniMASO’s components and settings, then examines coordination, cross-dataset transfer, and optimization cost. Results identify useful joint adaptation and coordination, and report favorable quality–cost trade-offs across the tested settings.

5.1 ABLATION STUDY OF UNIMASO

  • The staged prompt-then-model control scores 66.39 at 64.15M deployment cost, versus UniMASO’s 70.40 at 30.92M, with losses across all five benchmarks.On GPQA-Diamond, accuracy falls from 78.28 to 71.72 while cost rises from 33.02M to 70.24M.Source: §5.1 · p. 7, Table 3 · p. 8
  • Without topology-aware co-evolution, average score falls from 70.40 to 68.49 while deployment cost changes only from 30.92M to 31.20M.The largest score drop is 4.54 points on GPQA-Diamond; HotpotQA improves by 0.20 points.Source: §5.1 · p. 7, Table 3 · p. 8
  • Without cost-aware Pareto search, average score falls from 70.40 to 69.42 and deployment cost rises from 30.92M to 62.40M.On BFCL-V3, accuracy rises slightly from 92.40 to 92.60 as cost increases from 12.21M to 24.53M.Source: §5.1 · p. 7, Table 3 · p. 8
  • Cross-task memory records raise scores from 80.08 to 81.64 on AGIEval-MATH and from 76.77 to 77.78 on GPQA-Diamond, while own-task records remain most accurate.Cross-task costs are 52.93M and 29.43M, below own-task costs of 56.86M and 33.02M, respectively.Source: Table 4 · p. 8, §5.1 · p. 8

5.2 OPTIMIZATION DYNAMICS AND QUALITY–COST TRADE-OFFS

  • Three rounds reach 70.40, with a fourth adding 0.16 points; among tested search schedules, 3–2–2 yields the best quality–cost balance.The default Kmem = 2 gives the strongest average among tested retrieval sizes, and λ = 0.2 has the highest score and lowest deployment cost in the cost-weight sweep.Source: §5.2 · pp. 8–9

5.3 COORDINATION EFFICIENCY AND MODEL BEHAVIOR

  • UniMASO raises Communication Score from 72.84 to 73.76 and reduces Cost-of-Pass from 1.241 to 0.184 (85.2%) versus centralized +MASPO, with comparable Planning Score.Table 5 reports the coordination and efficiency comparison.Source: Table 5 · p. 9, §5.3 · p. 9
  • Qwen3.5-9B is selected most often for structured processing and summarization (43.5%), while DeepSeek-V4-Flash dominates reasoning and verification (45.5%).UniMASO’s primary-agent responses span a broader region than DeepSeek-V4-Pro + MASPO in both t-SNE projections, particularly on GPQA-Diamond.Source: §5.3 · p. 9, Figure 5 · p. 9, Figure 6 · p. 9

5.4 CROSS-DATASET TRANSFER OF OPTIMIZED CONFIGURATIONS

  • Cross-dataset UniMASO achieves higher accuracy at lower cost than Vanilla MAS and MASPO on both targets, with transfer accuracy within 1.17 points of in-domain optimization.Transfer raises deployment cost by 3.2% on AGIEval-MATH and 3.1% on MATH-500; Table 6 reports the comparison.Source: Table 6 · p. 10, §5.4 · p. 10

6 CONCLUSION

The conclusion presents UniMASO as joint prompt and model adaptation for heterogeneous multi-agent systems. Across five benchmarks and five topologies, it reports favorable quality–cost trade-offs and argues these findings favor joint adaptation for efficient collaboration.

6 CONCLUSION

  • UniMASO jointly adapts prompts and model assignments using cost-aware feedback search, topology-aware co-evolution, and cross-level trajectory memory.The conclusion reports favorable quality–cost trade-offs across five benchmarks and five execution topologies.Source: §6 · p. 10

A.1 TASK DATASETS

The evaluation covers five task areas and describes the comparison baselines and optimization algorithm. The algorithm alternates local cost-aware search with topology-aware co-evolution and stores evaluation trajectories for later rounds.

A.1 TASK DATASETS

  • Five benchmarks cover mathematical reasoning, scientific question answering, function calling, competitive programming, and multi-hop question answering.The evaluation reports benchmark-specific metrics and includes 1,585 evaluation instances per topology across five topologies.Source: §A.1 · pp. 14–15, Table 8 · p. 15

A.2 OPTIMIZATION BASELINES

  • The baselines include fixed-model prompt optimizers and an unoptimized homogeneous centralized MAS, enabling comparisons with prompt-only and unoptimized systems.The prompt-optimization baselines include TPE, SPO, MASPOB, and MASPO; their model assignments remain fixed in the described comparisons.Source: §A.2 · p. 15

B OPTIMIZATION ALGORITHM

  • UniMASO alternates cost-aware local beam search with topology-aware, memory-guided co-evolution, recording trajectories during evaluation for retrieval at round boundaries.The search state includes model assignment and relative inference cost in candidate selection.Source: B OPTIMIZATION ALGORITHM · p. 16, §A.2 · p. 16

C IMPLEMENTATION DETAILS

Implementation uses fixed optimization and evaluation settings for UniMASO and MASPO. UniMASO initializes task-agent models to Qwen3.5-9B and uses Qwen3.5-27B for optimization and evaluation.

C IMPLEMENTATION DETAILS

  • UniMASO initializes optimizable agents with Qwen3.5-9B, uses Qwen3.5-27B as optimizer and evaluator, and assigns DeepSeek-V4-Flash to the Co-Evolver.Its beam width is B = 2 and local, lookahead, and global utility weights are (0.4, 0.4, 0.2).Source: C IMPLEMENTATION DETAILS · pp. 16–17
  • MASPO uses Gemini 2.5 Pro as optimizer and evaluator, with relative input/output cost coefficients of 144/1,080.The coefficients are based on published Gemini 2.5 Pro and DeepSeek-V4-Pro API prices.Source: C IMPLEMENTATION DETAILS · p. 17

D MULTI-AGENT TOPOLOGIES AND FULL RESULTS

The topology definitions distinguish logical task roles from mechanical aggregation modules. Only LLM-backed task roles enter model assignment and deployment-time task-agent cost.

Scope

D.1 TOPOLOGY DEFINITIONS

The five topologies vary in role arrangement, communication, and downstream dependency. Table 9 compares UniMASO’s heterogeneous assignments with five homogeneous configurations on task quality and deployment cost.

Independent topology

  • Independent execution runs three agents on the same input without message passing, then mechanically combines their candidate solutions.The non-LLM aggregator is not an assignment variable, so this topology measures model and reasoning diversity without learned coordination.Source: §D.1 · p. 17

Centralized topology

  • Centralized execution sends three parallel specialists’ outputs to a terminal summarizer, yielding four LLM-backed roles and three directed edges.The summarizer handles cross-agent comparison, conflict resolution, and final answer formation, and is optimized separately because it receives concatenated upstream responses.Source: §D.1 · p. 17

Sequential topology

  • Sequential execution chains two predictor–reflector pairs, with each reflector checking the preceding prediction for errors, omissions, or unsupported conclusions.Upstream prompt and model choices can affect all later stages, making reflector capability relevant to error detection.Source: §D.1 · p. 17

Decentralized topology

  • Decentralized execution has three logical agents interact over two rounds, with each revisiting the task after seeing peers’ outputs.The repeated runtime instances share three logical prompt and model assignments; a mechanical voter combines final outputs and is excluded from deployment-time LLM cost.Source: §D.1 · p. 17

Dynamic topology

  • Dynamic execution selects among dataset-specific roles using the task, accumulated outputs, role descriptions, and execution history.The path can repeat or skip roles, has maximum depth five and a forced final-answer slot, and uses a fixed Qwen3.5-9B selector; the final role follows each dataset’s output contract.Source: §D.1 · pp. 17–18

Full results

  • Table 9 compares UniMASO’s selected heterogeneous configuration with five homogeneous configurations for each topology, reporting task quality and deployment cost.Source: Table 9 · p. 18

E METRICS AND COST ACCOUNTING

The section describes how task quality and token-weighted deployment cost are evaluated, including benchmark-specific metrics, an aggregate score, and fixed relative model-cost weights for reproducible ranking. It also outlines the focal-agent, intermediate, and final-answer evaluation levels used to guide topology- and role-specific prompt and model-assignment optimization.

E METRICS AND COST ACCOUNTING

  • Task performance uses accuracy for AGIEval-MATH, GPQA-Diamond, and BFCL-V3, Pass@1 for LiveCodeBench, and exact match for HotpotQA; the aggregate is their unweighted mean.Source: E METRICS AND COST ACCOUNTING · p. 18
  • Deployment cost is the token-weighted inference cost of task agents during final execution, excluding optimization overhead, evaluator calls, and Co-Evolver calls.Source: E METRICS AND COST ACCOUNTING · p. 18

E.1 RELATIVE COST MODEL

  • Model-cost coefficients are fixed relative token-price weights referenced to Qwen AI model prices, intended for reproducible cost ranking rather than exact provider charges.The weights enter cost-aware candidate selection through λ and do not alter task-quality metrics.Source: §E.1 · p. 19

Optimizer Prompt (Question Answering)

  • The question-answering optimizer prompt requires strict XML output containing an analysis, a model decision, a model name, and a complete optimized prompt that preserves placeholders.It specifically preserves {{question}} and {{context}}.Source: Optimizer Prompt (Question Answering) · p. 28

F.2.2 EVALUATOR PROMPTS

  • The evaluator compares focal-agent outputs, successor-consumed intermediate outputs, and final system answers using a binary protocol with task-specific criteria.Variants can assess function-schema validity, code correctness, or evidence grounding.Source: §F.2.2 · p. 28

Evaluator Prompt (Task-Output Comparison)

  • Task-output comparisons judge two responses against the agent’s role and requirements, prioritizing correctness and task-specific validity over length or confidence.Guidelines cover answer markers, code correctness and edge cases, QA evidence grounding, and function-call structure.Source: Evaluator Prompt (Task-Output… · pp. 28–29

Evaluator Prompt (Intermediate Comparison)

  • Intermediate comparisons prefer complete, accurate, clear outputs that give downstream agents enough detail to verify or build on them.The evaluator returns only A or B according to which output is more conducive to the correct final answer.Source: Evaluator Prompt (Intermediate… · p. 29

Evaluator Prompt (Final Answer Comparison)

  • Final-answer comparisons prioritize likely correctness over length, detail, or formatting, and choose between outputs using only A or B.When both answers seem plausible, the prompt favors clearer, error-free reasoning.Source: Evaluator Prompt (Final Answer… · pp. 29–30

F.2.3 CO-EVOLUTION PROMPTS

  • At each optimization round’s end, the Co-Evolver returns a final prompt and feasible model assignment, which the implementation merges subject to topology constraints.The static version receives the complete execution graph; the dynamic version receives a role pool and excludes the selector from model assignment.Source: §F.2.3 · p. 30

Co-Evolver Prompt (Static Topology)

  • Static-topology co-evolution uses topology, roles, dependencies, difficulty, capability, and input/output cost to produce a role-preserving prompt and assignments for all optimizable logical agents.It excludes mechanical nodes and is instructed not to use historical candidate prompts or trajectory scores to determine model assignments.Source: Co-Evolver Prompt (Static Topology) · pp. 30–31

Co-Evolver Prompt (Dynamic Topology)

  • Dynamic-topology co-evolution assigns models across five roles using role responsibility, route frequency, critical-path impact, task difficulty, capability, and cost, while excluding the Selector.It uses dataset, role-pool, selector-history, and route information, and preserves placeholders, delimiters, and required output structures in the role prompt.Source: Co-Evolver Prompt (Dynamic Topology) · p. 31

Related papers

Loading 2610.31743v2…