PaperlayerView exact revision on arXiv

Source-linked AI summary

Heterogeneous Agentic Coevolution

Wenqian Ye, Ziwei Guan, Zi Yang, Shivani Modi, Cici Cong Wu, Yunlong He, Yuxin Chen, Zhengyu Zhang, Liang Luo, Venkatesh Ranganathan, Jianhui Sun, Yantao Yao, Wenlin Chen, Buyun Zhang, Ellie Dingqiao Wen, Henry Kautz, Aidong Zhang

arXiv:2610.16078v1cs.AI

TL;DR

Specialized foundation models can plateau under supervised training while further improvements still depend on human design, and agent strategies must adapt as specialists advance. HACE coevolves specialists and agents through reciprocal learning and an evolving harness; across three domains, it outperforms evaluated self-improving methods.

  • Problem

    Specialized foundation models have domain expertise but may depend on human-designed improvements after supervised training converges, while the agent strategies used to advance them must adapt as they improve.

    Source: §1 · p. 2

  • Method

    HACE coevolves a specialist and an LLM agent through verified reciprocal learning, with agent exploration guiding specialist training and an evolving harness coordinating their interaction.

    Source: §1 · pp. 2–3

  • Results

    Across recommendation, protein function prediction, and material discovery, HACE outperforms evaluated evolution methods and achieves state-of-the-art performance beyond supervised convergence.

    Source: §1 · p. 3

  • Takeaways & Limitations

    Experiments across three domains and different architectures show heterogeneous models supporting each other’s development beyond initially converged specialists.

    Source: §5 · p. 10

    HACE studies one specialist with one agent, and findings are limited to the evaluated domains, architectures, and computational budgets.

    Source: A Limitations · p. 13

Abstract

from arXiv · show

Recent advances in self-improving LLM agents have demonstrated the potential of AI systems that drive their own intelligence. However, many domain-specific applications rely on specialized foundation models that possess strong domain expertise but remain dependent on human-designed improvements. Extending recursive self-improvement to these models presents a key challenge: as a specialist improves, the strategies used to advance it must also adapt. To address this challenge, we propose Heterogeneous Agentic CoEvolution (HACE), a framework that jointly evolves specialized foundation models and LLM agents through reciprocal learning. HACE improves the specialist through supervised learning while training the agent with on-policy reinforced self-training, guided by verified performance gains on the current specialist. The agent's exploration, in turn, informs subsequent specialist training, enabling both models to adapt to each other's progress. To sustain this reciprocal learning, HACE further employs evolutionary search to optimize their interaction harness, adapting how the models exchange information and select training experiences as their capabilities evolve. We evaluate HACE across three challenging domains: recommendation systems, protein function prediction in biology, and crystal stability prediction in material discovery. We show that HACE outperforms the evaluated self-improving methods on the primary task metrics across all three domains. These results shed light on heterogeneous coevolution as a universal approach to improving specialized foundation models through reciprocal interaction with a learning agent.

1 Introduction

HACE addresses the dependence of specialized foundation models on human-designed improvements by coevolving a specialist and an LLM agent through reciprocal learning. The introduction reports gains beyond supervised convergence across three domains.

1 Introduction

  • Specialized models acquire domain expertise but remain reliant on researchers to identify improvements after supervised training converges.Meanwhile, an agent’s interventions may become redundant as the specialist’s weaknesses are resolved, making continued agent learning important.Source: §1 · p. 2
  • HACE trains the specialist through supervision and the agent through on-policy reinforced self-training, with verified gains guiding agent learning and agent exploration guiding specialist experience selection.Each model’s progress changes which contributions or learning opportunities are useful to the other.Source: §1 · pp. 2–3
  • An evolutionary interaction harness adapts information exchange and training-experience selection as the models’ capabilities change.Harness search evaluates changes through their effects on task performance and specialist learning.Source: §1 · p. 2
  • The evaluation tests whether an evolving agent can create learning opportunities beyond the plateau of already-converged specialists across diverse architectures and large-scale datasets.The domains are recommendation systems, protein function prediction, and crystal stability prediction.Source: §1 · p. 2
  • Across recommendation, protein function prediction, and material discovery, HACE achieves state-of-the-art performance and outperforms all evaluated evolution methods.The evaluation starts with specialists already at supervised convergence, where continued training under the original procedure yields no further improvement.Source: §1 · p. 3

2 Related Work

Related work covers agent and harness optimization, coevolution, and agent distillation. HACE connects interaction-harness search with parameter learning in both a specialized model and an LLM agent.

Agent and harness optimization

  • Agent and harness optimization uses execution feedback (Hu et al., 2025), while reinforced self-training learns from task-reward-selected responses (Singh et al., 2024).HACE connects harness search with parameter learning in both the specialist and the agent.Source: §2 · p. 3

Coevolution and reciprocal learning

  • Prior coevolution work links learners to evolving rewards, environments, or procedures (Ma et al., 2024; Dennis et al., 2020); HACE studies reciprocal dependence between differently capable models.Agent exploration changes specialist training instances, while specialist predictions affect which agent responses pass verification.Source: §2 · p. 3

Agent distillation

  • Unlike distillation methods that keep teacher parameters fixed (Wang et al., 2026), HACE lets agent exploration guide specialist instance selection and verified contributions guide agent learning.Neither model has a fixed teacher role, and the agent can improve the specialist despite lower overall domain performance.Source: §2 · p. 3

3 Heterogeneous Agentic Coevolution

HACE couples specialist adaptation, agent learning, and harness evolution in a reciprocal loop. Its learning guarantee relates final specialist risk to accepted-experience coverage and loss.

3 Heterogeneous Agentic Coevolution

  • HACE uses an evolving harness to connect specialist adaptation, joint selection, and agent learning, so each learner shapes the other’s training experience.The harness governs information exchange and which experiences train each model.Source: §3 · p. 3, Figure 2 · p. 4

3.1 Reciprocal Learning

  • The agent explores using specialist predictions without seeing labels, and verified responses guide selection of labeled instances for specialist training with the original task loss.The harness controls the agent’s view and curriculum; changing the view can expose opportunities missed by further sampling under the same harness.Source: §3.1 · p. 4

3.2 Joint Selection

  • Joint selection evaluates the adapted specialist and proposed harness together, retaining the pair only when it improves joint predictions on validation data.The retained pair then defines the context for collecting fresh responses that improve on the current specialist.Source: §3.2 · p. 5
  • Fresh selected responses train the agent with on-policy reinforced self-training; if the dataset is empty, the agent remains unchanged.The contribution scores select responses but do not weight the fitting loss, and the dataset stays fixed during fitting.Source: §3.2 · p. 5

3.3 Learning Guarantee

  • The guarantee separates accepted exposure from the final specialist’s loss on accepted experience; sufficient source coverage and low relative loss yield a small risk bound.Accepted exposure depends on harness search, agent exploration, and joint selection; specialist optimization determines loss and retention of learning.Source: §3.3 · pp. 5–6
  • With probability at least 1−δ over the learning process, every final specialist S_T∈F has R_µ(S_T)→0 whenever v_T>0, v_T→∞, U_T(v_T)→0, and bL_T(S_T)/v_T→0.Source: §3.3 · p. 6

4 Experiments

Across recommendation, protein function prediction, and crystal stability prediction, HACE is evaluated against other self-improving methods using specialists trained to convergence. It leads on the reported primary comparisons, while ablations, scaling experiments, and evolution examples examine its components and behavior.

4 Experiments

  • Evaluation starts from specialists trained to convergence across recommendation, CAFA protein prediction, and WBM crystal stability, and compares HACE with other self-improving methods.Source: §4 · p. 6

4.1 Main Comparison

  • 12.1% average relative HR@10 gain over Base across 15 recommendation settings; HACE leads every setting.Source: Table 1 · p. 7, §4.1 · p. 7
  • On CAFA, HACE gains 0.0460 (ESM-2) and 0.0498 (ProtT5-XL) versus existing-method maxima of 0.0040 and 0.0032; WBM reaches 0.6541 NDCG.Source: Table 2 · p. 7, §4.1 · pp. 7–8

4.2 Ablation Studies

  • Removing agent optimization causes the largest Toys drop, from 0.0706 to 0.0622; removing harness or specialist optimization cuts CAFA NDCG by 0.0438 or 0.0435.The ablations report task-dependent contributions, with agent optimization mattering most on Toys and specialist optimization on WBM.Source: Table 3 · p. 8, §4.2 · p. 8

4.3 Agent and Model Scaling

  • Across ESM-2 sizes from 35M to 3B, HACE improves prediction; E4B's mean gain is 0.0460 versus 0.0012 for E2B and 0.0054 for 26B-A4B, not monotonically with agent size.Source: Table 4 · p. 8, §4.3 · pp. 8–9

4.4 Evolution Dynamics

  • HACE reaches the highest score in all three recorded trajectories; recommendation peaks mostly early, while protein and materials set later maxima.Figure 3 reports best recorded HR@10, Recall@10, and mean NDCG@3 and NDCG@5, with dashed lines marking the frozen specialist.Source: Figure 3 · p. 9, §4.4 · p. 9
  • Figure 4 examples pair prediction changes with harness edits: a matching product variant, protein terms in the top five, and a stable phase in the top three.Recorded edits include removing genre metadata, targeting difficult training instances, and hiding candidate ranks from the agent.Source: Figure 4 · p. 9, §4.4 · p. 10

5 Conclusion

Across three domains, HACE improves beyond initially converged specialists across different architectures. The conclusion presents heterogeneous models as supporting each other’s development and identifies directions for future study.

5 Conclusion

  • HACE evolves specialized foundation models, LLM agents, and their interaction through an updating harness.The models become sources of learning for one another.Source: §5 · p. 10
  • Experiments in three domains demonstrate improvements beyond initially converged specialists across different architectures.The conclusion frames these results as evidence that heterogeneous AI models can support each other’s development.Source: §5 · p. 10
  • Future work can study sustained coevolution, collaboration among multiple specialists, and autonomous model development for scientific discovery.Source: §5 · p. 10

A Limitations

The evaluation is bounded by its domains, architectures, computational budgets, and single-specialist, single-agent setup. The appendix also specifies mathematical notation conventions.

A Limitations

  • HACE coevolves one specialist with one LLM agent; the findings are limited to the evaluated domains, architectures, and computational budgets.The authors identify sustained coevolution and computation allocation as topics for further study.Source: A Limitations · p. 13
  • Compared methods update different components, so the comparison follows each method’s original formulation and evaluates complete systems.Table 3 separately examines the contribution of each HACE component.Source: A Limitations · p. 13

B Proofs and Implementation Details

  • The appendix uses log for the natural logarithm, |B| for set elements or dataset records, |z| for tokens, and 1{B} for an indicator.Empty sums are zero and empty products are one.Source: B Proofs and Implementation Details · p. 13

B.1 Specialist Learning

The specialist-learning analysis bounds source risk through accepted exposure and retained loss, with population consistency requiring additional sampling conditions. Its guarantees depend on instance-level exposure and on preserving learning through later updates.

B.1 Specialist Learning

  • Specialist updates use selected labeled training indices; replay adds retained-instance losses with fixed nonnegative weights, and zero weights disable replay.Source: §B.1 · p. 13
  • An exponential supermartingale and a stopping-time argument yield a bound uniform over rounds; the proof does not require independence across rounds.A union bound extends the threshold guarantee across the fixed predictor class.Source: §B.1 · p. 14
  • The task-risk bound applies where accepted exposure is sufficient and depends on exposure, retained loss, and the size of the specialist class.The analysis gives a high-probability guarantee across predictors and rounds.Source: §B.1 · p. 14
  • Risk tends to zero when exposure-related terms and final specialist loss relative to exposure vanish; for a fixed finite class, this requires zero minimum class risk.Source: §B.1 · p. 14
  • High overall acceptance does not ensure every instance is exposed, and later updates can forget instances despite small immediate losses.If acceptance has conditional probability zero in every round, the bound gives no learning conclusion.Source: §B.1 · p. 15
  • The population extension assumes independent source-pool draws and a class fixed independently of those draws; repeated training on a fixed pool does not remove the final sample term.Population consistency requires that term to vanish, for example through increasing independent source data while controlling class size.Source: §B.1 · p. 15

B.2 Agent Learning and Verification

The section formalizes how agent responses are sampled, verified, and used for learning, then relates fitting to verification under fixed or changing references. It also cautions that lower training loss alone does not guarantee improved response likelihood for each instance.

B.2 Agent Learning and Verification

  • Prompts combine each input with specialist predictions, exclude its label, and generate m conditionally independent responses for collection.Verification accepts responses that are valid and have positive reward; instances without evaluable supervision have no verified responses.Source: §B.2 · p. 16
  • With m conditionally independent responses, collection probability is 1−(1−αt(i))m; averaging response probability before collection can overestimate verification when m > 1.Equality holds for m = 1; with shared random context, independence must hold conditional on that context.Source: §B.2 · p. 16
  • Under a fixed reference, decoded likelihood gain gt greater than selection divergence kt is sufficient for improved verification; after reference changes, ηt < αt(egt−kt −1) is sufficient for increased verification.The bound compares the fitted agent against the old reference and accounts separately for changes in the next specialist and harness.Source: §B.2 · pp. 16–17
  • The agent trains on selected prompt-response records with a token-averaged loss, while contribution scores select records without weighting their losses.The loss includes prompt and response tokens, and the dataset remains fixed during fitting.Source: §B.2 · pp. 17–18
  • Lower total loss does not establish improved response likelihood for each instance, and decoded likelihood gains can differ from raw-token gains under transformed decoding.Temperature scaling, probability truncation, or omitting a stopping-token factor can change the correspondence.Source: §B.2 · p. 18
  • Increased verification alone does not establish the exposure or retained-loss conditions required by Theorem 1.The curriculum selects specialist training instances, while pair acceptance determines their contribution to exposure weights.Source: §B.2 · p. 18

B.3 Task and Implementation Details

This section specifies how candidate views, response verification, and joint predictions are evaluated, then gives the round procedure for proposing and accepting specialist-harness updates and training the agent.

B.3 Task and Implementation Details

  • The harness exposes a view from a fixed reference candidate pool, and responses are parsed into candidate rankings completed with omitted candidates in specialist order.Responses without valid candidate indices receive zero quality; verification compares response and specialist NDCG at the harness cutoff.Source: §B.3 · p. 19
  • When relevant candidates are available, the response score is bounded by the best ordering of the exposed view; changing the harness can change that view.If the specialist already reaches this ceiling, further response sampling in that view cannot yield a positive contribution.Source: §B.3 · p. 19
  • Joint predictions rank candidates by specialist score plus λ times agent score, with zero λ recovering the specialist ordering within the same view.Each state fits λ separately over a fixed grid using two validation folds and evaluates on the opposite fold.Source: §B.3 · p. 19
  • Each round mutates an archived high-scoring harness, selects specialist-training instances with a curriculum, and accepts the proposed specialist-harness pair only for a strictly positive validation improvement.A rejected proposal restores the specialist parameters, optimizer, calibration, and harness.Source: §B.3 · pp. 19–20
  • Agent learning uses fresh responses under the retained specialist-harness pair, keeps the highest positive-contribution original response per eligible instance, and leaves the agent unchanged if the dataset is empty.Labels are excluded from prompts and test data are reserved for final evaluation.Source: §B.3 · p. 20

C.1 Evaluation Details

Evaluation uses disjoint training, validation, and test sets, with validation guiding selection during coevolution and test data reserved for final evaluation. Metrics vary by domain and ranking task.

C.1 Evaluation Details

  • Training, validation, and test sets are disjoint; validation guides selection throughout coevolution, while test data do not inform training, calibration, or selection.Specialist-harness comparisons use the same validation batch and two folds, with internal calibration on a separate validation batch.Source: §C.1 · p. 20
  • The recommendation split holds out each test user's last interaction, while protein instances follow a CAFA temporal split and material structures are partitioned by chemical system.Protein inputs are restricted to information available at t0, and structures from one chemical system remain together.Source: §C.1 · p. 20
  • Recommendation uses HR@10; CAFA averages NDCG@5 and NDCG@10, while WBM averages NDCG@3 and NDCG@5.NDCG rewards recovering relevant candidates and placing them earlier within the reference pool.Source: §C.1 · p. 20
  • Protein evaluation includes precision, recall, Recall@10, Fmax, AUPR, and Smin, while materials evaluation includes F1, DAF, MAE, and R2.Lower values are better for Smin and MAE; higher values are better for the other listed materials metrics.Source: §C.1 · pp. 20–21

C.2 Complete Scientific Results

The supplied results passages identify the scopes of the CAFA protein-function and WBM materials tables and provide CAFA values without metric or method labels.

C.2 Complete Scientific Results

  • Table 5 covers protein function prediction on CAFA with two frozen backbones.The supplied passage does not provide its row or column labels.Source: Table 5 · p. 21
  • The CAFA table passage includes the values 0.7179±0.0014, 0.7180±0.0012, 0.7245±0.0007, 0.7222±0.0016, 0.7261±0.0016, 0.7185±0.0013, and 0.7615.The supplied passage does not identify the metric or method associated with each value.Source: §C.2 · p. 21
  • The CAFA table passage also includes the values 0.7122±0.0015, 0.7131±0.0016, 0.7156±0.0020, 0.7139±0.0015, 0.7175±0.0016, 0.7135±0.0016, and 0.7553.The supplied passage does not identify the metric or method associated with each value.Source: §C.2 · p. 21
  • Table 6 covers the material discovery task on WBM with CHGNet.The supplied passage does not include the table values.Source: Table 6 · p. 21

C.3 More Experimental Details

The appendix describes convergence checks and experimental settings, then details how the recommendation, protein, and crystal harnesses present candidates to the agent. Across domains, prompts combine task-specific information with specialist predictions, while harness search controls selected presentation choices.

C.3 More Experimental Details

  • The experiments vary specialist and agent sizes in separate sweeps and use three random seeds; ESM-2 representations stay frozen in the agent sweep while its prediction head is trained.The agent variants share a 30-round configuration with ESM-2 650M, and the reported HACE means for ESM-2 35M, 150M, and 3B add the Base means and gains.Source: §C.3 · p. 22
  • The specialists were treated as converged when three additional epochs changed the reported metric by less than 0.001 in every domain.The experiments test whether interaction with an evolving agent can create improvement opportunities beyond that plateau; ablations identify Equation 4’s acceptance gate as making continued training productive.Source: §C.3 · p. 22
  • Figure 4 records selection-score trajectories for one run per domain, with available per-round scores and their running maxima shown over 30 recommendation rounds and 50 protein and materials rounds.The CAFA record contains 18 rounds; these trajectories show observed selection scores, whereas the main tables report aggregate evaluation results.Source: §C.3 · p. 22
  • Specialists train on task labels, while agent learning uses fresh responses; uniform and uncertainty curricula and budget completion may select instances from either row.The exploration table describes relative improvement as determining eligibility for different learning objectives.Source: §C.3 · p. 23, Table 8 · p. 23

D.1 Prompts for Recommendation

  • The recommendation harness combines user interaction history with candidate rankings, while search varies history length, item metadata, and the split between leading predictions and alternatives.A common item terminology is used across MovieLens 1M and four Amazon datasets; prompts ask the agent to select alternatives or rank them, with a retention option for k = 1 under the stated condition.Source: §D.1 · p. 23

D.2 Prompts for Protein Function Prediction

  • The protein prompt combines protein metadata and available annotations with Gene Ontology candidates and specialist rankings; harness search varies annotation count, term-definition length, and shown metadata.The agent returns candidate IDs in the same ranking format as recommendation.Source: §D.2 · p. 24

D.3 Prompts for Material Discovery

  • The crystal prompt presents candidate compositions and available structure prototypes alongside specialist predictions; harness search varies prototype length and whether element information and specialist ranks appear.Candidate IDs identify agent selections independently of specialist ranks.Source: §D.3 · p. 25

Related papers

Loading 2610.16078v1…