PaperlayerView exact revision on arXiv

Source-linked AI summary

RankEvolve: A Reliable Multi-Agent Auto-Research Harness for Evolving Ranking Models

Zheng Chen, Linfeng Liu, Hong Li, Hong Yan

arXiv:2609.39551v1cs.AI

TL;DR

Long-horizon applied-ML research faces costly invalid runs and fragile execution, while the execution-accuracy benefit of composing coding-agent products at matched budget is not established. RankEvolve combines runtime-enforced workflow protocols, product-level agent composition, and persistent findings. It reports higher execution accuracy than the strongest matched-budget single-product baseline, with a transfer result on LitGPT.

  • Problem

    Whether composing complete coding-agent products improves execution accuracy beyond spending the same budget on one product remains unestablished, despite costly runs being vulnerable to subtle defects.

    Source: §1 · p. 2

  • Method

    RankEvolve compiles workflow protocols into runtime-enforced state machines and composes complete coding-agent products as execution-graph nodes.

    Source: §1 · p. 2, §2.1 · p. 3

  • Results

    62.5% versus 45.8% execution accuracy at matched budget, with a paired +16.7 points (95% CI [6.6, 26.7]); the effect also replicates on LitGPT at +12.5 points (95% CI [3.0, 22.0]).

    Source: §6 · p. 11

  • Takeaways & Limitations

    The reported evidence supports product-level composition as a measured execution-accuracy improvement on HSTU and LitGPT, while the specific topology contrast is not distinguishable.

    Source: §6 · p. 11

    Evidence covers one ranking codebase and LitGPT, both Python/PyTorch, so cross-domain transfer is not fully established; incomplete oracles and unmatchable provider-side FLOPs also constrain interpretation.

    Source: §6 · p. 11

Abstract

from arXiv · show

Auto-research agents, LLM systems that propose, implement, train, and evaluate model changes across iterations, promise to automate applied ML's experimental loop. Over long horizons, execution accuracy is a binding constraint: a change can silently leak held-out data, omit normalization, disconnect a gradient, or leave a train/eval flag unwired, invalidating expensive runs and compounding error across iterations. We present RankEvolve, an auto-research framework for evolving generative ranking models. An Executable Operating Protocol (EOP) declares phases, gates, branches, and loops, and the runtime enforces the compiled state machine. A meta-meta-harness composes complete black-box coding-agent products, including Claude Code and Codex, as execution-graph nodes that review and repair one another's work. In a budget-matched evaluation, heterogeneous composition raises all-oracle execution accuracy from the best single-product baseline of 45.8 percent to 62.5 percent (paired +16.7 points, 95 percent CI [6.6, 26.7]) while achieving a 10.4 percent silent critical-defect rate. An implemented knowledge layer carries findings, including negative results, across iterations. In a twelve-iteration deployment on the open-source HSTU recommender, RankEvolve reported NDCG@10 of 0.2192 on MovieLens-20M LARGE (+4.48 percent over the published anchor) and 0.1948 on BASE (+2.80 percent). ExecML-HSTU, seeded by incidents from that deployment, provides the oracle benchmark for the execution-accuracy evaluation. A pre-specified LitGPT transfer split replicates the heterogeneous-composition effect beyond recommendation (+12.5 points, 95 percent CI [3.0, 22.0]), and a paired ablation isolates per-step from full-protocol instruction injection. These results characterize when runtime-controlled composition of coding-agent products improves execution accuracy.

1 Introduction

Long-horizon model evolution is costly and fragile, while existing work does not establish whether composing complete coding-agent products improves execution accuracy at matched budget. RankEvolve combines runtime-enforced protocols with product-level agent composition and a knowledge layer, and reports higher execution accuracy than its strongest matched-budget single-product baseline.

Motivation

  • Long-horizon evolution is costly and fragile: expensive training runs can be invalidated by subtle defects, while accumulated context erodes process adherence.The unresolved question is whether product composition buys execution accuracy beyond spending the same budget on one product.Source: §1 · p. 2

Contributions

  • On the incident-seeded oracle benchmark, RankEvolve’s product composition raises execution accuracy from the best single-product baseline’s 45.8% to 62.5% at matched budget, establishing execution accuracy as a binding constraint in this setting.Source: §1 · p. 2
  • RankEvolve compiles versioned workflow phases, dependencies, gates, branches, and loops into a runtime-enforced state machine.Its meta-meta-harness binds complete coding-agent products into review–repair and plan–merge flows.Source: §1 · p. 2
  • A provenance-bearing knowledge layer carries findings, especially negative results, into later iterations.This is intended to avoid silently re-deriving prior dead ends.Source: §1 · p. 2

2 Anatomy of RankEvolve

RankEvolve combines a compiled, runtime-enforced workflow protocol with a harness that composes complete coding-agent products and a persistent knowledge layer. Its architecture separates long-horizon phase control from per-phase execution depth and diversity.

2.1 Executable Operating Protocols

  • An EOP declares workflow phases, dependencies, tools, gates, branches, and loops, then compiles them into runtime-enforced states and transitions.The historical ten-run playbook pilot found no fully faithful run, but lacked a matched runtime-controlled arm and changed control and context delivery together.Source: §2.1 · pp. 3–4

2.2 A meta-meta-harness

  • The meta-meta-harness binds complete coding-agent products as black-box work nodes, while deterministic runtime control handles routing, aggregation, and gates.Its deployed flow combines parallel plan-and-merge with dual implement–review; these patterns compose but do not imply automatic workflow search.Source: §2.2 · pp. 4–5

2 Anatomy of RankEvolve

  • The EOP fixes the workflow’s horizontal order, while the harness supplies each phase’s execution depth and diversity.The paper tests whether heterogeneous cross-checking improves correctness at equal budget rather than assuming it does.Source: §2.2 · p. 5

2.3 The knowledge layer: artifact and evidence state

  • The knowledge layer records artifacts, provenance, and negative outcomes, and carries code-linked lessons into later phases of the twelve-iteration HSTU deployment.It was active in the deployment, but the study does not isolate its marginal effect on proposal quality with a memory-on/off ablation.Source: §2.3 · p. 5

3 HSTU Deployment Case Study

The HSTU case study evaluates RankEvolve on public MovieLens benchmarks using canonical full-test NDCG@10 and reports a three-stage trajectory. The authors also replayed two exported recipes on four additional sequential-recommendation datasets without per-dataset retuning.

3.1 Scope, target, and evaluation

  • Evaluation reports canonical full-test NDCG@10 using leave-one-out full-corpus ranking over held-out users, distinct from faster subset evaluation that ran about 0.005–0.010 higher.Checkpoint values from one run are descriptive history, not independent seeds; MovieLens-32M has no published HSTU anchor.Source: §3.1 · pp. 5–6, §3.3 · p. 6

3.2 Three-stage trajectory

  • Stage III’s genre side feature reached the largest endpoint, 0.2192; Stage I composed known architectural components to 0.2161, while Stage II refinements were mostly within noise.Preference optimization produced the largest regression; these findings illustrate the workload and failure modes, not architectural novelty.Source: §3.2 · p. 6

3.3 Reported endpoints and integrity boundary

  • Both exported recipes improved over vanilla HSTU on all four additional sequential-recommendation datasets, cumulatively by up to +25%, without per-dataset retuning.On Foursquare-TKY, adding UDK changed the result from 0.0200 to 0.0202.Source: §3.3 · p. 6

4 Objective Execution-Accuracy Evaluation

The evaluation tests whether composing coding-agent products improves execution accuracy at matched observable budgets, using private HSTU and LitGPT tasks. Heterogeneous composition leads the stage-1 comparisons, while the deployed plan-merge flow and analyses of error complementarity and context scope qualify the findings.

4 Objective Execution-Accuracy Evaluation

  • The benchmark used 96 private tasks per repository and hidden oracles requiring regression, behavioral, scientific-safety, and evaluator-integrity checks to pass together.Conditions shared a frozen observable resource envelope; timeouts and overruns counted as failures.Source: §4 · pp. 6–8

4.1 Results

  • 62.5% HSTU EA for CC→Codex→CC exceeded the strongest budget-matched baseline, CC→CC→CC at 45.8%, by 16.7 points (95% CI [6.6, 26.7]).The paired comparison cleared the pre-specified threshold; silent critical-defect rate fell from 35.4% to 10.4%.Source: §4.1 · pp. 8–9, Table 4 · p. 8
  • The pre-specified LitGPT transfer contrast was +12.5 points (95% CI [3.0, 22.0]), with 56.2% EA for CC→Codex→CC versus 43.8%.The five matched-budget conditions cost $0.95–$1.02 per task; the one-pass CC reference was unmatched.Source: Table 4 · p. 8, §4.1 · p. 9

4.2 Second stage: the deployed plan-merge topology

  • The deployed plan-merge flow reached 70.8% HSTU EA, 64.6% LitGPT EA, and 6.2% CDR, but its +8.3-point paired lift over the comparator was not statistically distinguishable.At envelope B, it tied CC→Codex→CC at 62.5% EA; the deployed flow ran uncapped at $2.40 per task.Source: Table 4 · p. 8, §4.2 · p. 9

4.3 Mechanism and context scope

  • Lower error correlation aligned with greater complementary error mass and realized review gain: CC←Codex had ρ=0.21, D=0.18, and G=0.06, while CC←CC netted approximately zero.The fitted slope was 0.34 (95% CI [0.12, 0.56]); the paper frames the mechanism as complementary errors multiplied by conversion efficiency.Source: §4.3 · p. 10
  • Current-step context was never worse and its EA advantage grew from +2.1 points early to +10.4 late, while reducing wrong promotions, silent defects, and token cost.The runtime, state, history, artifacts, tools, budgets, and product were held fixed; only inactive phase instructions varied.Source: §4.3 · p. 10

5 Related Work

Related work spans stateful orchestration, graph and workflow optimization, multi-agent software engineering, automated discovery, and recommendation research agents. RankEvolve positions its focus as fixed-protocol composition of complete coding-agent products and evaluation of execution accuracy on real ML repositories.

Programmed state and orchestration

  • StateFlow (Wu et al., 2024) and LangGraph (LangChain, 2024) provide stateful orchestration; RankEvolve adds an ML-focused authoring layer, product-agent adapter, and provenance contract.The paper notes that LangGraph nodes can already wrap agents.Source: §5 · p. 10

Graph and workflow optimization

  • GPTSwarm, ADAS (Hu et al., 2025), and AFlow (Zhang et al., 2025) address graph composition or workflow search, whereas RankEvolve uses a human-authored fixed protocol.Its focus is whether externally developed products fail differently enough for fixed review–repair composition to pay off on real ML repositories.Source: §5 · p. 10

Multi-agent software engineering and evaluation

  • Software-agent and benchmark work includes AutoGen (Wu et al., 2023), MetaGPT (Hong et al., 2024), SWE-bench, and Harness-Bench (Yao et al., 2026); ExecML complements these with matched-budget semantic correctness, silent-defect, and error-complementarity measures.The paper distinguishes Harness-Bench as comparing configurations singly rather than composing them.Source: §5 · p. 10

Automated discovery and AutoML

  • RankEvolve applies the evolve-and-select principle of FunSearch, AlphaEvolve, and the AI Scientist line to training configurations and code patches scored by distributed training, rather than numeric-archive genetic search.Its HSTU history is presented as an application case, not a new recommender or search algorithm.Source: §5 · pp. 10–11

Auto-research for recommendation

  • Compared with YouTube and Meta research agents and open-setting Self-EvolveRec, RankEvolve evolves an open generative recommender at HSTU scale, composes complete agent products, and measures execution accuracy against hidden oracles at matched budget.The cited systems are Wang et al. (2026), Kumar et al. (2026), and Kim et al. (2026), respectively.Source: §5 · p. 11

6 Conclusion

RankEvolve combines protocol and runtime control, product-agent composition, and knowledge carryover; its evaluation supports the primary execution-accuracy claim and LitGPT transfer, while the specific topology contrast is inconclusive.

6 Conclusion

  • RankEvolve contributes an EOP authoring and runtime layer, a boundary for composing complete coding-agent products as graph nodes, and a knowledge layer carrying provenance, negative results, and incident lessons.The HSTU deployment grounds both the motivation and the benchmark tasks.Source: §6 · p. 11
  • 62.5% vs. 45.8% EA and paired +16.7 points (CI [6.6, 26.7]) support the primary claim; LitGPT replicates the effect at +12.5 points (CI [3.0, 22.0]).The decorrelation condition is met, but the second-stage topology contrast is not distinguishable.Source: §6 · p. 11
  • The evidence spans one ranking codebase and LitGPT, both Python/PyTorch, so cross-domain transfer is not fully established.The HSTU case study also used a human operator at protocol gates, and the knowledge layer’s marginal contribution was not isolated by an ablation.Source: §6 · p. 11

A Extended Related Work

  • Agent benchmarks increasingly use real software and ML-engineering tasks, while AutoML and architecture search automate choices within fixed spaces rather than research-level code changes.RankEvolve places its work in long-horizon evolution of an open generative recommender.Source: A Extended Related Work · p. 16
  • HSTU iterations draw on contrastive and self-supervised modeling, popularity-bias mitigation, cold-start stratification, and semantic-ID retrieval as building blocks rather than contributions.HSTU builds on sequential recommenders.Source: A Extended Related Work · p. 16

B Reproducibility

The reproducibility materials specify recurring HSTU recipe components, canonical evaluation, preserved run records, and planned artifact releases.

B Reproducibility

  • The recurring HSTU recipe includes PRISM modulation, input compression, time-decay weighting with half-lives of 30/60/180/720 days, local-ℓ2 self-supervision at t=0.05, dropout 0.1, hard negatives, and an additive genre side-feature with α ∈[0.05, 0.20].Experiments use the public HSTU codebase.Source: B Reproducibility · p. 16
  • Canonical evaluation is full-test NDCG@10 over all held-out users; subset evaluations and single-point maxima are excluded from the leaderboard.Per-run trajectories and configurations were preserved across redundant storage tiers.Source: B Reproducibility · p. 16
  • The planned code release includes the failure-overlap analyzer, leaderboard schema, boost-last-K patch, and per-run artifact records with status, configuration, checkpoint logs, and local anchor reproductions.Source: B Reproducibility · p. 16

C Runtime, Protocol, and Evaluation Details

The appendix documents runtime and evaluation artifacts, HSTU configuration caveats, and the formalization and interpretation of error complementarity.

C Runtime, Protocol, and Evaluation Details

  • The appendix collects the EOP enforcement contract, ExecML corpus composition, decorrelation derivation and per-pair results, historical faithfulness pilot, runtime patterns, transfer table, and context-scope ablation.Source: C Runtime, Protocol, and Evaluation… · p. 16
  • Table 5 locates the contribution by comparing control representation, node granularity, and objective, rather than making a feature-count claim.Source: Table 5 · p. 17
  • Table 6 records HSTU configurations; extending the baseline 200 →500 training schedule to 700 lost accuracy, while Table 7 limits runtime guarantees to admissible transitions and tested backend paths.Runtime control cannot ensure correct scientific reasoning or code.Source: Table 6 · p. 17, Table 7 · p. 18
  • Error complementarity grows linearly as binary error correlation falls at fixed marginals; raw 1 −ρ alone does not account for differing marginal failure rates and recovery opportunities.Source: C Runtime, Protocol, and Evaluation… · pp. 16–17
  • ExecML contains 96 private tasks per repository, with independently validated oracles; task, not repeated agent attempt, is the statistical unit, and Table 9 rows are descriptive with source tasks jointly resampled.Source: Table 8 · p. 18, Table 9 · p. 18

C.1 Evaluation protocol: oracle, statistics, and ablation design

The evaluation combines hidden-oracle scoring, budget-matched agent conditions, error-complementarity analysis, and a controlled prompt-scope ablation; the EOP and inferencer interfaces specify the broader execution workflow.

C.1 Evaluation protocol: oracle, statistics, and ablation design

  • A benchmark task pairs an immutable repository snapshot and change request with a hidden oracle that requires regression, behavioral, scientific-safety, and evaluator-integrity checks to pass.The primary endpoint is all-oracle execution accuracy; silent critical-defect rate captures runnable faults that can invalidate scientific conclusions.Source: §C.1 · pp. 18–19
  • The six conditions compare single-product and review–repair flows, including heterogeneous CC→Codex→CC and its reverse; the oracle-selected best-of-N is a non-deployable upper bound.The total resource envelope and observable inference budgets are matched across flows and controls, though provider-side FLOPs are unavailable.Source: §C.1 · p. 19
  • Directional complementarity measures partner-exclusive errors available for rescue; realized rescue depends on detection and successful repair, with shared-error rescues and reviewer-introduced harms reported separately.The analysis subtracts new errors introduced by review.Source: §C.1 · p. 19
  • The decorrelation mechanism is used only when β1’s interval excludes zero and held-out prediction beats a marginals-only model; otherwise complementarity, rescue, and harm are descriptive.The design estimates independent product errors before review and uses task-clustered bootstrap intervals and held-out prediction.Source: §C.1 · pp. 19–20
  • The context-scope ablation forks the same persisted runtime checkpoint into fresh sandboxes with matched graph, history, tools, budgets, gate responses, product, and active instruction; only inactive EOP content changes.Repeated active-phase content controls wording and recency, while a length-matched neutral-padding arm tests generic prompt-length effects.Source: §C.1 · p. 20, Table 11 · p. 20
  • Prompts, budgets, effect threshold, and baseline family are frozen before private evaluation; the primary test is max-over-baselines, with paired tests, clustered intervals, and predeclared failure handling.The interpretation rules specify how positive, null, negative-gain, and transfer outcomes will be reported.Source: §C.1 · p. 20

D The Model-Evolution EOP

  • The model-evolution EOP moves through target and strategy setup, code and data investigation, research proposals, implementation and experiment branches, then a user-confirmed loop or conclusion.Its phase instructions and confirmation gates are represented as workflow transitions.Source: D The Model-Evolution EOP · pp. 21–22

E Inferencer-Primitive Interfaces

  • The higher-order Dual and BTA primitives compose into a PTI flow; each combines an inferencer with retry, fallback, timeout, checkpoint, and resume support for recovery from pod eviction.Problem-specific split, worker, merge, and repair reducers are supplied at the call site, and Dual’s review–repair rounds are capped by K.Source: E Inferencer-Primitive Interfaces · p. 22

F Example Flow Configuration

The example declares a nested plan-then-implement flow: parallel peer-reviewed planning flows are merged, then the integrated plan is reviewed and refined. Node factories, parameters, and repeated blocks define its structure, with per-lane product bindings enabling heterogeneous execution.

F Example Flow Configuration

  • The deployed plan topology nests parallel peer-reviewed subtask flows inside a breakdown-and-aggregate planner, then wraps the integrated plan in a review-and-fix loop.The plan stage feeds a plan-then-implement node; implementation proceeds only after its gate passes.Source: F Example Flow Configuration · pp. 22–23
  • Node types use target and factory declarations, while cascading parameters and repeat blocks expand the configured flow into copies.The repeat block hands each copy the corresponding list entry.Source: F Example Flow Configuration · pp. 22–23
  • The defaults bind nodes to Claude Code, while the per-lane flow_inferencers list binds the two planning lanes to Claude Code and Codex in the deployment.The example parameters set two flows per subtask, a maximum of three breakdown subtasks, and three consensus iterations.Source: F Example Flow Configuration · pp. 22–23
  • Dual proposes an output, reviews it, and repairs it while severity exceeds a threshold, approval is absent, and retry rounds remain.BreakdownAggregate runs worker flows in parallel and merges their results; PlanThenImplement gates execution on the plan.Source: F Example Flow Configuration · p. 23

G Knowledge-Layer Design and Implementation

The knowledge layer links code-resident learning notes to a central wiki and an entity graph, then writes, retrieves, and reconciles those learnings across phases. The case-study findings include persistent failure overlap, a 0.2192 NDCG@10 endpoint, and a leak-free genre-feature win whose gains cannot be attributed to memory alone.

G Knowledge-Layer Design and Implementation

  • The deployment evaluated the knowledge layer end to end but had no memory-on/off ablation, so the observed model gains cannot be attributed to that layer alone.The layer was implemented and active in the reported HSTU experiments.Source: G Knowledge-Layer Design and… · pp. 24–25
  • The knowledge layer stores learnings as notes in relevant code folders, abstracts transferable findings in a wiki, and links both tiers through documents and an entity graph.Wiki entries cite local notes, and the entity graph connects recurring models, features, metrics, and defects.Source: G Knowledge-Layer Design and… · p. 25
  • A proactive subagent records modeling, execution, and operational learnings; phase-specific retrieval selects relevant notes, while a reconciliation agent resolves conflicts through the wiki.Conflicts that evidence cannot settle are flagged for human review.Source: G Knowledge-Layer Design and… · p. 25

H.1 Diagnostic-driven research: failure overlap

  • On a 0.2127 checkpoint, 75.2% of last-position failures overlapped second-last failures, and the seen-label diagnostic biases overlap downward.The overlap was interpreted as evidence of persistent user/history difficulty, not failure tied only to a held-out item.Source: §H.1 · p. 26
  • The diagnostic motivated training-side boost-last-K supervision, but those runs did not finish, so no result was reported.The intervention reweights internal positions without touching the held-out target.Source: §H.1 · p. 26

H.2 Evolution-trajectory details

  • Stage I's tuned composition of known blocks raised NDCG@10 from 0.2098 to 0.2161 (+3.0%); later refinements mostly stayed within noise, and DPO reduced NDCG@10 by 35.9% relative to its own reference.Novel methods proposed at iteration 12 did not finish within the study.Source: §H.2 · p. 27
  • 0.2192 NDCG@10 (+4.48%) was the largest numeric result, from the genre side-feature on the LARGE stack.Genre, year, and popularity then plateaued at approximately 0.219.Source: §H.2 · p. 27

H.3 The largest win: leak-free side features

  • The genre feature was fused additively on the encoder input, with supervision computed from raw item embeddings so positives and sampled negatives were treated symmetrically.The first implementation leaked genre information through the loss; the leak-free version recovered the win, with fine-tuning outperforming from-scratch training.Source: §H.3 · p. 27

H.4 Negative results

Several plausible transfers to HSTU failed or hurt, including preference optimization, focal reweighting, test-time augmentation, and longer sequences. Diagnosing these negatives surfaced mismatched assumptions and informed the loop's requirement to analyze failures before proposing again.

H.4 Negative results

  • DPO reduced NDCG@10 by 35.9% relative to a verified 0.2191 reference, as preference accuracy approached 1.0 while ranking quality fell.The rejected top-K items shared genre similarity with targets, so the objective devalued partially co-relevant candidates; a leak-free re-derivation produced approximately zero change.Source: §H.4 · p. 28
  • Focal loss on tail items yielded no gain after three bug fixes, because HSTU's sampled-softmax already balances per-item gradients.The extra focal multiplier pushed mass away from the dense head.Source: §H.4 · p. 28
  • Position-averaged test-time augmentation gave approximately 0.2190 versus 0.2192, with no shared target labels across positions.Naive multi-position evaluation instead leaked information, inflating results to 0.2517/0.2577.Source: §H.4 · p. 28
  • Extending LARGE sequence length from 500 to 700 reduced NDCG@10 while increasing wall time by 35%.The result was consistent with per-step underfitting and attention dilution.Source: §H.4 · p. 28

I Agent Behavior and Extended Discussion

  • Across these failures, the agent transferred techniques without re-deriving their assumptions, so the loop requires post-hoc diagnosis before the next proposal.That diagnosis surfaced the leakage explanations described for the failed approaches.Source: §H.4 · p. 28

I.1 Agent Behavior Analysis

The agent corrected implementation and data-leakage bugs, rejected several directions on evidence, and adjusted later impact estimates after an early reproduction gap. The discussion identifies persistent human needs and argues that diagnosis, rather than generating plausible add-ons, was the harder and more valuable step.

I.1 Agent Behavior Analysis

  • The agent fixed three focal-loss bugs, a genre-feature label leak, and a preference-pair training-overlap flaw, verifying each fix before rerunning.Genre leakage and preference-pair overlap surfaced through anomalous loss and validation-NDCG trajectories rather than failed unit tests.Source: §I.1 · p. 29
  • It rejected focal loss, all three preference-optimization variants, position-averaged test-time augmentation, and long-context based on evidence rather than quietly dropping them.Each rejection included explicit root-cause analysis.Source: §I.1 · p. 29
  • An early reproduction gap against literature-claimed +4–8% gains led the agent to discount later expected-impact estimates by ∼50%, while effort estimates stayed optimistic by ≈2–3×.Source: §I.1 · p. 29

I.2 Discussion

  • The discussion reports parallel training behind one monitor, leak-aware evaluation, and copy-on-modify with checkpoint mirroring as practices that worked.Leak-aware evaluation caught subtle data leaks; isolated code and redundant checkpoints made the run robust to pod evictions.Source: §I.2 · p. 29
  • Human steering remained necessary because the agent favored add-ons, misestimated effort and lift, and could not recognize a structurally wrong direction before spending an iteration.It produced no genuine architectural novelty in the study; every win was a composition, and the envisioned MAP-Elites/best-K archive was not implemented.Source: §I.2 · p. 29
  • The study’s novel failure-overlap→boost-last-K artifact came from analysis, and the authors would direct more effort toward diagnosis than generating plausible add-ons.They identify diagnostic steps as where an auto-research agent’s value on a hard problem is concentrated.Source: §I.2 · p. 29

Related papers

Loading 2609.39551v1…