Beneath the Diff: Diagnosing and Mitigating Algorithmic Mode Collapse in Code-Level Autonomous Research Loops

• 50–70% fewer semantic clusters emerge over 500 iterations across four NLP tasks while surface edit diversity stays essentially flat, despite edits continuing on different code lines.Surface diversity is measured by mean pairwise normalized edit distance; semantic clusters are derived from summarized edit descriptions.Source: §1 · p. 1, Figure 1 · p. 2
• By iteration 300, in-loop gains overstate held-out gains by a factor of 2.0 to 2.6 across tasks as the proposal repertoire narrows and overfits the in-loop signal.Source: §1 · p. 2
• DAPS reduces semantic-cluster decay by 69.1% and improves relative faithfulness by 81.6% averaged across tasks, while matching or slightly trailing vanilla autoresearch on in-loop optimization speed.The advantage also persists on blind sets that no loop component accesses.Source: §1 · p. 2
• DAPS combines category-coverage reweighting, persistent semantic memory against similar accepted edits, and a validation gate that reverts to the last faithful state when audit-metric divergence exceeds a calibrated threshold.The gate uses a sparingly consulted audit metric; blind sets are never accessed by any loop component.Source: §1 · p. 2
• The study operationalizes collapse with four diversity axes—surface, semantic, mechanism, and corpus similarity—and a faithfulness audit, finding that surface-only measures miss the failure across three proposer LLMs.It also reports long-horizon NLP trajectories with logged proposals, diffs, executions, and held-out evaluations.Source: §1 · p. 2

Autonomous Research Agents

• AI-Scientist, Agent Laboratory, AI-Researcher, and AI Co-Scientist orchestrate LLM research workflows; this work instead analyzes experimentation-loop dynamics (Lu et al., 2024; Schmidgall et al., 2025; Tang et al., 2026; Gottweis et al., 2025).Prior work evaluates end-to-end research quality, while this paper focuses on the experimentation loop.Source: §2 · p. 2

Diversity Collapse in Recursive Training

• Recursive-training and self-play studies report surface variation alongside underlying pattern collapse, whereas code-level ARLs use external execution signals and a distinct proposer-prior/metric-overfitting mechanism (Shumailov et al., 2024; Kovač et al., 2025; Zhu and Xie, 2026; Li et al., 2026; Mishra, 2026).The paper adopts the semantic-versus-surface framing but argues that collapse mechanisms based on model-generated data do not directly apply.Source: §2 · p. 3

Reward Hacking and Goodharting

• Algorithmic mode collapse is framed as proposal-layer Goodharting: edits that reliably improve the in-loop proxy can be selected even when the underlying system has not improved (Christiano et al., 2017; Pan et al., 2022; Gao et al., 2023).Gao et al. (2023) document that the gap between proxy and gold rewards widens with optimization pressure.Source: §2 · p. 3

Plagiarism of AI-Generated Research

• Building on evidence that AI-generated research ideas often repeat prior work without attribution, this paper asks whether proposed modifications are essentially retrieved from the proposer’s training data (Gupta and Pruthi, 2025).Source: §2 · p. 3

3.1 Code-Level ARL: Setup and Notation

• The loop accepts code edits according to changes in the optimization metric, which is the proposer’s only directly optimized signal.An ARL is represented as a pipeline, LLM proposer, executor, and acceptance rule; each iteration proposes a textual edit description and code patch, then evaluates the modified pipeline.Source: §3.1 · p. 3

3.2 Multi-Axis Diversity Instrumentation

• Four edit-level diversity measures distinguish lexical variation from changes in intent, mechanism, and similarity to prior research patterns.Surface diversity uses pairwise diff distances over the latest 20 accepted edits; semantic diversity clusters intent descriptions. Mechanism entropy uses nine frozen categories covering 96.4% of pilot edits, with labeling agreement κ = 0.84; low entropy alongside high surface diversity signals mode collapse. Corpus similarity compares descriptions with 1.1M pre-2024 arXiv abstracts and is descriptive, not proof of retrieval.Source: §3.2 · pp. 3–4

3.3 The Three-Tier Faithfulness Audit

• The protocol separates the optimized in-loop metric, an audit metric used by the gate, and a blind metric inaccessible to every loop component.The audit metric is checked every 10 iterations and can affect the trajectory through reverts and value-free notices, so it is a validation signal rather than a fully held-out test; the blind metric is evaluated once per run as the headline faithfulness evaluation. A widening faithfulness gap indicates proxy gains without commensurate downstream improvement.Source: §3.3 · p. 4

3.4 DAPS: Diversity-Aware Proposal Sampling

• DAPS combines category-coverage reweighting, persistent edit memory, and a validation gate to target algorithmic mode collapse.CCR samples and classifies N proposals, then reweights candidates based on recent mechanism-category frequencies without changing execution; PEM rejects candidates too similar to the last 200 accepted edit descriptions. The gate uses the audit signal to revert to a checkpoint within a task-calibrated faithfulness threshold and return a structured failure summary to the proposer.Source: §3.4 · p. 4

4.1 Experimental Setup

• Experiments span pretraining, instruction tuning, reasoning fine-tuning, and prompt optimization, with independent audit evaluations and blind sets that no loop component accesses.The study compares DAPS with eight baselines, reports in-loop, audited, and blind gains plus diversity and collapse diagnostics, and evaluates the main pipeline with a separate Aider implementation.Source: §4.1.1 · p. 5, §4.1.2 · p. 5, §4.1.3 · p. 6

4.2 Main Results and Analysis

• DAPS achieves the highest audited faithfulness on all four tasks while remaining within one standard deviation of the best in-loop gain; VANILLA AR’s audited faithfulness ranges from 0.38 to 0.49.R-DIVERSE-A and PRISM-A raise audited faithfulness from about 0.43 to about 0.58 averaged across tasks, but reduce in-loop gain by 3% to 13% relative to VANILLA AR on T2 and T3.Source: §4.2 · p. 6, Table 2 · p. 7

4.3 Faithfulness Under Matched Audit Access

• DAPS improves relative faithfulness over VANILLA AR by 83.7% blind and 81.6% audited, retaining a 0.74 blind faithfulness ratio versus 0.79 audited.With zero audit access, CCR+PEM surpasses both published diversity mitigations on both faithfulness tiers and cluster decay; audit access alone reaches at most 0.67 audited and 0.62 blind faithfulness, with cluster decay of at least 0.55.Source: §4.3 · p. 7

4.4 Dissecting the Collapse

• On T2, VANILLA AR’s mechanism entropy falls from 1.76 to 1.18 nats while surface diversity stays at 0.66–0.68; its faithfulness gap grows from about 0.05 to 0.17, versus about 0.07 under DAPS.Corpus similarity rises from 0.60 to 0.72 under VANILLA AR; across tasks, entropy-gap closure tracks faithfulness-gap closure, while surface diversity correlates with neither.Source: §4.4 · p. 7, Figure 2 · p. 8, Table 4 · p. 8

4.5 Ablation Study

• CCR and PEM together reduce cluster decay from 0.68 to 0.28, while HVG alone raises audited faithfulness from 0.44 to 0.67 but leaves cluster decay at 0.55.The full combination has the highest audited faithfulness, the lowest cluster decay, and in-loop gain within 2% of VANILLA AR; component effects are sub-additive.Source: §4.5 · pp. 7–8, Table 5 · p. 8

4.6 Robustness Across Proposer LLMs

• Across three proposer LLMs, VANILLA AR exhibits algorithmic mode collapse and DAPS reduces entropy collapse and the faithfulness gap; method rankings persist across proposers.Llama-3.3-70B shows the most severe baseline collapse, while Claude Opus 4.7 has the highest in-loop gain at every iteration count.Source: §4.6 · p. 8

4.7 Robustness Across ARL Frameworks

• Under Aider, VANILLA’s audited faithfulness ratio is 0.48 and mechanism entropy is 1.22 nats; DAPS-AIDER raises faithfulness to 0.76 and entropy by 0.43 nats.Aider’s surface diversity is 0.74 versus 0.66 for autoresearch, but semantic and mechanism diversity track closely; results indicate collapse and its mitigation are not framework-specific.Source: §4.7 · pp. 8–9, Table 6 · p. 8

4.8 Efficiency Analysis

• DAPS reaches 80% of VANILLA AR’s terminal gain in 163±9 iterations versus 154±8 for VANILLA AR, a difference close to seed-to-seed variance.DAPS adds approximately 1.4% to per-iteration wall-clock on T2, with amortized HVG accounting for most of the cost.Source: §4.8 · p. 9

5 Conclusion and Future Work

• Surface edit diversity remains intact as semantic and mechanism-level diversity concentrate, while in-loop gains transfer decreasingly to held-out evaluation.The paper calls this failure mode algorithmic mode collapse.Source: §5 · p. 9
• A four-axis diagnostic makes algorithmic mode collapse measurable, and DAPS mitigates it without harming optimization speed.Under a three-tier protocol separating the in-loop metric, the gate's audit signal, and a blind evaluation inaccessible to loop components, DAPS's advantage persists on data the loop never influenced.Source: §5 · p. 9
• Future work could examine collapse under frontier-scale executors and longer horizons, extend diagnosis beyond the preliminary non-ML loop, and study multi-agent loops.Source: §5 · p. 9

Limitations

• The study is limited to scales accessible on one A100; a 1,000-iteration run shows collapse deepening, while behavior at frontier models or 10k+ iterations remains open.The headline claims concern code-level ARLs editing ML pipelines; the performance-engineering loop is only a preliminary probe, and the ML-centric taxonomy needs re-derivation for other domains, although the derivation protocol is domain-general.Source: Limitations · p. 10
• The HVG audit metric is a validation signal, not an untouched final test, and even independent blind benchmarks may share idiosyncratic biases with the in-loop metric.This is why the paper reports blind evaluations as headline numbers.Source: Limitations · p. 10
• Corpus similarity VT is descriptive, agrees only moderately with human novelty judgments, and should not be treated as direct evidence of retrieval from pretraining.Source: Limitations · p. 10

Ethical Considerations

• The authors caution that agents that seem to improve while overfitting their evaluations pose a misinformation risk in science, and frame the diagnostic instrument and DAPS as tools for transparency about generalization.They release training-pipeline edits and analysis scripts, use public benchmarks under their licenses, and report that volunteer annotators consented and labeled code-edit descriptions containing no personal data.Source: Ethical Considerations · p. 10

A A Conceptual Model of Algorithmic Mode Collapse

• A peaked proposer prior, category-dependent acceptance, and self-imitation can concentrate proposals on a narrow set of mechanism categories, regardless of whether those categories contain the largest true improvements.Pretraining and instruction data may favor familiar edits, while the acceptance rule favors lower-variance edits; conditioning on accepted edits then reinforces those preferences.Source: A A Conceptual Model of Algorithmic… · p. 12
• This concentration can let the in-loop metric keep rising through finer edits while audit and blind gains depend on whether favored categories contain genuine capability improvements, widening the faithfulness gap.The dynamics do not select for capability-affecting edits.Source: A A Conceptual Model of Algorithmic… · p. 12
• The model predicts that surface edit diversity may remain largely orthogonal to collapse, broader categorical priors delay and soften collapse, and higher sampling temperature will not arrest it.Temperature redistributes probability within categories but does not change the proposer’s category distribution.Source: A A Conceptual Model of Algorithmic… · p. 13
• Memory-based deduplication should modestly reduce semantic-cluster cycling, while category-coverage incentives should more strongly reduce collapse by reweighting category probabilities.The model predicts that held-out audits can undo edits exploiting the gap between in-loop and audited metrics, which is invisible to the loop itself.Source: A A Conceptual Model of Algorithmic… · p. 13
• The four diagnostic axes operationalize the predictions, and PEM, CCR, and HVG correspond respectively to deduplication, category-coverage incentives, and held-out validation.The model predicts no temperature intervention because raising temperature should not help; it also predicts deeper rather than self-correcting collapse over longer runs, as observed in Appendix L.Source: A A Conceptual Model of Algorithmic… · pp. 12–13

B Efficiency: Per-Component Overhead and Cross-Task Convergence

• On T2, HVG dominates DAPS overhead at approximately 3.5 seconds per iteration, while CCR and PEM are negligible.HVG cost amortizes 35-second audit evaluations every K=10 iterations.Source: B Efficiency: Per-Component Overhead… · p. 13, Table 7 · p. 13
• The single blind evaluation adds 35 seconds per run, or 0.02% of a 300-iteration budget; total overhead on T1, T3, and T4 is 1.4%, 1.2%, and 0.7%, respectively.T1’s validation perplexity evaluation over a 0.5B-token shard takes the longest.Source: B Efficiency: Per-Component Overhead… · p. 13
• DAPS and VANILLA AR are statistically indistinguishable through approximately 150 iterations and remain within one standard deviation at T=300 on T1, T3, and T4.Iterations to reach 80% of VANILLA AR’s terminal gain were 187±14 vs 192±16 on T1, 108±9 vs 109±10 on T3, and 96±7 vs 98±8 on T4.Source: B Efficiency: Per-Component Overhead… · p. 13

C Hyperparameter Sensitivity

• At τm=0.95, PEM rejects almost no candidates and audited faithfulness collapses toward VANILLA AR’s 0.46; at τm=0.75, excessive rejection starves the proposer and damages in-loop gain.M is forgiving in the central range, but M=50 allows semantic cycling within a W=20 window, while M=500 over-suppresses legitimate revisits.Source: C Hyperparameter Sensitivity · p. 13
• τc is flat in the tested range; aggressive auditing (τh=70th, K=5) slightly raises the audited metric at an in-loop-gain cost, while loose auditing (τh=90th, K=20) permits Goodhart edits to accumulate.The chosen defaults sit near the knee of both trade-offs.Source: C Hyperparameter Sensitivity · pp. 13–14

D Qualitative Analysis of Collapse

• VANILLA AR reaches 74% optimizer or schedule micro-tuning on T1 and 61% repeated reasoning-temperature or arithmetic-loss edits on T3, while audited ARC-Easy accuracy stays flat; DAPS remains heterogeneous through iteration 300.The DAPS examples include changes to normalization, curriculum filtering, and attention-mask handling; the passages caution that heterogeneous edits are not necessarily better edits.Source: D Qualitative Analysis of Collapse · p. 14

E Qualitative Edit Examples

• The examples are paraphrased, trimmed accepted-edit diffs from T1 single-GPU nanochat-style pretraining on Karpathy (2026)’s autoresearch, where edits are limited to train.py.Hunk headers show iteration indices and mechanism categories; the full verbatim logs will be released.Source: E Qualitative Edit Examples · p. 14

E.1 Early-Phase Diversity Under VANILLA AR

• Within its first ∼60 iterations, VANILLA AR explores four mechanism categories, including optimizer, architecture, data, and regularization edits.The examples come from one seed on T1.Source: §E.1 · p. 14

E.2 Late-Phase Collapse Under VANILLA AR

• Five late VANILLA AR edits modify different identifiers but all fall into optimizer or scheduling categories, illustrating preserved surface diversity alongside collapsed mechanism diversity.This pattern underlies the reported entropy collapse H300=1.18 nats in Table 4.Source: §E.2 · pp. 14–15

E.3 Late-Phase Diversity Under DAPS

• In the same iteration window, five DAPS edits span five mechanism categories; CCR and PEM upweight rare categories and suppress near-duplicates without prohibiting any category.Scheduling edits still occur, but no longer dominate the window.Source: §E.3 · p. 15

E.4 Connection to the Diagnostic Axes

• The T1 comparison shows similar surface diversity under both methods, but lower semantic cluster count and mechanism entropy and higher corpus similarity under VANILLA AR.The windows are described as representative rather than extremal.Source: §E.4 · p. 15

F Summary of Validity, Robustness, and Scope Checks

• Across every listed check, the direction of algorithmic mode collapse and ordering VANILLA AR < diversity baselines < DAPS are preserved.Table 9 indexes the threats and checks; outcomes and numbers are reported in the referenced appendices, not restated here.Source: F Summary of Validity, Robustness, and… · p. 15
• The headline claims are scoped to ARLs editing ML pipelines at single-GPU executor scale for up to 1,000 iterations; evidence outside that setting is preliminary.Source: F Summary of Validity, Robustness, and… · p. 15

G Three-Tier Evaluation Protocol: Roles, Per-Task Results, and Absolute Values

• Only mopt is visible to the proposer; maudit is read through scalar comparisons, and mblind is evaluated once after termination by an offline script with no channel back into the loop.The audit signal is accessed by HVG every K=10 iterations and for calibration of τh over the first 30 iterations; audit datasets are disjoint from the other roles within each task.Source: G Three-Tier Evaluation Protocol:… · pp. 15–16, Table 10 · p. 16
• Robustness checks vary the summarizer, embedding model, clustering, and mechanism taxonomy, and also use human labels, taxonomy-free entropy, an AST-based detector, and metric-only Δfaith.The supplied passages identify these checks but do not report their outcomes.Source: G Three-Tier Evaluation Protocol:… · p. 16
• At T=300, DAPS retains its advantage on uninfluenced sets; VANILLA AR’s comparable audit-to-blind drop is attributed to benchmark idiosyncrasy, and the gate fires 2.7 ± 0.9 times per run.The blind metric is evaluated after the run by an offline script with no channel back into the loop.Source: Table 11 · p. 16, G Three-Tier Evaluation Protocol:… · p. 16
• T1 log-perplexity reductions are 0.058 nats (VANILLA AR) and 0.108 nats (DAPS), versus in-loop loss reductions of 0.142 and 0.139.These absolute values reproduce the T1 ratios reported in Table 2.Source: Table 12 · p. 16, G Three-Tier Evaluation Protocol:… · p. 16
• In-loop T2 dev win-rate rises from 38.4 to 47.3 (VANILLA AR) and 47.2 (DAPS), while T3 GSM8K dev exact match rises from 31.2 to 42.9 and 42.7.Table 12 reports absolute audit-metric values, with T=300 entries averaged over 3 seeds.Source: Table 12 · p. 16, G Three-Tier Evaluation Protocol:… · p. 16

H Construct-Validity Controls

• The controls apply 12 published-recipe known-good edits in isolation to T1 and T3, and 8 proxy-only edits targeting task-specific dev ordering, judge phrasing, or answer formats.Examples of known-good edits include rotary position embeddings, a SwiGLU MLP, fused optimizer kernels, sequence-length filtering, and corrected initialization scaling.Source: H Construct-Validity Controls · pp. 16–17
• ρ separates genuine from proxy-only gains (ρ ≈1 vs ρ ≈0); RANDSEARCH transfers 0.74 to 0.81 on the same task pairs, while adaptive VANILLA AR has ρT < 1.The passage argues that the task pairs support near-full transfer and that adaptive optimization induces VANILLA AR’s lower ρT.Source: H Construct-Validity Controls · p. 17, Table 13 · p. 17
• Across 8 audit components and 4 blind sets, VANILLA AR < diversity baselines < DAPS on all 8 components; VANILLA AR ranges 0.33 to 0.52 and DAPS 0.69 to 0.85.The ordering holds across 12 independent evaluations in total.Source: H Construct-Validity Controls · p. 17

I Robustness of the Diagnostic Pipeline

• Across summarizer, embedding, and clustering variants, per-window Ct trajectories correlate above 0.93 and the VANILLA AR–DAPS gap persists; human labels and AST detection also corroborate collapse.Human-label mechanism entropy falls from 1.72 to 1.21 nats, close to the parser-based 1.76 to 1.18; AST detection finds optimizer or schedule edits in 71% of late T1 edits versus 74% for the parser. ∆faith uses only executed metric values and widens as Ct and Ht collapse.Source: I Robustness of the Diagnostic Pipeline · p. 17, Table 14 · p. 17

J Taxonomy Variants and a Taxonomy-Free Estimate

• Table 15 reports that collapse and mitigation persist across alternative taxonomy granularity and boundary placements; the taxonomy-free KL estimate drops 0.44 nats under VANILLA AR and 0.07 under DAPS.The alternatives include coarse, fine, and randomly repartitioned taxonomies, plus human labels; semantic cluster count Ct is another taxonomy-free measure.Source: J Taxonomy Variants and a… · pp. 17–18, Table 15 · p. 17

K Human Validation of Corpus Similarity

• Vt correlates with human novelty ratings at Spearman −0.47 (95% CI −0.60 to −0.31), while changing the summarizing LLM shifts Vt by only 0.021 on average.Three blind raters assessed 120 descriptions; interrater Krippendorff α was 0.61. The authors retain Vt as a secondary axis, not evidence of retrieval.Source: K Human Validation of Corpus Similarity · p. 18

L Horizon and Target-Scale Extensions

• Over the longest tested horizon, collapse deepens rather than self-corrects while DAPS remains stable; the first target-model scale increase reproduces the 1B pattern.Table 16 extends T2 along two axes with one seed per new configuration; executor and target scale, and horizons beyond 1,000 iterations, remain open.Source: L Horizon and Target-Scale Extensions · p. 18, Table 16 · p. 18

M A Preliminary Non-ML Autonomous Research Loop

• In a preliminary Python log-analytics loop, semantic clusters fall from 11 to 5 (∆C = 0.55), while ρT is 0.58 under VANILLA AR versus 0.81 under DAPS.Surface diversity stays nearly flat at 0.71 to 0.70, and 66% of late accepted edits are caching or micro-optimizations; the study is small and single-domain, so headline claims remain scoped to ML-pipeline editing.Source: M A Preliminary Non-ML Autonomous… · p. 18

N Baseline Tuning Grids

• Baselines received the same tuning budget as DAPS calibration: three-point hyperparameter grids on T2’s first 50 iterations, with the best settings fixed across tasks and seeds.The grids covered HITEMP temperature, R-DIVERSE-A similarity threshold and memory size, PRISM-A coverage reward, REFLEX-ION summary length, and HO-EARLYSTOP/HO-REVERT audit intervals; RANDSEARCH used a fixed corpus of 200 modifications.Source: N Baseline Tuning Grids · p. 18