PaperlayerView exact revision on arXiv

Source-linked AI summary

Online Surrogate Repair: Decoupling High-Fidelity Feedback from Search Length in Closed-Loop Discovery

Xiaotang Feng, Philip Torr, Bruno Andreis

arXiv:2609.07655v2cs.LGcond-mat.mtrl-scics.AI

TL;DR

Closed-loop discovery can produce candidate designs cheaply, but reliable feedback is costly, while fixed surrogates retain errors that optimization may exploit. Online surrogate repair uses sparse high-fidelity evaluations to update surrogate feedback during a longer agent search. The paper reports that global prediction error and maximum regret are distinct objectives, and that repair reduces oracle feedback requirements in synthetic environments and MADE.

  • Problem

    Candidate generation can be inexpensive, but reliable scientific feedback is costly, and static benchmarks do not show how approximation errors affect policies that repeatedly act on model feedback.

    Source: §1 · p. 2

  • Method

    Online surrogate repair uses inexpensive surrogate feedback for most interactions and sparse high-fidelity evaluations of archived proposals to update the surrogate during agent search.

    Source: §1 · p. 2

  • Results

    Global prediction error and maximum regret are distinct repair objectives, and OSR reduces oracle feedback requirements in controlled synthetic environments and MADE under LLM and non-LLM proposal policies.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Surrogate repair should be evaluated through downstream optimization, and a limited oracle budget can improve feedback throughout a search without an oracle evaluation after every episode.

    Source: §1 · p. 2

    The framework requires tabular problems and a tabular foundation model that incorporates labels in context; out-of-distribution environments beyond the model’s capabilities were not tested.

    Source: §5 · p. 9

Abstract

from arXiv · show

Closed-loop AI scientists can generate candidate designs at low marginal computational cost, whereas reliable feedback may require wet-lab synthesis, characterization, or high-fidelity computation. Addressing this imbalance through custom laboratory automation remains infrastructure-intensive and costly, while replacing new experiments with a fixed surrogate leaves persistent model errors that can be amplified by optimization. We propose \emph{online surrogate repair} (OSR), a closed-loop algorithm that uses sparse high-fidelity evaluations to update the surrogate throughout a longer agent search conducted primarily with inexpensive surrogate feedback. An acquisition rule selects which designs from the agent's accumulated proposals receive high-fidelity evaluation, and the resulting labels update the surrogate used in subsequent episodes. Across controlled synthetic environments, we demonstrate that improving global surrogate fit does not necessarily reduce maximum regret, whereas Q90-UCB and expected improvement (EI) substantially reduce regret by directing evaluations toward regions that determine the optimizer's decisions. On MADE, controls receiving high-fidelity feedback after every episode require $6.36$--$7.23\times$ more oracle queries to match Online EI under two LLM orchestrators and $10.27\times$ more under the non-LLM Chemeleon+MLIP workflow. Online surrogate repair introduces a novel third feedback regime between fixed-surrogate operation and high-fidelity feedback after every episode, separating the frequency of high-fidelity evaluation from the duration of the agent's search.

1 INTRODUCTION

Closed-loop discovery can generate designs cheaply, but validating them is costly, and fixed surrogates can preserve errors that steer optimization. Online surrogate repair uses sparse high-fidelity labels to update feedback during a longer search, reducing oracle feedback needs while targeting optimization performance rather than global fit alone.

1 INTRODUCTION

  • Candidate generation can scale cheaply, but discovery may be limited by validated outcomes, and static benchmarks do not show how approximation errors affect repeatedly acting policies.In computational materials discovery, candidates still require ranking or validation by potentials, electronic-structure calculations, or experiments.Source: §1 · p. 2
  • Self-driving laboratories require specialized infrastructure and continued spending per physical evaluation, while fixed-surrogate errors persist and can steer optimization toward localized false maxima.A false maximum can determine the search trajectory even when global predictive error is low.Source: §1 · p. 2
  • OSR uses inexpensive surrogate feedback for most interactions while an acquisition rule selects archived proposals for sparse oracle evaluation and updates the surrogate for later episodes.The agent retains control of proposal generation and design selection.Source: §1 · p. 2
  • Global prediction error and maximum regret are distinct repair objectives, and acquisition rules can substantially improve surrogate optimization performance.Repair is evaluated by downstream optimization because the surrogate is intended to guide search.Source: §1 · p. 2
  • OSR reduces oracle feedback required for closed-loop optimization in controlled synthetic environments and MADE under both LLM and non-LLM proposal policies.The approach lets a limited oracle budget improve the surrogate throughout the search rather than requiring evaluation after every episode.Source: §1 · p. 2

2 RELATED WORK

Prior work spans scientific agents, sequential optimization, and surrogate modeling, but these approaches typically assign search or model-improvement roles differently. OSR leaves proposal generation to the agent and uses sparse labels from its evolving proposal archive to repair feedback for subsequent search.

2.1 SCIENTIFIC AGENTS AND CLOSED-LOOP DISCOVERY

  • OSR is proposal-policy agnostic: it changes how costly evidence updates feedback and can accompany either an LLM agent or a fixed generator.MADE focuses on budgeted, closed-loop materials discovery (Malik et al., 2026).Source: §2.1 · p. 3

2.2 ACTIVE LEARNING, BAYESIAN OPTIMIZATION, AND MULTI-FIDELITY SEARCH

  • Unlike standard Bayesian optimization, where acquisition is the proposal policy, OSR acquisition selects archived agent proposals for high-fidelity labels, allowing episode and oracle-evaluation counts to vary independently.Multi-fidelity optimization also chooses among information sources (Poloczek et al., 2017).Source: §2.2 · p. 3

2.3 SURROGATE VERIFIERS AND TABULAR FOUNDATION MODELS

  • Tabular foundation models can incorporate measurements as context rows without task-specific training, while OSR acquires labels from an evolving archive to improve subsequent closed-loop optimization.Unlike active in-context learning from a fixed pool for global prediction (Treerath & Pittorino, 2026), OSR evaluates effects on the search guided by the repaired surrogate.Source: §2.3 · p. 3

3 METHODS

OSR separates frequent agent search with inexpensive surrogate feedback from sparse oracle evaluations that update the surrogate. The section defines this repair process and describes acquisition rules for choosing designs to evaluate.

3.1 PROBLEM DEFINITION

  • OSR evaluates repair on global NRMSE and normalized maximum regret, appending oracle labels to the context rather than training model parameters.The regret comparison uses the best design found by a high-budget reference search versus the design found by surrogate-guided search.Source: §3.1 · pp. 3–4

3.2 ONLINE SURROGATE REPAIR FOR CLOSED-LOOP AI SCIENTISTS

  • Across episodes, the agent receives surrogate feedback while scheduled oracle queries select archived proposals and update the context for subsequent episodes.The measured design need not be the episode’s own proposal; expected improvement is the strongest tested repair rule, yielding the Online EI instantiation.Source: §3.2 · p. 4

3.3.1 TARGET AGNOSTIC METHODS

  • Random sampling draws from the world’s design distribution as a control, while Global-UQ selects the largest 80% predictive interval among geometry-derived candidates.Global-UQ is described as the natural policy when the target is global fit.Source: §3.3.1 · p. 5

3.3.2 OPTIMUM SEEKING METHODS

  • Peak validates the surrogate-predicted incumbent, Q90-UCB queries the highest optimistic quantile, and G2P(16) splits 32 queries equally between Global-UQ then Peak.Q90-UCB targets designs with the largest plausible upside, where labels can expose overestimates or confirm genuine optima.Source: §3.3.2 · pp. 5–6

3.3.3 IMPROVEMENT AWARE ACQUISITION

  • Expected improvement scores designs by expected margin over the best observed label, integrating the full predicted upper tail and updating the incumbent after each query.Unlike Q90-UCB, EI references the verified incumbent and stops prioritizing regions that cannot improve the best observed response.Source: §3.3.3 · p. 6

4 EXPERIMENTS

Across synthetic optimization and MADE materials discovery, online repair uses sparse oracle queries to improve search outcomes relative to static repair or full-feedback baselines. The experiments also show that optimizer-facing regret and global predictive error can favor different repair strategies.

4.1 STATIC SURROGATE REPAIR

  • Across four static-repair settings, Q90-UCB and EI reduced mean maximum regret by 0.424 and 0.409, versus 0.063 and 0.052 for Global-UQ and Random.Global-UQ and Random produced the largest mean NRMSE improvements, while optimizer-targeting methods yielded little global error improvement.Source: §4.1 · pp. 6–7

4.2 SYNTHETIC WORLDS

  • The synthetic study used a ReAct agent for T = 200 episodes on 20 held-out deterministic worlds with a single global maximum, comparing online repair at Q = 32 with controls at Q = 200.The experiments also included a noisy-observation condition on the same worlds.Source: §4.2 · pp. 7–8
  • At matched Q = 32, Online EI reduced mean regret by 44% versus Static EI and closed 76% of the gap to Online EI Control (Q = 200), using 16% of its queries.With noisy oracle observations, Online EI at Q = 32 retained 19% lower mean regret than Static EI and again had lower mean regret than Oracle Control; Online EI Control had the lowest mean regret.Source: §4.2 · pp. 7–8

4.3 MADE BENCHMARK

  • On MADE, each of 30 chemical systems received three T = 300 runs under each of two LLM orchestrators; Online EI used Q = 32 versus Q = 300 for full-feedback baselines.The benchmark uses MACE for surrogate feedback and Orb-v3 as oracle, and counts novel structures within τ = 0.1 eV/atom of the convex hull as discoveries.Source: §4.3 · p. 8
  • Control required 7.23(28) and 6.36(20) times as many oracle queries to match Online EI under DeepSeek and Luna, respectively.These acceleration factors are lower bounds because Control failed to catch up within 300 queries in 21 and 7 of the 90 runs.Source: §4.3 · pp. 8–9
  • At Q=32, Online EI achieved 130 discoveries with the non-LLM Chemeleon+MLIP workflow, 93% of Control’s 141, and an AF of 10.27.It also reached 82% and 67% of the corresponding Controls’ Q=300 discovery counts and led both full-feedback baselines in unique compositions and space groups at matched budget.Source: §4.3 · p. 9

5 DISCUSSIONS AND CONCLUSION

The discussion concludes that global predictive accuracy and maximum regret are misaligned objectives, and presents online surrogate repair as sparse oracle updates during a longer agent search. The authors identify representation, model capability, and untested agent and surrogate settings as boundaries on the current evidence.

5 DISCUSSIONS AND CONCLUSION

  • The authors conclude that global prediction error and maximum regret are misaligned surrogate objectives, while online repair requires substantially fewer oracle queries than full feedback on synthetic worlds and MADE.They recommend evaluating feedback models through the searches they guide alongside predictive accuracy.Source: §5 · p. 9
  • The framework requires a tabular problem representation and a tabular foundation model that can incorporate new labels in context; some out-of-distribution environments may exceed model capabilities and were not tested.The authors report not encountering such environments in their experiments.Source: §5 · p. 9
  • OSR and Online EI were not tested with test-time training or reinforcement-learning agents, other surrogate classes, or more advanced optimization-aware schedules.Source: §5 · p. 9

A. Gilad Kusne, Heshan Yu, Changming Wu, Huairuo Zhang, Jason Hattrick-Simpers, Brian DeCost,

The supplied passages are bibliography entries spanning materials discovery, active learning, generative design, and scientific automation.

A MADE EVALUATION METRICS

The appendix defines MADE discovery and diversity metrics, including query-budget acceleration measures, and describes the synthetic worlds used for controlled experiments. The worlds use held-out cohorts and prepared initial contexts and audit samples.

A MADE EVALUATION METRICS

  • MADE reports discoveries, AUDC, unique compositions, unique space groups, enhancement factor, acceleration factor, and unmatched runs.AF is the mean per-run ratio of query counts to reach the target discovery count; unmatched counts runs where Control does not reach that target within 300 queries.Source: A MADE EVALUATION METRICS · p. 15
  • AUDC can exceed one under online repair because multiple discoveries may occur per oracle query; reported Table 3 values are means over 90 runs except Unmatched.Source: A MADE EVALUATION METRICS · p. 15

B SYNTHETIC WORLD CONSTRUCTION

  • Synthetic worlds are deterministic mixed-type tabular SCMs, and the held-out cohort W5241–W5260 contains four worlds per each of five mechanism families.The cohort is disjoint from surrogate training; worlds vary in nonlinear mechanisms and include smooth, thresholded, oscillatory, localized-basin, and categorical/quantized families.Source: B SYNTHETIC WORLD CONSTRUCTION · p. 15
  • Each world uses a 1,000-row initial context and a 50,000-row audit sample; the reference response is the larger maximum from the audit sample and guided search.Static EI and Global-UQ choose 32 queries in one batch from a shared locked pool of 10,000 rows.Source: B SYNTHETIC WORLD CONSTRUCTION · p. 15

C.1 ONLINE EI QUERY-BUDGET SWEEP

The query-budget sweep shows that Online EI continues to improve across an eightfold budget range, with diminishing gains as the budget grows.

C.1 ONLINE EI QUERY-BUDGET SWEEP

  • Online EI's final normalized regret improves monotonically from Q=8 to Q=64.The sweep covers 20 synthetic worlds with TabPFN; lower regret is better.Source: Table 4 · p. 16, §C.1 · p. 16
  • A 0.0366 mean-regret reduction from Q=8 to Q=32 compares with 0.0071 from Q=32 to Q=64, showing diminishing returns.Source: §C.1 · p. 16
  • At Q=8, Online EI reaches 0.0918, ahead of Static EI at Q=32 (0.0984) and no repair (0.1432).Source: §C.1 · p. 16

C.2 SELECTION GEOMETRY OF ONLINE ACQUISITION RULES

Acquisition geometry is associated with final regret, but neither acquisition distance nor incumbent-update frequency alone tracks the performance ordering.

C.2 SELECTION GEOMETRY OF ONLINE ACQUISITION RULES

  • EI has the lowest mean regret, 0.0552, despite moving 1.9× farther per step than Q90-UCB and 4.5× farther than Peak.Peak and Tab-AICL have mean regrets of 0.0976 and 0.1169, respectively.Source: Table 5 · p. 16, §C.2 · p. 16
  • Peak's concentrated acquisitions and Tab-AICL's broad coverage yield mean regrets of 0.0976 and 0.1169, so neither local concentration nor broad coverage is sufficient for low regret.Source: §C.2 · p. 16
  • Peak, Q90-UCB, and EI update the incumbent 6.1, 6.0, and 5.6 times per world, respectively, despite differing final regrets.Each method also takes smaller steps following an incumbent update; the measurements establish association, not a causal mechanism.Source: §C.2 · pp. 16–17

D FULL STATIC SURROGATE REPAIR RESULTS

The full static-repair results are reported as a maximum-regret and NRMSE matrix across two tabular models and two context sizes.

D FULL STATIC SURROGATE REPAIR RESULTS

  • Table 6 reports the complete final maximum-regret and NRMSE matrix for static surrogate repair.Source: D FULL STATIC SURROGATE REPAIR RESULTS · p. 17
  • The comparison covers Q=32 with TabPFN-3 and TabICLv2 at context sizes N ∈ {100, 1000}.Source: Table 6 · p. 17
  • Within each model-and-context block, bold marks the best result and underlining marks the second-best result for each metric.Source: Table 6 · p. 17

E COST ESTIMATION

The cost estimate compares Online EI with Control at matched discovery counts, while the adaptation section specifies a regression version of Tab-AICL's hybrid selection rule.

E COST ESTIMATION

  • At matched mean discovery counts, Online EI adds $0.33 per DeepSeek trajectory and $0.68 per Luna trajectory relative to Control.These are estimated LLM charges; the passage describes the increases as small compared with a typical wet-lab synthesis and characterization cycle.Source: E COST ESTIMATION · p. 17, Table 7 · p. 18
  • The cost table reports per-trajectory LLM costs in USD and H100-seconds, with discovery counts and Control catch-up also tabulated.Source: Table 7 · p. 18

F TAB-AICL ADAPTATION

  • Tab-AICL replaces classification entropy with the surrogate's 80% predictive-interval width and, for B=1, selects the shortlisted design maximizing distance from the labelled context.Numerical features are standardized and categorical features ordinal-encoded; the surrogate, Q=32 budget, schedule, and archive match Online EI, leaving the selection rule as the difference.Source: F TAB-AICL ADAPTATION · p. 18

G ADDITIONAL MADE RESULTS

At a matched 32-query budget, Online EI improves best-material quality over Control under the two LLM orchestrators and retains most full-feedback quality across three workflows. Additional results compare discovery and diversity and distinguish audited from budgeted evaluation.

G ADDITIONAL MADE RESULTS

  • Discovery results use post-hoc oracle audits of all episode proposals, while budgeted Online EI uses only its 32 run-time oracle outcomes for every metric.Audit labels do not enter the search or surrogate context, and audit oracle calls are excluded from feedback budget Q.Source: G ADDITIONAL MADE RESULTS · p. 19
  • 10.2% and 42.4% higher Top-1 scores, and 24.5% and 68.8% higher Top-5 scores, for Online EI versus Control under DeepSeek and Luna, respectively.Top-5 quality AUC increases by 28.9% and 137.2%, respectively; Chemeleon+MLIP is near parity with Control on these budgeted quality comparisons.Source: Table 8 · p. 19, G ADDITIONAL MADE RESULTS · pp. 19–20
  • 95.0–98.4% of Control’s 300-query mean Top-1 score and 93.6–97.5% of its Top-5 score are retained using 10.7% of feedback queries over the same 300-episode horizon.Budgeted AF-top estimates are 24.8, 34.9, and 45 for DeepSeek, Luna, and Chemeleon+MLIP; some include lower-bound contributions when Control misses the target within 300 queries.Source: G ADDITIONAL MADE RESULTS · p. 20
  • Chemeleon+MLIP leads Online EI discovery counts, Luna leads mean Top-1 and Top-5 under both accountings, and Chemeleon leads Top-5 quality AUC.Source: G ADDITIONAL MADE RESULTS · p. 20
  • Budgeted Online EI’s discovery EF is 1.451 and 1.938, with AF of 2.04 and 2.19 under DeepSeek and Luna; Chemeleon+MLIP reaches near parity with Control.Source: Table 9 · p. 20, G ADDITIONAL MADE RESULTS · p. 20
Loading 2609.07655v2…