Large Discovery Models: Empirically-grounded Model-Based Open-Ended Search

• Scientific discovery is sequential search over vast, structured, potentially unbounded hypotheses or designs, where only limited candidates receive costly empirical evaluation and observations update subsequent search.The operational setting includes physical experiments, simulations, computational tests, and other external evidence.Source: §1 · pp. 2–3
• LLMs can generate and refine structured candidates but lack reliable external value estimates, while Bayesian optimisation uses uncertainty-aware surrogates yet may miss candidates outside its representation and search support.Discovery therefore requires both an expanding search frontier and empirical allocation of limited experimental resources.Source: §1 · p. 3
• The framework distinguishes exploitation of high-value reachable candidates, exploration of uncertain candidates within current support, and discovery of previously unrepresented candidates or relationships.Its contributions include an acquisition-tilted search policy, regret analysis separating surrogate error from search suboptimality, and evaluation across programs, biological sequences, and molecules.Source: §1 · pp. 4–5
• LDM couples an LLM proposal model with a continually updated probabilistic surrogate whose acquisition function allocates refinement, computation, and costly evaluations using predicted reward and epistemic uncertainty.The recurrent loop incorporates new observations to redirect subsequent generation and search.Source: §1 · pp. 3–4
• 2.4× larger validation-BPB reduction, 18.2% lower mean binding energy, and 62.4% and 63.1% higher molecular hypervolume than the cited LLM-only and Bayesian-optimisation baselines, respectively.These results span AutoResearch, antibody design after 200 evaluation steps, and multi-objective molecular optimisation.Source: §1 · p. 5

Problem formulation

• The search seeks high-reward designs in an evolving hypothesis universe whose full extent is unknown and must be expanded through new candidates, representations, and design families.Evaluation is costly, black-box, and potentially noisy, while the relevant search space is progressively constructed during discovery.Source: §2 · pp. 5–6

Surrogate knowledge

• A probabilistic surrogate summarizes empirical reward knowledge with posterior mean µ_t(x) and uncertainty σ_t(x), but does not enumerate the full hypothesis space.It conditions on previously evaluated designs and noisy feedback, describing the search’s current modelling support.Source: §2 · p. 6

Epistemic regimes

• The framework partitions hypotheses into known knowns, known unknowns, unknown knowns, and unknown unknowns according to evaluation, reachability, surrogate support, and retrieval status.All four regimes can persist: external information requires explicit retrieval, and mathematical membership in a space does not imply search access or calibrated modelling.Source: §2 · p. 6

Search operations

• Exploitation selects high-reward, low-uncertainty known knowns; exploration evaluates reachable uncertain designs; discovery expands the frontier by surfacing hypotheses beyond current effective reach.Discovery converts unknown unknowns into known unknowns, exploration converts known unknowns into known knowns, and exploitation operates within known knowns.Source: §2 · p. 6

Scope and transition

• The framework focuses on discovery, exploration, and exploitation in a single sequential process, while leaving memory retrieval and federated discovery for future work.It is conceptual rather than algorithm-specific; the next section instantiates these operations with a generative model, probabilistic surrogate, and acquisition-guided search policy.Source: §2 · p. 7

3 The Large Discovery Model

• LDM combines a generative foundation model, probabilistic surrogate, and model-based acquisition function to guide sequential inverse design.The generator proposes structured candidates, the surrogate grounds predictions in empirical observations and epistemic uncertainty, and acquisition values direct development and evaluation.Source: §3 · pp. 7–8

3.1 An Acquisition-Tilted Search Distribution

• The search policy exponentially tilts the generative prior toward candidates with greater acquisition value while KL regularization preserves plausible designs.The coefficient η controls the trade-off: larger values emphasize acquisition, whereas smaller values keep the policy closer to the generative prior; the construction is model-agnostic and connects empirical surrogate feedback to sequential evaluation.Source: §3 · pp. 8–9, §3.1 · p. 9

3.2 Surrogate Modelling of the Reward Function

• Gaussian-process surrogates provide sample-efficient Bayesian updates and non-parametric uncertainty estimates that support acquisition functions balancing predicted reward against unexplored regions.The predictive mean and variance instantiate exploitation and exploration, while the LLM reservoir supplies discovery; together they realize the discovery–exploration–exploitation triangle.Source: §3.2 · pp. 10–11

3.3 Acquisition-Tilted Inference-Time Search

• Because exact sampling is intractable, LDM approximates the tilted policy by acquisition-weighting finite LLM-generated pools, with larger pools improving fidelity at higher computational cost.The generator-agnostic scheme supports stochastic or deterministic selection, batched search, and multiple modalities; prefix-value extensions enable beam search and Monte Carlo Tree Search, whose full evaluation is left for future work.Source: §3.3 · pp. 11–12

3.4 Consolidating acquisition-tilted inference via model fine-tuning

• LDM-TTS consolidates acquisition-guided search experience into the proposal model, forming a recurrent search–learn cycle that improves starting distributions across states and tasks.Unlike reward-guided training, it supervises state-dependent acquisition value, balancing predicted quality, epistemic uncertainty, constraints, and Pareto-frontier expansion rather than merely imitating high-reward outcomes.Source: §3.4 · pp. 13–14, Figure 6 · p. 13
• Acquisition reweighting converts finite-sample candidate pools into a training distribution that approximates the tilted policy while retaining the LLM’s structured generative prior.The pool determines which candidates enter, and the exponential acquisition factor supplies empirical policy improvement.Source: §3.4 · p. 14
• LDM records counterfactual candidates and surrogate scores as search experience, then weights retained traces so proposer learning amortises repeated search without becoming identical to the online policy.Candidates from the same history expose exploitation, exploration, feasibility, and frontier-expansion values at that search state.Source: §3.4 · p. 14
• Reasoning augmentation attaches concise rationales that identify evidence, diagnose progress, and explain whether each action exploits, explores, preserves diversity, satisfies constraints, or expands support.The rationale explains why the next experiment is worth running rather than how to derive a known answer, and can support additional test-time discovery reasoning.Source: §3.4 · p. 14
• The fast search loop recalibrates values from new observations, while the slow learning loop consolidates recurring acquisition-guided decisions across LDM-TTS trajectories for transfer testing.The training objective supports reasoning-augmented learning with λz = 1 or action-only learning with λz = 0; experiments test reservoir improvement and transfer to unseen targets and tasks.Source: §3.4 · pp. 14–15

4 The Algorithm

• LDM iteratively fits a Gaussian-process surrogate, constructs an acquisition function, samples promising candidates, evaluates noisy rewards, and updates its dataset and generative context.If the surrogate detects a stale regime, the LLM appends reflection feedback to the context before subsequent proposals.Source: §4 · pp. 15–16

4.1 Policy Construction and Realisation

• The LLM base measure can directly generate complete designs or indirectly define a parameterised search region sampled externally, improving validity handling or pool-size scalability.Candidates are selected deterministically by top-b acquisition scores or stochastically with Gumbel-top-b sampling without replacement; pool size N controls test-time compute.Source: §4.1 · p. 16

4.2 Surrogate on the Dynamic Support

• When the LLM selects an active subspace A_t, the Gaussian process is fit on historical observations within A_t rather than the full design space X, then updated as that support changes.The reduced support allows the kernel and mean function to be defined directly in the parameterised search space.Source: §4.2 · p. 16

4.3 Practical Implementation

• The framework extends to constrained and cost-aware acquisition, decomposed surrogate feedback, EHVI for multi-objective optimisation, and without-replacement batch selection.These extensions expose predictive mean, epistemic uncertainty, constraint slack, and estimated cost separately when useful for proposal updates.Source: §4.3 · pp. 16–17

Regret decomposition

• The discovery gap measures missing high-quality designs outside the current reservoir support, while optimisation regret measures selection error among currently reachable candidates.The reservoir controls whether promising regions are discovered, whereas acquisition tilting controls selection within those regions.Source: §5 · pp. 17–18

Assumptions

• The analysis assumes UCB calibration under standard kernel-bandit conditions and reservoir coverage with one-sided Hölder regularity around the global optimum.The coverage radius converts distance from the optimum into an objective gap.Source: §5 · p. 18

Sampling analysis

• The sampling-shortfall term decreases when acquisition tilting is sharper or when the reservoir assigns more probability mass to near-best-UCB candidates.The near-UCB mass κ_t(ζ_t) measures how much proposal probability is available near the best current acquisition value.Source: §5 · pp. 18–19

Regret bound

• Theorem 1 bounds LDM’s average regret by discovery gap, GP-UCB optimisation error, and LDM sampling shortfall, with probability at least 1 − δ_gp − δ_samp.Corollary 1 extends the same right-hand side to simple regret.Source: §5 · p. 19

LLM role

• The LLM improves the bound by proposing candidates closer to the optimum and allocating more probability to high-acquisition designs, reducing coverage and sampling errors.Its proposal is structured and history-dependent rather than uniform over the full design space.Source: §5 · p. 19

6.1 Overview of the Case Studies

• LDM spans neural-network training programs, antibody CDRH3 sequences, and multi-objective small molecules, testing one discovery engine across discrete, executable, and open-ended hypothesis spaces.The benchmarks use in-silico or digital-oracle evaluation, while wet-lab validation remains future work.Source: §6 · p. 19, §6.1 · p. 20

6.1.1 Autonomous research loop

• In autoresearch, LDM revises a stalled program parameterisation, expanding search support and enabling a second improvement phase after an initial plateau.The comparison fixes the coding model, environment, and five-minute-per-run budget while contrasting the original LLM-only loop with full LDM.Source: §6.1.1 · pp. 20–21

6.2.1 Test-time search scaling and acquisition functions

• 0.93421 val bpb is achieved by LDM versus approximately 0.956 for the pure-LLM loop after 875 experiments, showing the importance of surrogate-based value guidance.The pure-LLM loop lacks a surrogate, acquisition, and uncertainty signal, leaving its research log as narrative memory rather than a searchable value landscape.Source: Figure 7 · p. 21, §6.2.1 · p. 26

6.1.2 Antibody CDRH3 design

• Antibody design tests acquisition-guided selection over a 20^L discrete CDRH3 space with unknown, costly sequence-to-affinity mappings, using policy or direct generation variants.The task evaluates one candidate per iteration across five PDB targets with 200 evaluations per target, while Policy generates larger structured candidate pools than Direct.Source: §6.1.2 · p. 22

6.1.3 Small-molecule drug discovery

• Molecular optimisation uncovers unrelated scaffolds through staged Pareto-front expansion, demonstrating exploration beyond the local neighbourhoods reached by standard BO or seed-based expansion.The trajectory progresses through scaffold identification, substituent screening, objective bifurcation, feature recombination, and final fine-tuning for KRAS G12D objectives.Source: §6.1.3 · pp. 23–24, Figure 9 · p. 24, Figure 10 · p. 24

6.2.1 Test-time search scaling and acquisition functions

• Larger test-time budgets improve search, discovery avoids fixed-support plateaus, and UCB and EI outperform posterior-mean selection, with UCB performing best in autoresearch.For molecular design, larger budgets help and scalarised mean acquisition can outperform EHVI when screening pools are small; antibody scaling is weaker and target-dependent.Source: Figure 11 · p. 25, §6.2.1 · p. 26

6.2.2 Hyperparameter sensitivity

• On molecular design, T_LLM = 1.25 gives the highest final hypervolume, increasing acquisition temperature generally helps, and acquisition tilt dominates final Pareto-front expansion.Antibody design is less temperature-sensitive because Policy mode compensates for weak sequence-level priors by parameterising structured search regions and generating larger pools.Source: §6.2.2 · pp. 26–27, Figure 12 · p. 27

6.3.1 AutoResearch

• 0.0727 validation-BPB reduction lets LDM outperform the LLM-only baseline by 2.4× under the matched autoresearch budget, reaching val bpb 0.93421.A matched B200 run reaches val bpb = 0.902291 in 59 runs, ahead of forge (0.9264) and overmind (0.9274).Source: §6.3.1 · p. 27

6.3.2 Antibody CDRH3 design

• Policy-based LDM variants consistently outperform direct-generation variants in antibody CDRH3 design, achieving affinity comparable to AntBO while direct variants are only marginally better than standalone LLMs.Policy mode expands the candidate pool while the surrogate selects evaluations, addressing sparse and rugged protein fitness landscapes without additional oracle calls.Source: §6.3.2 · pp. 28–29, Figure 13 · p. 28

6.3.3 Small-molecule drug discovery

• Both LDM variants substantially outperform random search and MOBO on molecular Pareto hypervolume, with Direct-Softmax strongest because broad scaffold exploration outweighs ReaSyn’s improved synthesizability for early lead discovery.MOBO plateaus after roughly 40 evaluations, whereas direct generation sustains front expansion across the full budget.Source: §6.3.3 · p. 29, Figure 14 · p. 29

6.4 Learning Discovery Experience with Fine-Tuning

• Fine-tuning improves the proposal reservoir inside the same acquisition-guided LDM loop, with reasoning-augmented training additionally teaching surrogate-grounded research rationales.The fine-tuned proposer changes proposal generation while the online surrogate continues estimating value and uncertainty from experimental history.Source: §6.4 · p. 29

6.4.1 AutoResearch

• 0.9788 val bpb is achieved by the reasoning-augmented fine-tuned AutoResearch proposer, beating the Qwen3-Coder-30B reference at 0.9801 and all other tested proposer configurations.Without reasoning, the fine-tuned model reaches 0.9829; base models with and without reasoning reach 0.9825 and 0.9965.Source: §6.4.1 · p. 30, Figure 15 · p. 30

6.4.2 Antibody CDRH3 Design

• Across five antibody antigens, both fine-tuned variants achieve lower binding energies than their corresponding base proposers, with reasoning-augmented fine-tuning lowest on average.All variants use the same acquisition-guided loop, expected-improvement acquisition, and parallel evaluation budget.Source: §6.4.2 · p. 30, Figure 16 · p. 31

6.4.3 Multi-objective Small-Molecule Discovery

• 26.660±2.608 final hypervolume is achieved by the reasoning-augmented fine-tuned proposer on KRAS G12D, exceeding the same student without reasoning and both teacher variants.The result demonstrates transfer beyond the source task, although the reported intervals overlap.Source: §6.4.3 · p. 31, Figure 17 · p. 32

6.4.4 Out-of-distribution Generalisation

• Fine-tuned policies generalise to unseen antibody targets and an entirely unseen protein task, matching or exceeding in-distribution models on most targets while clearly beating base proposers.The exception is 1H0D C, whose narrow optimum requires de novo motif design rather than candidate selection; overall, gains favor reasoning-augmented policies and reflect exploit-explore-discover search with domain-dependent policy or direct-generation modes.Source: §6.4.4 · pp. 31–32, Figure 18 · p. 32, Figure 19 · p. 33

7.1 Large Language Models and Their Limitations in Scientific Discovery

• LLMs have advanced through scaling, emergent reasoning, in-context learning, instruction following, post-training, and test-time search, but remain poorly calibrated for open-ended scientific discovery.Scientific decisions require external value signals because internal confidence is unreliable, while frontier agents achieve only 21.5/50 on ResearchClawBench and 21.0% on PaperBench versus 41.4% by PhD researchers (Xu et al., 2026; Starace et al., 2025).Source: §7.1 · pp. 34–35

7.2 Statistical Learning for Scientific Discovery

• Statistical learning spans finite and infinitely-armed bandits and Bayesian optimisation, offering uncertainty-aware exploration but struggling with unbounded, semantic, combinatorial, and constraint-rich scientific design spaces.LDM treats the LLM as an adaptive reservoir and uses a missing-mass view of the discovery gap, while Bayesian optimisation combines surrogate predictive mean and variance through acquisition functions such as EI and UCB.Source: §7.2 · pp. 35–36

7.3 Hybrid Approaches: LLM-Guided Discovery

• Hybrid methods combine LLM generation or representations with Bayesian optimisation, but critiques find unreliable LLM uncertainty and weak feedback sensitivity, motivating an explicit calibrated GP posterior.Examples include OPRO, LLAMBO, BoChemian (Rankovic and Schwaller, 2023; Ranković et al., 2025), LABO, and LLM-assisted kernel or language-space methods.Source: §7.3 · p. 37
• LDM differs from offline dLLM (Yuan et al., 2026) by operating online with sequential expensive evaluations, selecting designs for both immediate value and future knowledge.Its central synthesis is to retain the GP surrogate for calibrated uncertainty, use the LLM as a candidate reservoir, and make the acquisition function the value signal for inference-time search.Source: §7.3 · p. 37

8 Conclusion, Limitations, and Outlook

• LDM couples LLM structured generation with Bayesian-surrogate uncertainty in a generation, evaluation, and update loop that expands the search frontier and allocates experimental resources.The framework addresses complementary limitations: LLMs lack reliable objective-value and uncertainty estimates, while Bayesian optimisation struggles to propose valid candidates in complex discrete spaces.Source: §8 · pp. 37–38
• The framework formalises exploitation, exploration, and discovery through a KL-regularised acquisition-guided objective with a closed-form acquisition-tilted search distribution.Its regret decomposition separates generative coverage gaps, surrogate fitting error, and sampling-strategy suboptimality.Source: §8 · p. 38
• LDM supports direct generation, indirect parameterisation, decomposed iterative refinement, and post-training amortisation that reduces inference-time cost while preserving search quality.Reasoning-augmented supervised fine-tuning transfers high-budget test-time search policies into model weights.Source: §8 · p. 38
• Across neural-network training, antibody design, and molecular optimisation, LDM outperformed reported LLM-only and BO-only baselines, while search-frontier expansion, compute scaling, and fine-tuning improved efficiency.Cross-target and cross-task transfer showed stable fine-tuning gains, indicating generalisable discovery decision logic rather than task-specific memorisation.Source: §8 · p. 38
• The Gaussian-process surrogate scales cubically with observations, batch sampling increases LLM inference overhead, and manually specified kernels and representations limit cross-domain automation.Performance also depends on pretrained domain knowledge, while external knowledge and data are not yet systematically incorporated into the closed loop.Source: §8 · p. 38
• Future work will retrieve external knowledge, federate parallel discoveries, use sparse Gaussian processes, and automate surrogate kernel and prior specification across domains.Preference learning and reinforcement learning are proposed to internalise acquisition-guided decision logic in the generative distribution.Source: §8 · pp. 38–39

A Notation

• The notation defines the design space, candidate and optimal designs, objective rewards, evaluation history, search context, generative distribution, surrogate statistics, acquisition policy, and evaluation-budget variables.It also specifies inference configuration, candidate-pool and batch sizes, GP kernel and representation terms, confidence parameters, coverage quantities, refinement branches, and reward thresholds.Source: A Notation · pp. 47–48

B Detailed Proofs

• The detailed-proofs section presents full proofs for Proposition 1, Lemma 1, and Theorem 1.These proofs are organized into separate subsections for the proposition, lemma, and theorem.Source: B Detailed Proofs · p. 48

B.1 Proof of Proposition 1

• Proposition 1 shows that the variational objective is tight exactly when q = π_t, using nonnegative KL divergence and normalization into Z_t.The equality condition follows from KL(q∥π_t) ≥ 0 with equality iff q = π_t.Source: §B.1 · p. 49

B.2 Proof of Lemma 1

• Lemma 1 fixes the round and history so the reservoir support, inference-configured proposal, and acquisition function are treated as fixed.The acquisition function is defined as a_t(x) = µ_t(x) + √β_tσ_t(x).Source: §B.2 · p. 49
• Lemma 1 derives a tail bound for the relevant event and converts it into a high-probability statement by selecting η according to ρ_t and κ_t(ζ_t).Because ρ_t ∈ (0, 1) and κ_t(ζ_t) ≤ 1, the resulting z is non-negative and the bad-event probability is at most ρ_t.Source: §B.2 · pp. 49–50

B.3 Proof of Theorem 1

• Theorem 1 decomposes regret into discovery and optimisation terms, controlling the latter through UCB calibration and Lemma 1.The discovery component is governed by reservoir coverage, while the optimisation component is bounded on simultaneous good events.Source: §B.3 · p. 50
• Theorem 1 establishes the sampling event by applying Lemma 1 at each round, bounding conditional failure by ρ_t, and using iteration plus a union bound.This yields a simultaneous event over t = 1, . . . , T.Source: §B.3 · p. 50
• On the joint good event, UCB optimism and Lemma 1 control the optimisation term, with GP-UCB information-gain variance bounds and Cauchy–Schwarz completing the argument.The proof then combines the good-event probabilities with a final union bound.Source: §B.3 · pp. 50–51

C Technical Implementation Details

• LDM uses a Gaussian Process to model black-box rewards from historical observations, providing predictive means and epistemic uncertainty for acquisition-guided search.The appendix follows the acquisition-tilted search framework and expands surrogate training and sampling execution.Source: §C.1 · p. 52, C Technical Implementation Details · p. 52

C.1.1 Prior Specification

• The GP uses a constant mean, design-space-adapted positive-definite kernels, and jointly estimated kernel, noise, and prior-mean parameters.Examples include RBF kernels for continuous spaces and string kernels for sequence spaces.Source: §C.1.1 · p. 52

C.1.2 Posterior Inference

• Bayesian conditioning yields closed-form Gaussian posterior predictions whose mean estimates expected reward and whose variance quantifies epistemic uncertainty.Posterior inference uses the cumulative historical dataset, kernel matrix, cross-covariance vector, observed rewards, and the constant-mean prior.Source: §C.1.2 · pp. 52–53

C.1.3 Hyperparameter Estimation

• GP hyperparameters are fitted by Type-II maximum likelihood with gradient-based optimisation such as L-BFGS and re-estimated after each iteration.This adapts to the design space’s smoothness and noise level while improving surrogate fitting and uncertainty calibration.Source: §C.1.3 · p. 53

C.2 Acquisition Functions

• LDM supports EI, UCB, and EHVI acquisition functions that convert GP posterior reward and uncertainty into search-value signals for single- and multi-objective discovery.EI balances expected returns with exploration, while UCB adjusts exploration through β_t; EHVI targets hypervolume gains without pre-specified objective weights.Source: §C.2 · p. 53, §C.2.1 · p. 53, §C.2.2 · p. 53

C.2.1 Expected Improvement

• Expected Improvement prioritises expected gain over the best observed reward, balancing exploitation and exploration through an exploration-adjustment parameter ξ.EI is presented for single-objective optimisation.Source: §C.2.1 · p. 53

C.2.2 Upper Confidence Bound

• Upper Confidence Bound combines predictive mean and uncertainty, with β_t controlling the shift between exploitation-biased and exploration-biased search.UCB is described as having theoretical cumulative-regret guarantees (Srinivas et al., 2010).Source: §C.2.2 · p. 53

C.3 Batch Acquisition Sampling

• For parallel evaluation, LDM computes acquisition-tilted weights from LLM generation probabilities and acquisition values, adds independent Gumbel noise, and selects the top k candidates without replacement.This produces the Plackett–Luce weighted batch distribution (Kool et al., 2019), preserving acquisition weighting and preventing duplicate selections.Source: §C.3 · p. 54

Experimental regimes

• The experiments contrast multi-turn autoresearch with single-step molecule and CDRH3 proposal tasks, where surrogate acquisition selects candidates for expensive oracle evaluation.Autoresearch maintains a persistent artifact, whereas molecule and CDRH3 rounds generate or parameterise candidate pools before selection.Source: D Ablation Studies · p. 54

D.1 Pure LLM research loop

• The pure-agent ablation removes calibrated BO value to test how far reflection and log-reading can advance research without a surrogate posterior.The agent can maintain research state, repeatedly edit train.py, and change the search-space representation as mechanisms are discovered.Source: D Ablation Studies · p. 54

D.2 LDM test-time search

• LDM test-time-search ablations vary inner search budget, discovery mechanism, and acquisition choice while retaining the value model.On molecules and CDRH3, they test whether larger proposal and acquisition-selection budgets improve single-step design rounds.Source: D Ablation Studies · pp. 54–55

D.1 Pure LLM research loop

• 0.956 val bpb is the pure LLM loop’s plateau, above LDM’s 0.93421 result on autoresearch.The pure loop spans 875 no-BO, no-acquisition experiments and progresses through multiple research regimes before plateauing.Source: Figure 20 · p. 55

D.1 Pure LLM-based research loop

• The pure LLM loop forms hypotheses, edits and tests code, preserves improvements, and changes research regimes using reflection reports, labelled memories, and retrieved positive and negative examples.Its trajectory moves from dense scaling to sparse associative memory, dense-core rebalancing, and repeat-first frontier tuning under the runtime budget.Source: §D.1 · pp. 56–57, Table 4 · p. 56
• The pure LLM loop improves validation BPB from approximately 1.069 to 0.956, outperforming the no-discover baseline at 0.9767 but remaining behind LDM at 0.93421.After an initial sequence of breakthroughs, roughly 500 further experiments mostly perform local confirmation and grid search around the same frontier.Source: §D.1 · pp. 55–57
• Without posterior means, uncertainties, and acquisition scores, the LLM loop retains narrative memory rather than a searchable value landscape, limiting calibrated exploration after breakthroughs.LDM converts the same history into a surrogate posterior and acquisition value for local refinement and boundary-moving discovery steps.Source: §D.1 · p. 57

D.2 Test-time Search for LDM

• LDM test-time scaling varies only cheap inner-loop computation under a fixed outer evaluation budget, including proposer count, refinement branches, retained candidates, and acquisition function.The theory predicts gains when added candidates are evaluated by a calibrated value model rather than language plausibility alone.Source: §D.2 · p. 57

D.2.1 Small-molecule drug discovery

• proposer64 bo64 achieves the strongest final Pareto front, while increasing proposals without matching acquisition bandwidth produces smaller or unstable gains.Figure 21 reports mean Pareto hypervolume over expensive molecular evaluations for Qwen3.5-9B Direct-Softmax LDM-TTS, with shaded one-standard-deviation bands.Source: Figure 21 · p. 58, §D.2.1 · p. 58
• proposer32 bo32 dominates smaller molecular budgets through most of the run, while proposer64 bo64 makes the largest early jump and maintains the highest final hypervolume; proposer64 bo32 performs worse.K controls the breadth of the LLM SMILES reservoir, whereas M controls how much of that breadth the BO/EHVI loop can screen and retain.Source: §D.2.1 · pp. 57–58
• At small molecular search budgets, mean- or UCB-style scalarised acquisitions outperform strict EHVI, but EHVI overtakes them as the screening budget grows.EHVI requires enough bandwidth to identify sparse non-dominated points, whereas scalarised acquisitions can smooth optimization when screening is thin.Source: §D.2.1 · pp. 58–59, Figure 22 · p. 58

Acquisition functions

• Calibrated acquisitions, including EI and UCB variants, outperform mean-style and random baselines across five antigens at a proposer budget of 300.The result indicates that acquisition choice matters once enough candidates are available for Bayesian-optimization screening, even with weak biological priors.Source: §D.2.2 · p. 59

CDRH3 budget scaling

• The budget ablation varies only the cheap LLM-side proposal breadth while fixing the outer oracle budget and allowing acquisition scoring over all valid, deduplicated candidates.Budget N denotes N candidate CDRH3 sequences or neighbourhood samples generated before the next expensive oracle query.Source: §D.2.2 · p. 59
• Larger inference-time proposal budgets substantially lower the attainable binding-energy plateau across antigen targets, although the largest budget does not always win because of nonmonotonicity.Source: Figure 24 · p. 60, §D.2.2 · p. 60
• The smallest budget, 36, improves quickly but plateaus higher, while larger pools lower the attainable binding-energy plateau without additional oracle calls.The nonmonotonic target dependence is attributed to rugged sequence landscapes, finite seeds, noisy surrogates, and redundant or lower-quality regions in expanded pools.Source: §D.2.2 · p. 60

Implications

• Across single-step design, test-time scaling helps when added samples are exposed to a calibrated value model, with biological sequence spaces requiring broader coverage than richly pretrained SMILES spaces.In CDRH3 design, acquisition filtering converts a larger discrete reservoir into stronger best-so-far binding energy rather than selecting candidates by language plausibility alone.Source: §D.2.2 · p. 60

D.3 Hyperparameter ablations for LDM.

• Hyperparameter sweeps produce only weak sensitivity on antibody design: Direct-Softmax slightly favors low TLLM, Policy-Softmax marginally favors η = 4, and all error bars overlap.The curves report mean best-so-far Absolut energy over five seeds, with lower values better; temperature changes do not significantly alter final binding energy.Source: §D.3 · pp. 60–61, Figure 25 · p. 61

E Additional Fine-Tuning Analyses

• The appendix provides complementary fine-tuning evidence beyond the main-text results on AutoResearch, antibody design, and small-molecule discovery.These analyses supplement rather than repeat the main fine-tuning results.Source: E Additional Fine-Tuning Analyses · p. 61

E.1 Fine-Tuning as Value Distillation

• The three-stage workflow performs high-budget acquisition-guided search, translates numerical Bayesian-optimization evidence into semantic rationales, and fine-tunes a student proposer for fast low-budget inference.The supervision emphasizes reading evaluated history, diagnosing stagnation, and selecting information-adding moves rather than only producing domain-specific solutions.Source: Figure 26 · p. 62, §E.1 · p. 62, Table 5 · p. 63
• LDM fine-tuning distills epistemic acquisition value—the strategy for choosing scarce experiments under uncertainty—rather than merely reward values or high-reward designs.The pipeline compiles high-budget test-time search into an amortized student through reasoning-augmented data and supervised next-token prediction.Source: §E.1 · pp. 61–62, Figure 26 · p. 62
• The distilled student uses surrogate predicted mean and uncertainty to exploit proven scaffolds while exploring regions where activity predictions remain uncertain.In the KRAS G12D example, it preserves a strong molecular scaffold while varying polar groups and ring systems under uncertainty.Source: §E.1 · p. 63

E.2 Cross-Molecule Transfer without Reasoning Augmentation

• The no-reasoning transfer ablation keeps the LDM state and action schema, trains on acquisition-weighted source-task traces, and evaluates on a different molecular task.It tests whether acquisition-guided behavior transfers without the explicit research-progress rationale zt,i.Source: §E.2 · p. 64
• 22.279±3.828 versus 16.489±5.867 final hypervolume demonstrates positive cross-molecule transfer from acquisition-weighted fine-tuning without explicit reasoning augmentation.The source and target molecular tasks differ, so the improvement reflects transferred research-management policy rather than memorization of source-task molecules.Source: §E.2 · p. 64

E.3 Supplementary Quantitative Tables

• The supplementary tables retain omitted G12C results and collect the numerical cross-domain comparisons underlying the three main-text fine-tuning figures.Table 6 reports reasoning-augmentation ablation hypervolumes at evaluation budget 80 over five seeds, while Table 7 reports terminal metrics across three fine-tuning studies.Source: §E.3 · p. 64, Table 6 · p. 64, Table 7 · p. 64

E.4 In-Distribution Single-Task Fit

• All three single-task losses converge within a few dozen steps, with held-out curves closely tracking training curves across autoresearch, small-molecule, and antibody CDRH3 design.Figure 27 reports training loss as solid curves and held-out evaluation loss as markers, confirming faithful in-distribution fitting before acquisition-loop evaluation.Source: §E.4 · p. 64, Figure 27 · p. 65
• Held-out losses are 0.06 for autoresearch, 0.38 for small-molecule design, and 0.70 for antibody CDRH3 design.These absolute differences reflect domain-specific action spaces and target lengths rather than failure to fit.Source: §E.4 · p. 64
• The in-distribution fit is verified before evaluating the distilled policies in the acquisition loop.Training and held-out losses are tracked on reasoning-augmented training data for all three domains.Source: §E.4 · p. 64, Figure 27 · p. 65

E.5 Case-level Out-of-distribution Generalisation

• The out-of-distribution evaluation holds out antibody targets 1FBI X and 1H0D C, training on the other three targets and testing within the acquisition-guided loop.The held-out antigens’ sequences and binding landscapes were never seen during training.Source: §E.5 · p. 65
• Qwen3.5-9B reaches −110.57 on 1FBI X versus base −109.5 and in-distribution −113.4, and −95.4 on 1H0D C, surpassing in-distribution −94.6 and base −90.9.Source: §E.5 · p. 65
• The transfer result cannot be attributed to memorisation because the model never observed the evaluated antigens.The distilled acquisition policy therefore transfers to unseen targets.Source: §E.5 · p. 65

E.6 Task-level Out-of-distribution Generalisation

• On four of five antigens, the task-level OOD model matches the in-distribution model, reaching −113.1, −110.3, −106.6, and −104.2 on the respective targets.It falls short on 1H0D C, reaching −89.4 versus −94.6 for the in-distribution model and showing higher seed variance.Source: §E.6 · p. 65
• On 1H0D C, the task-level model selects an existing promising candidate, whereas the in-distribution model composes the narrow aromatic-hydrophobic motif de novo.The exception arises because the target’s optimum is unusually shallow and confined to a narrow basin requiring specific domain design.Source: §E.6 · p. 66
• The task-level experiment transfers acquisition-guided exploration and exploitation, but not de novo protein design, which requires in-domain data.This distinction explains success where selection alone finds optima and failure on 1H0D C, whose optimum lies in a narrow motif-specific basin.Source: §E.6 · p. 66

E.7 Distilling Acquisition Value into the Weights

• Hiding GP values makes the model infer epistemic state from observed outcomes, rather than reading posterior mean, uncertainty, and acquisition values directly.The adopted variant receives only tried candidates and real outcomes, while the alternative embeds Gaussian-process values in the prompt; both regimes reach the same acquisition decisions.Source: §E.7 · pp. 66–67
• The hidden-value model recognizes exhausted regions and initiates exploration from experience, reproducing the same explore decision as the GP-informed model.Its reasoning uses prolonged lack of improvement, low outcome spread, and weak patterns to open a new search dimension.Source: §E.7 · p. 67
• Only 1.3% of reasoning traces explicitly cite surrogate numbers, while 97.2% reason from observed outcomes such as stalls, noise, and best-so-far.Thus hiding GP values changes reasoning almost not at all while matching deployment, where the proposer receives no GP values.Source: §E.7 · p. 67
• −115.9 on 1FBI X and −93.1 on 1H0D C show that the GP-hidden proposer improves task-level out-of-distribution antibody design over GP-in-prompt reasoning.The GP-hidden proposer achieves the best average binding energy and best results on three of five antigens; Table 8 compares the two OOD augmentation regimes.Source: §E.7 · p. 67, Table 8 · p. 68