PaperlayerView exact revision on arXiv

Source-linked AI summary

Information-Dense Synthesis for Molecular Discovery

Kasper K. Jakobsen, Eli N. Weinstein

arXiv:2610.08495v1stat.MLcs.LGphysics.chem-phq-bio.BM

TL;DR

Sparse molecular activity maps make individual-candidate discovery difficult because most candidates fail. LIDS designs and tests stochastic mixtures as pools to encode information about the optimum. The paper reports order-of-magnitude empirical speedups and notes that the advantage over individual synthesis diminishes with infinitely many experiments.

  • Problem

    On sparse molecular activity maps, most individual candidates fail and existing lab-in-the-loop algorithms can perform no better than random guessing.

    Source: §1 · p. 1, §2 · p. 2

  • Method

    LIDS uses generative stochastic synthesis to design and test molecule mixtures as pools, optimizing experiments for information about the highest-activity molecule.

    Source: §1 · p. 1, §3.1 · p. 4

  • Results

    Order-of-magnitude speedups are reported across diverse simulated DNA and peptide activity landscapes.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Information-dense synthesis offers a complementary way to scale laboratory feedback by increasing the information carried by experiments rather than the datapoint count.

    Source: §7 · p. 12

    The advantage over individual synthesis decreases to zero as the number of experiments approaches infinity.

    Source: §5.2 · p. 8

Abstract

from arXiv · show

Machine learning can accelerate molecular discovery by designing molecules and planning experiments. However, many scientific challenges demand molecules with very rare properties, and in this sparse setting, existing algorithms offer little gain over random guessing. We propose a method to efficiently search large regions of molecular space using algorithmically controlled stochastic synthesis. Rather than design, make and test individual molecules, we design and make complex mixtures, test them as a pool, then deconvolute the molecule-activity map. We optimize synthesis to encode maximal information. Theoretically, this approach can reduce the number of experiments required to find the optimal molecule among $d$ candidates from $\mathcal{O}(d)$ to $\mathcal{O}(\log d)$ or $\mathcal{O}(1)$. In simulation, on estimated protein fitness landscapes, it finds active molecules with an order of magnitude fewer experiments than existing Bayesian optimization methods.

1 Introduction

Rare active molecules make individual-candidate lab-in-the-loop search uninformative, while LIDS designs and tests mixtures to encode information about the optimum. The paper reports theoretical speedups and order-of-magnitude empirical gains on simulated molecular landscapes.

1 Introduction

  • On sparse molecular problems, most individually tested candidates fail and existing algorithms can perform no better than random guessing.The passage attributes this to failed candidates providing almost no information about the optimum.Source: §1 · p. 1
  • LIDS designs, builds, and tests complex molecule mixtures instead of individual candidates, using algorithmically controlled stochastic synthesis.Synthesis is optimized to maximize information about the molecule with the highest activity.Source: §1 · p. 1
  • The LIDS recipe uses controlled stochastic synthesis, approximate Bayesian computation for mixture design, and pooled assays.The listed contributions cover synthesis control and approximate inference for designing and learning from mixtures tested as pools.Source: §1 · p. 2
  • The paper presents theoretical results showing exponential speedups over methods that design and test molecules individually.This is listed as a contribution of the paper.Source: §1 · p. 2
  • Order-of-magnitude speedups are demonstrated across diverse simulated DNA and peptide activity landscapes.The experiments compare LIDS with existing methods on simulated landscapes.Source: §1 · p. 2

2 Problem Setting

Molecular discovery seeks a molecule with desired activity, but sparse activity maps make individual-candidate search fundamentally difficult. The proposed alternative tests mixtures so experiments can convey information about many molecules at once.

2 Problem Setting

  • The optimization target is a molecule x⋆ that maximizes activity f over the explored molecular space.The passage gives peptide sequences and receptor activation as examples of the space and activity.Source: §2 · p. 2
  • With d candidates and exactly one uniformly unknown active molecule, random screening has expected regret (d − 1)/2, and no policy can do better.The result assumes a noiseless assay and no prior knowledge of the active candidate.Source: §2 · p. 2
  • Sparse activity maps leave individual tests with no information to extrapolate from: each reveals only whether that molecule works.The paper states that algorithms therefore cannot outperform random screening in the one-active-candidate setting.Source: §2 · p. 2
  • The proposed approach designs, synthesizes, and tests molecule mixtures so each experiment can carry information about many candidates.The overview contrasts pooled mixtures q with individual molecules x, particularly for sparse activity maps.Source: §2 · p. 3, Figure 2 · p. 3

3 Method

LIDS represents experiments as synthesizable molecule mixtures, measures their pooled activity, and selects mixtures to maximize information about the optimum. It combines a stochastic synthesis model, an assay model, sequential design, and inference from pooled observations.

3 Method

  • The assay model assumes mixture outcomes depend linearly on the weighted activities of constituent molecules, with drug interactions described as possible lower-order violations.For pure compounds, the mixture reduces to a point mass and recovers the standard noisy single-molecule assay.Source: §3 · p. 3

3.1 Synthesis models

  • The synthesis model maps experimental parameters ψ to achievable mixture measures qψ, with solid-phase synthesis producing sequence mixtures through randomized amino-acid additions.The described model combines multiple syntheses and their relative concentrations; the reported laboratory time and resources are effectively those for synthesizing S pure sequences.Source: §3.1 · p. 4

3.2 Assay models

  • The assay model focuses on Gaussian measurement noise with known standard deviation σ, while noting that Bernoulli or Poisson noise is also possible.Prior noise-level knowledge may come from control experiments.Source: §3.2 · p. 4

3.3 Designing mixtures

  • Mixture design maximizes expected information gain about the optimal molecule x⋆; standard Thompson sampling, UCB, and expected improvement can collapse to single-molecule designs and fail to provide major sparse-problem speedups.Because EIG is difficult to compute explicitly, the method uses a Barber-Agakov variational bound and an approximate posterior decoder.Source: §3.3 · p. 5

3.4 Inference from mixtures

  • Inference approximates the posterior over the molecule-activity model and returns the maximum a posteriori molecule as the best estimate of x⋆.The procedure uses automated approximate Bayesian inference and can switch from MCMC to stochastic variational inference as the posterior concentrates.Source: §3.4 · p. 6

4 Related Work

LIDS connects pooled molecular tests with group testing, compressed sensing, and information-gain experimental design, while extending these ideas to iterative learning of sequence-activity models. Its pooled testing does not require individual-molecule barcoding, and its causal formulation treats multiple molecules as affecting an outcome.

4 Related Work

  • LIDS extends group testing and compressed-sensing ideas to iterative experiments that learn sequence-activity models from model-designed mixtures.Prior work includes pooled tests in molecular discovery and stochastic DNA mixtures delivered into the same cell.Source: §4 · p. 6
  • Pooled testing does not require barcoding individual molecules, so LIDS is not restricted to DNA or DNA-encoded libraries.The passage notes potential application to other molecule classes.Source: §4 · p. 6
  • LIDS uses expected information gain to design mixtures, whereas related methods apply information gain to other optimization settings or optimize distributions for computational convenience.The cited comparisons include uncertainty autoencoders and distribution-optimization methods.Source: §4 · p. 6
  • The paper replaces the conventional y = f(x) + ϵ model with a hierarchical causal model in which multiple molecular subunits affect an outcome.The corresponding estimator is described as Bayesian distributional regression.Source: §4 · p. 6

5 Theory

Information-dense synthesis can rapidly search large molecular spaces, though experimental limitations slow it. The theory gives exponential speed-ups in idealized settings and describes trade-offs as assumptions are relaxed.

5.1 Exponential speed ups

  • 1 experiment can identify x⋆ under noiseless discrete activity, a well-specified model, and unrestricted choice of mixtures; O(1) optimization also holds for sparse real-valued activity.These are analytic examples with idealized assumptions.Source: §5.1 · p. 7
  • ⌈log2 d/E⌉ batched experiments find the active molecule in the 1-sparse binary-test setting with a uniform prior and a well-specified model.The EIG design performs a 2^E-ary search by testing mixtures containing half the remaining candidates.Source: §5.1 · p. 7
  • d = ∼10^60 small molecules could, the authors suggest, be searched with three 96-well plates, versus ∼10^60 individual tests.The estimate is based on the stated small-molecule-space size and Proposition 1.Source: §5.1 · p. 7

5.2 Limitations and tradeoffs

  • High noise can reduce EIG to zero, higher synthesis yield can increase it, and spreading mixtures trades higher signal entropy against the SNR advantage of a single molecule.Synthesis control must also achieve a high inner product between the mixture and the molecule-activity map being learned.Source: §5.2 · p. 7
  • As noise increases, the optimal trade-off between entropy and SNR shifts toward mixtures with smaller support dq.The pattern is reported for the numerical Example 3 comparison with information bounds.Source: §5.2 · p. 8
  • Mixtures can offer major early gains, but their advantage over individual synthesis vanishes asymptotically; the covariance result concerns learning all of fθ, not directly the optimum x⋆.The authors conjecture similar behavior for learning the optimum.Source: §5.2 · p. 8

6 Empirical Results

Across simulated sparse synthetic and protein-language-model landscapes, LIDS found optimal or active molecules in fewer experiments than individual-synthesis methods. Its performance advantage narrowed as assay noise increased, while tested synthesis errors had little effect until substantial corruption.

6 Empirical Results

  • Comparisons matched the number of syntheses to batch size to equalize synthesis cost, using batches of 12 designs for the synthetic oracle and 96 for PLM landscapes.When assay cost is the bottleneck, the authors note LIDS can use more syntheses per batch to reach higher expected information gain.Source: §6 · p. 8, §6.1 · p. 9, §6.2 · p. 11

6.1 Sparse synthetic oracle

  • At PSNR = 80 dB, LIDS found the synthetic optimum in a median of 2 rounds (under 8 in all 10 repeats), while individual methods usually failed within 100.The comparison is reported for the sparse synthetic oracle; the table reports rounds until finding the optimum.Source: §6.1 · p. 9, Table 1 · p. 9
  • LIDS designs complex stochastic syntheses that cover its remaining uncertainty at each round.The paper illustrates the designs and posterior beliefs over two rounds on the sparse synthetic oracle.Source: §6.1 · pp. 9–10

6.2 Protein language model oracle

  • On the common PLM landscape, LIDS found the global optimum in a median of 3 rounds versus 27 for GP methods; on RuBisCO, its median was 2 rounds.On RuBisCO, individual methods failed to find the optimum within 100 rounds in most repeats.Source: Table 2 · p. 9, §6.2 · p. 12
  • On SazCA, LIDS missed the unique global optimum but found nonzero-activity sequences within 5 rounds, while alternatives found none after 100.This distinguishes finding active molecules from finding the unique optimum.Source: §6.2 · p. 12
  • Removing stochastic synthesis eliminated LIDS’ advantage, and neither Thompson sampling with the same model and prior nor a GP with additional prior information recovered it.The ablations replace stochastic designs with their optimum or use individual synthesis.Source: §6.2 · p. 12
  • LIDS remained robust to model misspecification; synthesis errors substantially reduced performance only when 40% of each mixture component was corrupted.Performance degrades gradually toward individual synthesis as assay or synthesis noise increases.Source: §6 · p. 9, §6.2 · p. 12

7 Discussion

The discussion reports large speedups from information-dense stochastic synthesis on hard molecular discovery problems. It also notes that LIDS has only been evaluated in simulation and trades fewer wetlab experiments for substantially more computation.

7 Discussion

  • LIDS achieved very large speedups on hard molecular discovery problems by using information-dense stochastic synthesis instead of synthesizing and testing individual molecules.Source: §7 · p. 12
  • LIDS has so far been evaluated only in simulation, and uses much more computation than individual synthesis despite requiring fewer wetlab experiments.Scaling to larger molecules and more complex surrogate models will require accelerating Bayesian experimental design.Source: §7 · p. 12
  • LIDS offers a complementary scaling route through more precise synthesis control, higher yields, and higher assay signal-to-noise, increasing the information carried by datapoints.Source: §7 · p. 12

B Proof of Proposition 1

Under a uniform, noiseless sparse-oracle setting, random selection has minimum expected regret. The section also shows that common convex acquisition functions select a single molecule rather than a mixture, and describes a variational approach to EIG.

B Proof of Proposition 1

  • The randomized policy has expected regret (d −1)/2, and no policy achieves lower expected regret under the stated setting.Source: B Proof of Proposition 1 · p. 20

C Popular acquisition functions are not useful to optimize mixtures

D Derivation of variational lower bound on EIG

E.1 Architecture

  • The variational distribution maps a batch of experimental outcomes to position-wise independent categorical distributions over peptide residues.Source: §E.1 · p. 23

E.2 Training

The appendix describes model training and posterior approximation, alongside a sparse neural activity-map model and computational approximations used for mixture evaluation and optimization. The approximation of the activity maximizer is exact when the model filters are disjoint.

E.2 Training

  • The variational distribution parameters ϕ and mixture parameters ψ are jointly optimized by stochastic gradient ascent on ∇ψ,ϕ IBA(x⋆; ψ,ϕ), using AdamW with learning rate 0.01, momentum 0.9, and weight decay 10−5.Models are trained on 100000 θ samples drawn from the prior π(θ) or approximate posterior p(θ | D), for 1000 epochs for the sparse oracle or 5000 for the PLM oracle.Source: §E.2 · p. 23

F Details on Bayesian neural network fθ; F.1 Sparsity-inducing prior

F.2 Regularization

  • A regularization term is added to prevent bump-function mode collapse and encourage disjoint filters, which tightens the argmax approximation.Source: §F.2 · p. 24

F.3 Inner product ⟨fθ, qψ⟩

  • The mixture’s functional inner product can be computed in closed form and is evaluated in log space for numerical stability.Source: §F.3 · pp. 24–25

F.4 Approximation of x⋆

G Posterior approximation

  • Posterior inference uses NUTS, switching to mean-field SVI when r-hat exceeds 2, an indication that the posterior is too focused for efficient NUTS sampling.Source: G Posterior approximation · p. 26

H Proof of Example 1

In idealized noiseless settings, EIG-optimized mixtures can identify an optimum in one experiment for a discrete activity map or two parallel experiments for a 1-sparse map. The appendix also examines batched search, detection thresholds, information bounds, and the role of point-mass designs in maximizing SNR.

H Proof of Example 1

  • One EIG-optimized experiment identifies x⋆ when the activity map is discrete, the model is well specified, arbitrary mixtures are available, and the assay is noiseless.Source: H Proof of Example 1 · p. 26

I Speedup for noiseless 1-sparse molecule-activity maps

J Proof of Example 2

  • Batched EIG search partitions candidates into approximately equal-sized sets, reducing the remaining possible optima across successive rounds.Source: J Proof of Example 2 · p. 27

K Information gain under a detection threshold

L Proof of Proposition 2

  • The EIG is bounded both by the prior entropy of the signal’s finite set of values and by the capacity of an additive Gaussian-noise channel.Source: L Proof of Proposition 2 · p. 28

L.1 SNR(q) is maximized at ¯q = δx

  • The variance of mixture activity is maximized at an extreme point of the design space, corresponding to a single-molecule delta design.Source: §L.1 · p. 29

M Proof of Example 3; EIG bound from Proposition 2

True MI

  • The EIG can be expressed as the expected KL divergence between the prior and posterior.Source: True MI · p. 30

N Proof of Proposition 3

The proof targets information about the activity map and argues that restricting sampling to delta-function measures preserves the asymptotic-variance optimum; the remaining passages specify simulation and confidence-interval procedures.

N Proof of Proposition 3

  • Restricting sampling to delta-function measures preserves the asymptotic-variance optimum over all positive measures.The proof targets information about the activity map, assuming Gaussian observation noise and measures with a fixed 1-norm.Source: N Proof of Proposition 3 · pp. 30–31

O Details on sparse synthetic oracle simulations

  • The sparse synthetic oracle retains molecules within Hamming distance 1 of the optimum, where every single mutation has lower activity.It is constructed by sparsifying prior activity-map draws and setting the maximum activity to one.Source: O Details on sparse synthetic oracle… · p. 31

P CI calculation

  • A logit transformation with the delta method keeps confidence intervals within [0, f(x⋆)].The naive standard error can otherwise produce intervals outside the possible activity range.Source: P CI calculation · p. 32

Q Details on PLM oracle simulations

Q.1 Importance sampling ⟨f, q⟩

  • The experiment expectation under q is approximated by importance sampling from a defensive mixture of q and the PLM distribution.The proposal assigns half its mass to each distribution.Source: §Q.1 · p. 32

R Ablation of posterior approximation

Figure 9 compares LIDS posterior approximations across different posterior-sample counts M under the Sec. 6.2 setup.

R Ablation of posterior approximation

  • Figure 9 compares LIDS posterior approximations across different posterior-sample counts M under the Sec. 6.2 setup.Source: Figure 9 · p. 33

Related papers

This paper cites

Loading 2610.08495v1…