Source-linked AI summary
Conditioning by adaptive sampling for robust design
David H. Brookes, Hahnbeom Park, Jennifer Listgarten
TL;DR
The paper addresses design with stochastic, potentially black-box property oracles whose predictions may become pathological away from training data. It proposes Conditioning by Adaptive Sampling, which estimates the conditional distribution of realistic inputs given desired properties. The method yields competitive results against alternative approaches, including on protein fluorescence, while supporting discrete spaces and non-differentiable oracles.
Problem
Direct oracle optimization can exploit unreliable out-of-distribution predictions and generate unrealistic inputs, motivating prior information about accurate or realistic designs.
Method
CbAS uses model-based adaptive importance sampling to estimate a prior over realistic inputs conditioned on desired property events, including rare events.
Results
The paper reports competitive results compared with alternative solutions, including approaches requiring differentiable oracles, on a protein fluorescence problem.
Takeaways & Limitations
The method enables design in discrete spaces with potentially non-differentiable oracle models while incorporating prior information and handling rare conditioning events.
Takeaways & Limitations
CbAS assumes access to examples from an underlying distribution representing oracle-training inputs or realistic examples, which are used to fit the prior.
Abstract
from arXiv · showhide
We present a new method for design problems wherein the goal is to maximize or specify the value of one or more properties of interest. For example, in protein design, one may wish to find the protein sequence that maximizes fluorescence. We assume access to one or more, potentially black box, stochastic "oracle" predictive functions, each of which maps from input (e.g., protein sequences) design space to a distribution over a property of interest (e.g. protein fluorescence). At first glance, this problem can be framed as one of optimizing the oracle(s) with respect to the input. However, many state-of-the-art predictive models, such as neural networks, are known to suffer from pathologies, especially for data far from the training distribution. Thus we need to modulate the optimization of the oracle inputs with prior knowledge about what makes `realistic' inputs (e.g., proteins that stably fold). Herein, we propose a new method to solve this problem, Conditioning by Adaptive Sampling, which yields state-of-the-art results on a protein fluorescence problem, as compared to other recently published approaches. Formally, our method achieves its success by using model-based adaptive sampling to estimate the conditional distribution of the input sequences given the desired properties.
1. Predictive-model based design
Predictive-model-based design can replace costly laboratory measurements with oracle-guided search, but direct optimization may exploit unreliable out-of-distribution predictions. CbAS instead combines oracle information with a prior over realistic or training-like inputs.
- Motivation: Directed Evolution relies on repeated, expensive laboratory measurements to identify and reproduce high-performing protein variants.Computational approaches can replace property measurements with regression-model oracles.
- Motivation: Directly optimizing predictive oracles can produce unrealistic sequences because many models behave pathologically far from their training data.The prior may encode either regions where the oracle is accurate or what constitutes a realistic input.
- Formal framing: Design is formulated by modeling p(x, y) = p(y|x)p(x) and sampling inputs from a conditional distribution such as p(x|y ≥ ymax).The prior p(x) can be learned with generative models trained on realistic examples.
- CbAS: Conditioning by Adaptive Sampling uses a statistical, prior-aware framework that supports black-box oracles, discrete inputs, and rare desired events.Its approach draws on conditional density estimation, model-based optimization, and Cross Entropy Methods.
- CbAS: Compared with related approaches, CbAS avoids differentiating the oracle and back-propagating through discrete inputs while explicitly incorporating prior information.The paper distinguishes this design objective from Bayesian Optimization, which acquires new ground-truth labels.
3. Methods
The method estimates a prior distribution conditioned on desired property events, including rare maximization events. It uses adaptive importance sampling with progressively relaxed conditions and a search model updated from weighted samples.
- Problem setup: CbAS seeks inputs X with high probability of satisfying a property desideratum, including maximization or specification objectives.The method is especially described for discrete sequence design but also applies to real-valued inputs.
- Problem setup: The oracle p(y|x) supplies a property distribution, from which event probabilities such as P(S|x) = p(y ≥ γ|x) can be computed.For maximization, S contains property values at least as large as a target maximum; specification conditions on a target value.
- Problem setup: The prior p(x|θ^(0)) is a generative model fitted to examples representing the oracle’s training distribution or realistic inputs.The desired conditional is approximated with a sufficiently rich search model q(x|φ).
- Conditional estimation: CbAS estimates the desired conditional by minimizing KL divergence between the target conditional and a parameterized search distribution.This objective balances satisfying the conditioning event with adherence to the prior.
- Rare events: Naive Monte Carlo estimation is unreliable when the desired event is rare because P(S|x) is vanishingly small for most prior samples.The resulting estimate has high variance and may require arbitrarily many samples.
- Adaptive sampling: CbAS overcomes rare-event sampling with iterative adaptive importance sampling, using relaxed conditions S^(t) and proposal distributions r^(t)(x) that gradually approach the desired event.After initialization, each proposal uses the previous search model, while thresholds are updated from sampled property values.
4. Experiments
The experiments evaluate CbAS on illustrative one-dimensional problems and simulated protein fluorescence design, comparing it with alternative methods under a fixed sequence budget. CbAS approximates desired conditional distributions and avoids the untrustworthy oracle regions that mislead methods without a prior.
- Illustrative example: The experiments use one-dimensional toy examples to study prior influence, oracle pathologies, and whether CbAS estimates desired conditional densities.The toy ground truth is a superposition of two unnormalized Gaussian bumps, and the goal is to maximize the ground-truth property using only an oracle and a prior.
- Illustrative example: The smaller-data oracle increases where the ground-truth function decreases, while CbAS assigns little conditional density to regions where the oracle is biased.The authors note that the prior can sometimes lead the search astray, but the example illustrates how CbAS addresses oracle pathology.
- Illustrative example: After 50 iterations with Q = 1 and M = 100, CbAS search distributions converge to close approximations of the target conditional distributions in both toy cases.Convergence is assessed qualitatively and through KL divergence between search and target distributions.
- Protein fluorescence: The protein fluorescence experiments compare CbAS with seven alternative approaches on a real fluorescence dataset using simulations with a known ground truth.The comparison includes methods requiring differentiable oracles and methods using VAE or no input prior.
- Protein fluorescence: CbAS results are relatively insensitive to the quantile update parameter Q in supplementary experiments.The main experiments use Q = 1.
- Protein fluorescence: The methods are compared under a constant sequence budget, with three randomly initialized runs per oracle model and a budget of 10,000 sequences.The reported protein-design results average the three runs.
- Protein fluorescence: Methods without a prior are led into untrustworthy oracle regions, whereas CbAS achieves the highest ground-truth fluorescence and the correct oracle-prediction darkness ordering.The figure evaluates ground truth across oracle-prediction percentiles; higher bars indicate better fluorescence maximization.
5. Discussion
The paper presents CbAS as a method for discrete design with potentially non-differentiable oracles, using conditional density modelling and latent-variable importance sampling. It reports competitive results against alternative approaches, including methods requiring differentiable oracles.
- Contributions: CbAS addresses discrete design problems and can use potentially non-differentiable oracle models.The method avoids requiring oracle differentiation or back-propagation through discrete inputs.
- Contributions: CbAS performs approximate conditional density modelling for rich model classes, including settings involving rare events.The paper identifies this as a central methodological contribution.
- Contributions: CbAS leverages latent-variable structure to achieve exact importance sampling when model densities cannot be computed exactly.This is presented as a separate contribution of the method.
- Results: CbAS yields competitive results compared with alternative solutions, including approaches that require differentiable oracles.The comparison is reported as part of the paper’s final contribution.
Supplementary Information: Conditioning by adaptive sampling for robust
The supplied passage identifies the paper’s authors: David H. Brookes, Hahnbeom Park, and Jennifer Listgarten.
- Authorship: The paper lists David H. Brookes, Hahnbeom Park, and Jennifer Listgarten as authors.No experimental or methodological content is provided in this passage.
S1. Algorithm
Algorithm 1 implements CbAS by scoring samples with an oracle, weighting them by oracle-tail probabilities, retraining a generative model, and iterating with a relaxed threshold.
- Iteration: Each iteration samples inputs from the current generative model, evaluates expected oracle values, and sets the threshold to the Qth percentile of scores.The threshold is denoted γ(t).
- Reweighting: Samples are reweighted by one minus the oracle CDF at the current threshold before the generative model is retrained.The updated model parameters are produced by weighted training data.
- Inputs: CbAS takes an oracle, an oracle CDF, generative-training and generative-probability procedures, plus optional Q and M parameters.Q controls threshold relaxation and M controls the number of generated samples per iteration.
- Iteration: The algorithm repeats the sampling, scoring, weighting, and retraining cycle until returning the generated set and weights.The procedure increments the iteration counter after retraining.
S2. Generalization to specification
CbAS extends from property maximization to property specification by conditioning on a narrow interval around a target value and adaptively shrinking intermediate intervals.
- For a target value y0, the desired set is an infinitesimally small interval S = [y0 − ϵ, y0 + ϵ].Here, ϵ is a small positive value.
- CbAS retains the maximization procedure while centering intermediate sets S(t) on y0.The interval width, rather than a threshold, is updated at each iteration.
- Each width γ(t) is set to the Qth percentile of the absolute deviations |yi − y0| from sampled expected property values.The sampled inputs follow the current search distribution p(x|θ(t)).
S3. Generalization to multiple properties
CbAS generalizes to multiple properties under conditional independence, and its updates can be adapted for models that require variational or other approximate inference.
- Multiple properties: Multiple properties Y1, ..., YK can be handled with corresponding desired sets S1, ..., SK when the properties are conditionally independent given X.Each property has an independent oracle.
- Multiple properties: The joint event probability is formed from the individual P(Si|x) terms and substituted for P(S|x) and P(S|θ) in the CbAS equations.This recovers the CbAS procedure for multiple properties.
- Approximate inference: When maximum likelihood cannot be solved exactly, CbAS remains applicable if an approximate inference procedure such as variational inference is available.The update can also be modified for models permitting other approximate MLE methods.
- Approximate inference: Variational inference replaces the maximum-likelihood objective with a lower bound involving a latent variable z and approximate posterior r(z|x, ψ).This bound underlies the variational CbAS update.
- Approximate inference: The variational bound is tight when the approximate posterior is rich enough to exactly match the true model posterior.This motivates a variational-inference-specific update equation.
- Approximate inference: For VAE-based search distributions, the update gives time dependence to the approximate-posterior parameters ψ.The same derivation can produce approximate updates for weighted maximum-likelihood methods.
S5. Using Samples from Previous Iterations
CbAS uses importance sampling to incorporate samples from previous iterations, with a prior-based weight structure that preserves their usefulness more effectively than in several alternative weighted maximum-likelihood methods.
- Importance sampling allows weighted maximum-likelihood updates to reuse samples generated in previous iterations.The objective then includes current and earlier samples.
- Exact marginal likelihoods are unnecessary when an ELBO-based method can calculate the required likelihood ratio approximately.This applies to intractable latent-variable models.
- As the gap between current and sampling iterations grows, likelihood ratios may become extremely small.This is a potential limitation when incorporating increasingly old samples.
- Very small likelihood ratios can reduce the effective sample size of the Monte Carlo estimate to little more than that of an iteration without old samples.
- CbAS-specific reuse: In CbAS, the weighting uses the prior density rather than the current search-model density in the likelihood ratio.This differs from the general importance-sampling expression used by other weighted methods.
- CbAS-specific reuse: CbAS old-sample weights change across iterations only through the updated relaxed desired property S(t).Their impact therefore does not diminish merely because more iterations have elapsed.
- CbAS-specific reuse: This weighting structure suggests retaining old samples may be more useful in CbAS than in DbAS, RWR, and CEM-PI.The comparison concerns the persistence of old-sample impact under their respective weighting schemes.
S6. Experimental Details
The experiments compare CbAS with related weighted maximum-likelihood methods using specified neural-network oracles and generative-model architectures, including VAEs and GAN-related variants.
- Compared methods: RWR, DbAS, and CEM-PI update generative-model parameters by weighted maximum likelihood over samples drawn from the current model.
- Compared methods: The compared methods differ in how they define sample weights.
- Compared methods: CEM-PI weights samples according to whether their probability of improvement exceeds the iteration-specific threshold β(t), with Q set to 0.8.The probability-of-improvement function is denoted PI(x).
- Oracle and models: The VAE encoder maps sequences through Dense(50) and Dense(40) to latent mean and log-variance vectors of length 20.The decoder maps the 20-dimensional latent input through Dense(50), Dense(L*20), reshape, and column softmax.
- Oracle and models: For the Gomez-Bombarelli methods, a predictive network maps the latent space to property space using Input(20) → Dense(50) → Dense(1).
- Oracle and models: FB-VAE uses the 80th percentile of initial-training-set property values as its threshold for assigning binary sample weights.This setting gave the best performance and was used for all reported tests.
- Oracle and models: FB-VAE samples sequence realizations from the VAE decoder distribution rather than taking decoder argmax outputs.This modification accommodates a VAE generator in the FB-GAN framework.
- Oracle and models: For GFP fluorescence, the oracle is an ensemble of neural networks with architecture Input(L, 20) → Flatten(L*20) → Dense(20) → Dense(2).The ensembles are trained without adversarial examples.
S7. Fluorescence data set
The fluorescence dataset retains 34,256 proteins from the higher-fluorescence mode of an initially bimodal set. Supplementary figures examine oracle behavior across oracle ensembles and CbAS sensitivity to Q.
- Fluorescence data set: 34,256 protein sequences were retained from 50,000 fluorescent proteins, selecting the higher-fluorescence mode of the bimodal distribution.The retained set corresponds to the higher-fluorescence mode.
- Oracle comparisons: Figure S1 compares method trajectories across ensemble-of-one, ensemble-of-five, and ensemble-of-20 oracles.Each oracle is shown across three random initializations.
- Oracle behavior: Figure S2 compares ground-truth fluorescence with mean oracle predictions for three oracle ensembles across two test sets.The test sets contain 1,851 below-20th-percentile sequences and 27,404 sequences at or above the 20th percentile.
- Parameter sensitivity: Figure S3 compares CbAS trajectories across Q values using the ensemble-of-one oracle.The figure reports relative insensitivity to Q over the range [0.5, 1.0].