PaperlayerView exact revision on arXiv

Source-linked AI summary

SABLE: Scalable Affine Bayesian Latent-space Exploration

Erik Bodin, Lisa Maria Kreusser, Donney Fan, Henry Moss, Geoff Pleiss, Nicolas Durrande

arXiv:2610.31495v1cs.LGcs.AIstat.ML

TL;DR

High-dimensional latent spaces and expensive evaluations make optimisation difficult when the evaluation budget is small relative to dimension. SABLE combines a Bayesian affine surrogate with reduced-dimensional acquisition, scheduled off-span exploration, and recompression. Across synthetic and scientific tasks, it reports high sample efficiency up to D = 40,500 and tens-of-milliseconds decisions on timing benchmarks.

  • Problem

    High-dimensional latent spaces can greatly exceed the affordable evaluation count, leaving too little data to identify a flexible global surrogate without added structure.

    Source: §1 · p. 2

  • Method

    SABLE uses a Bayesian affine working model, decomposes queries into in-span and off-span components, schedules the off-span norm, and recompresses observations to O(M) pseudo-observations.

    Source: §1 · p. 2

  • Results

    Across synthetic and scientific tasks, SABLE achieves high sample efficiency up to D = 40,500, with tens-of-milliseconds decisions on timing benchmarks.

    Source: §5 · p. 10

  • Takeaways & Limitations

    SABLE handles latent optimisation in the N < D regime without restricting admissible queries to a learned subspace.

    Source: §5 · p. 10

    SABLE assumes an isotropic latent distribution and a useful affine signal; objectives dominated by higher-order or quantised structure are a weaker regime, and its schedule requires Nmax in advance.

    Source: §5 · p. 10

Abstract

from arXiv · show

Complex objects such as engineering designs and proteins are often difficult to optimise because their design spaces exhibit high-dimensional structure and objective evaluations are expensive. Generative models provide continuous latent representations, but their latent spaces can have thousands to tens of thousands of dimensions, creating a regime where the latent dimension $D$ greatly exceeds the evaluation budget. We introduce SABLE, a Bayesian optimisation method for high-dimensional latent spaces with an isotropic distribution and limited budget. SABLE uses a Bayesian affine surrogate for prediction and acquisition, treating higher-order structure as surrogate discrepancy. After $N$ observations, the slope update has dimension at most $N-1$, yielding a decomposition of each query into an in-span component optimised by expected improvement (EI) in at most $N$ dimensions and an off-span component governed by a scalar norm. We show that the off-span norm implicitly selected by affine EI can lead to underexploration, and SABLE instead prescribes it using a finite-horizon schedule, without restricting search to a learned subspace. A recompression scheme reduces information to $\mathcal O(M)$ pseudo-observations, bounding refitting cost by $\mathcal O(M^2D+M^3)$ while keeping parameter inference, prediction, and acquisition decisions to tens of milliseconds. Across synthetic benchmarks and latent-space optimisation tasks, SABLE achieves high sample efficiency and reduces decision time by several orders of magnitude at large budgets.

1 INTRODUCTION

Latent-space optimisation can make difficult scientific and engineering design problems more tractable, but expensive evaluations and very high latent dimensions leave too little data to identify a flexible global surrogate. SABLE uses a Bayesian affine working model, decouples in-span EI from scheduled off-span exploration, and compresses observations to bound refitting costs.

1 INTRODUCTION

  • When N ≪ D, limited evaluations cannot identify even a global linear model without additional structural assumptions.Scientific latent dimensions cited include D = 6,000 and D = 40,500.Source: §1 · p. 2
  • SABLE uses a Bayesian affine working model, treating higher-order variation as surrogate discrepancy while keeping the full query domain admissible.Uncertainty in the discrepancy and signal variance scales is marginalised.Source: §1 · p. 2
  • EI selects the in-span component while a finite-horizon schedule prescribes the off-span norm, avoiding affine EI’s tendency toward underexploration.The decomposition permits EI optimisation in at most N coordinates independently of D.Source: §1 · p. 2, Figure 1 · p. 2
  • Recompression reduces accumulated observations to O(M) pseudo-observations, making posterior-fitting storage and per-round refitting depend on M rather than N.The full domain of admissible future queries is preserved.Source: §1 · p. 2, Figure 1 · p. 2

2 PROBLEM SETUP AND AFFINE WORKING MODEL

The setup formulates optimisation over latent inputs from a pretrained generator with an isotropic Gaussian prior and an expensive black-box objective. A Bayesian affine component supplies predictions and acquisitions, while higher-order variation is treated as discrepancy rather than ruled out.

2 PROBLEM SETUP AND AFFINE WORKING MODEL

  • The latent objective is f(g(z)), where g is a pretrained generator and f is an expensive black-box objective; the latent prior is isotropic Gaussian.The evaluation budget is specified in advance.Source: §2 · p. 3
  • The affine working model uses degree-zero and degree-one components, with higher-order structure collected as discrepancy rather than assuming the objective is globally affine.Restricting optimisation to the sphere removes radial input-norm variation.Source: §2 · p. 3
  • Because signal and discrepancy scales can be weakly identified when N ≪ D and affect predictive uncertainty, SABLE marginalises over their variance hyperparameters.Posterior expectations are approximated using stratified samples from the hyperparameter posterior.Source: §2 · p. 3

3 SABLE: FINITE-DATA GEOMETRY AND CONTROLLED EXPLORATION

SABLE exploits the low-dimensional geometry informed by limited observations: acquisition separates an in-span direction from a scalar off-span norm, while a finite-horizon schedule controls exploration. Recompression bounds model-refitting growth as observations accumulate.

3.1 FINITE-DATA POSTERIOR GEOMETRY

  • After N observations, the affine slope posterior changes in at most N−1 directions, independently of ambient dimension D.The remaining orthogonal directions retain their prior mean and variance.Source: §3.1 · pp. 3–4

3.2.1 REDUCED ACQUISITION GEOMETRY

  • Rotational symmetry makes off-span orientations predictively equivalent, so acquisition can optimise over the design-span coordinates without restricting the query domain.The design span has dimension at most N.Source: §3.2.1 · p. 4

3.2.2 EI WITH PRESCRIBED EXPLORATION

  • Affine EI’s implied off-span allocation can depend strongly on fitted variance scales, which may be unreliable when higher-order variation acts as discrepancy.SABLE instead prescribes the off-span norm with a finite-horizon schedule and lets EI select the in-span direction.Source: §3.2.2 · p. 5
  • The schedule trades current in-span exploitation against directional information usable over remaining evaluations, annealing off-span exploration to zero at the horizon.Its reference-resolution parameter is chosen independently of fitted variance scales; practice uses different values for smooth versus noisy or quantised objectives.Source: §3.2.2 · p. 5

3.3 BOUNDED-COST MODEL REFITTING

  • Recompression reduces accumulated observations to O(M) pseudo-observations and preserves centred slope–response and intercept cross-statistics.Discarded spectral information is summarised, with a log-determinant correction used in approximate marginal-likelihood evaluation.Source: §3.3 · pp. 5–6

3.4 COMPUTATIONAL SCALING

  • With compression budget M, storage and refitting costs become O(MD) and O(M^2D+M^3), rather than O(ND) and O(N^2D+N^3) without compression.For fixed M, posterior-fitting storage and refitting cost are independent of N once recompression is active.Source: §3.4 · p. 6

4 EXPERIMENTS

Experiments test compression and off-span exploration on controlled objectives, then evaluate SABLE on long-horizon and scientific latent-space tasks. SABLE performs strongly across the scientific problems and retains low decision costs at large budgets.

State compression

  • At N = 5,000, M = 100, 300, and 500 differed by at most 0.003 in best-so-far value; M = 10 trailed uncapped by 0.30–0.32 on non-affine objectives.For M = 100, 300, and 500, the mean gaps from uncapped were 0.003, 0.047, 0.035, and 0.022 across the affine-to-severe cases, without systematic growth over N = 1,000–5,000.Source: §4 · p. 6, Figure 2 · p. 7

Separating exploration from model fit

  • On non-affine objectives, MLE overfit higher-order structure and EI explored mostly fresh directions, while MAP and marginalisation assigned almost no norm off-span.With MLE, the recovered degree-one component was near zero; MAP and marginalisation recovered it better but tended to confine search to the early span.Source: §4 · p. 7
  • The prescribed schedule improved degree-one recovery and optimisation under MAP or marginalisation, but could not compensate for poor surrogate fit under MLE.Driving the schedule with the fitted variance ratio remained below the SABLE schedule on non-affine objectives.Source: §4 · pp. 7–8

Controlled long-horizon evaluation

  • At D = N = 10,000, SABLE attained the highest endpoint value on all three non-affine controlled objectives.Figure 3 reports mean best-observed-value curves and endpoint attained values over ten runs.Source: Figure 3 · p. 8, §4 · p. 8

Scalability

  • At N = D = 10,000, M = 300 required 5.5 ms per decision versus 79.5 s for the uncapped representation; GP baselines took 430–900 s per decision at their 48-hour-limited budgets.On the scientific tasks, SABLE's compression had no cost to sample efficiency; its no-schedule ablation matched or exceeded Fan et al. (2026) on every task while deciding about 90 times faster.Source: §4 · pp. 8–9

Scientific latent-space optimisation

  • SABLE attained the highest mean best observed value on every scientific problem except Peptides; on Calorimeter DS3 it scored 0.66 versus 0.55 for sep-CMA-ES.The table reports best observed values over ten seeds; sep-CMA-ES leads on Peptides.Source: §4 · pp. 8–9, Figure 4 · p. 9

Extending the budget without slowdown

  • At N = 20,000 on Calorimeter DS3, SABLE led sep-CMA-ES on all ten runs at every budget and matched its final value at about half the budget.SABLE's per-round cost is independent of N once compression budget M is reached.Source: §4 · p. 10

5 CONCLUSION

SABLE combines data-sized acquisition, controlled off-span exploration, and pseudo-observation compression without restricting queries to a learned subspace. It achieves sample efficiency up to D = 40,500, while its stated scope assumes isotropic latents and a useful affine signal.

5 CONCLUSION

  • Across synthetic and scientific tasks, SABLE achieves sample efficiency up to D = 40,500 with tens-of-milliseconds decisions on timing benchmarks.It combines data-sized acquisition, controlled off-span exploration, and bounded-cost pseudo-observation compression without restricting queries to a learned subspace.Source: §5 · p. 10

A.1 SPHERICAL AFFINE DECOMPOSITION

  • The spherical-harmonic decomposition of F into degree-zero, degree-one, and higher-order components is unique and does not require the higher-order residual to be small.Source: §A.1 · p. 14

A.2 PRIOR ROTATIONAL SYMMETRY

  • The affine likelihood is equivariant under simultaneous rotations of designs and slope coefficients, so the prior working model has no preferred coefficient orientation.Conditioning can introduce directional preference only through the evaluated-design geometry.Source: §A.2 · p. 14

A.3 ANALYTICAL TREATMENT OF THE INTERCEPT

  • With a flat intercept prior, the conditional intercept posterior is Gaussian, and integrating the intercept contributes a q_N(θ)^−1/2 factor to the marginal likelihood.Source: §A.3 · p. 14

A.4 HYPERPOSTERIOR DISCRETISATION

  • The hyperposterior is approximated on a fixed grid over log signal and discrepancy scales, then compressed to K equally weighted atoms by deterministic stratified resampling for acquisition.Source: §A.4 · p. 15

A.5 RECOMMENDATION READOUT

  • The recommendation maximises the hyperparameter-marginalised posterior mean of the affine surrogate over the sphere; if the posterior slope mean is zero, the best observed query is returned.Source: §A.5 · p. 15

B.1 DATA-INFORMED COEFFICIENT SUBSPACE

Conditional on covariance parameters, observations update the affine slope in at most N−1 directions, and this rank bound persists under more general Gaussian residual covariance. The resulting rotational symmetry and uncertainty contrast clarify how the data-informed subspace relates to off-span exploration.

B.2 GENERAL RESIDUAL-COVARIANCE EXTENSION

  • With a general Gaussian residual covariance, posterior slope updating remains confined to at most N−1 directions, although the informed directions and metric can change.Source: §B.2 · p. 17

B.3 POST-DATA ROTATIONAL SYMMETRY

  • After observing data, subspaces of the same dimension within the orthogonal complement of the design span are posterior-equivalent under the affine working model, including after variance marginalisation.Source: §B.3 · p. 17

B.4 EFFECTIVE INFORMED DIMENSION AND UNCERTAINTY CONTRAST

  • The effective informed dimension is strictly below the algebraic rank for finite positive signal-to-noise ratio, distinguishing how many directions are updated from the strength of those updates.Source: §B.4 · p. 18
  • A large fitted signal-to-noise ratio can make off-span queries attractive, but omitted higher-order structure can distort that ratio, motivating a separate finite-horizon reference schedule.Source: §B.4 · p. 18

C SEQUENTIAL ACQUISITION IN DATA-SIZED COORDINATES

SABLE performs EI acquisition in data-sized coordinates while prescribing the off-span norm with a finite-horizon schedule. The schedule uses a scalar reference model and an approximate objective, and selected queries are reconstructed in the ambient space.

C SEQUENTIAL ACQUISITION IN DATA-SIZED COORDINATES

C.1.1 WORKING BASIS

  • An orthonormal basis for the design span is obtained from the positive-eigenvalue directions of the Gram matrix.Source: §C.1.1 · p. 18

C.1.2 PREDICTIVE MEAN AND VARIANCE

  • For a query on the sphere, the conditional predictive distribution depends on its design-span coordinates only.Source: §C.1.2 · p. 19

C.2 CANDIDATE DIRECTIONS

  • The union of stationary-direction families for individual hyperparameter atoms is a candidate construction, not an exhaustive characterization of stationary points of marginal EI.Candidates are rescored using the full hyperparameter-marginalised acquisition.Source: §C.2 · p. 19

C.3 ACQUISITION SCORING AT THE PRESCRIBED NORM

  • At the prescribed off-span norm, each candidate direction is scored with full marginal EI, and the highest-scoring candidate is selected.Source: §C.3 · p. 19

C.4 QUERY RECONSTRUCTION

  • Because the predictive distribution and acquisition value are invariant to off-span orientation, reconstruction uses a fresh orthogonal direction only to materialise the query.Thus ambient dimension enters at reconstruction, not acquisition optimisation.Source: §C.4 · p. 20

D FINITE-HORIZON REFERENCE SCHEDULE FOR OFF-SPAN EXPLORATION

D.2 FINITE-HORIZON OBJECTIVE

  • The schedule approximates exploitation with a root-mean-square proxy, omits further information acquisition by later evaluations, and ignores residual in-span uncertainty.Source: §D.2 · p. 21

D.3 SCHEDULE PROPERTIES

The schedule makes off-span exploration decrease over a finite horizon, with its behavior controlled by a reference-resolution parameter. The section also describes how local resolvability informs that parameter and how bounded-state refitting preserves or approximates the affine model’s information.

D.3 SCHEDULE PROPERTIES

  • The prescribed off-span allocation never increases, and exploration cannot restart after it stops; it is zero in the final round.While exploration remains active, the schedule’s reference information increases.Source: §D.3 · p. 21

D.4 INTERPRETATION OF THE REFERENCE RESOLUTION PARAMETER

  • λref represents assumed information from a fresh-direction probe, not the fitted signal–discrepancy ratio or a direct prescription of off-span norm.Larger λref delays the earliest possible termination, allowing small but persistent probes.Source: §D.4 · p. 22

D.5 LOCAL RESOLVABILITY AND REFERENCE RESOLUTION REGIMES

  • Local response may be smooth, noise-dominated, or unresolved at small scales, motivating diagnostics for whether fine perturbations reveal directional information.A small perturbation can produce no observed change for locally constant or discretely decoded objectives.Source: §D.5 · p. 23, §D.5.1 · p. 23

D.5.1 CHARACTERISING LOCAL RESOLVABILITY

  • Q tracks how objective change per unit perturbation varies across scales, while Z measures the fraction of probed directions with exactly zero change.The diagnostic uses 40 evaluations; in the reported validation, resolved objectives had median Q ≈1 and noise-dominated objectives median Q ≈8.Source: §D.5.1 · pp. 23–24

D.5.2 REFERENCE-RESOLUTION REGIMES

  • The fine-resolution regime uses λref = 10^5 for smooth, well-resolved objectives, while the coarse regime uses λref = 10 for objectives with obscured or unresolved fine-scale changes.The coarse value follows a criterion requiring a full off-span probe to resolve at least 90% of reference directional variance; neither value is a tuned optimum.Source: §D.5.1 · p. 24, §D.5.2 · pp. 24–25

E BOUNDED-COST MODEL REFITTING

  • The data-informed posterior rank can grow with the evaluation horizon while remaining independent of ambient latent dimension.The section uses this spectral ordering to motivate bounded-state online refitting.Source: E BOUNDED-COST MODEL REFITTING · p. 25

E.1 IDEAL SPECTRAL TRUNCATION

  • Ideal spectral truncation preferentially retains directions with larger eigenvalues because they produce stronger posterior updates, and the section derives stability results for EI under truncation.These results assume a fixed dataset and fixed variance hyperparameters.Source: §E.1 · p. 26

E.2 COMPRESSED OBSERVATION STATE; E.3 RECOMPRESSION AND EVIDENCE CORRECTION

  • Orthogonal observation-space compression preserves affine likelihood statistics, while recompression retains leading eigendirections and adds a log-determinant correction to reduce evidence bias.The state is reduced to pseudo-observations, and the correction addresses determinant contributions omitted from that state.Source: §E.2 · p. 27, §E.3 · p. 27

E.4 STABILITY UNDER BOUNDED-COST REFITTING

The section analyzes how streaming recompression affects the affine posterior and expected improvement (EI), then describes how the algorithm keeps exploratory directions fresh. Its stability results distinguish fixed hyperparameter weights from the additional approximation introduced when those weights are refit.

E.4 STABILITY UNDER BOUNDED-COST REFITTING

  • Recompression preserves the centered slope–response statistic exactly, leaving the centered design Gram matrix as the approximate sufficient statistic for the slope posterior.This characterization is conditional on the variance hyperparameters.Source: §E.4 · pp. 27–28
  • For fixed variance hyperparameters, the section gives bounds on predictive and EI differences under streaming compression.The marginal-EI result also applies when exact and compressed calculations share the same hyperparameter distribution.Source: §E.4 · pp. 29–30
  • These results do not bound changes in hyperparameter weights caused by refitting from the compressed marginal likelihood.A log-determinant correction is used to reduce this separate approximation source.Source: §E.4 · p. 30

E.5 PRESERVING FRESH OFF-SPAN DIRECTIONS

  • A fixed orthonormal frame supplies unused directions, so compression cannot cause exploration to revisit directions omitted from the fitting state.The frame requires O(D) additional state.Source: §E.5 · p. 30

F ORTHOGONAL FRAMES FOR OFF-SPAN DIRECTIONS

  • For bounded-state runs, a seeded randomized DCT-II frame replaces the Haar frame and uses O(D) stored state rather than the O(ND) explored-span representation.Its columns are exactly orthonormal but are not Haar-distributed.Source: F ORTHOGONAL FRAMES FOR OFF-SPAN… · p. 31

G.1 ZONALFUNCTIONBENCH: CONTROLLED OBJECTIVES ON THE SPHERE

ZonalFunctionBench provides rotationally invariant objectives on the sphere for controlling how much structure lies outside the affine surrogate class. Its settings vary the balance between affine and higher-order components.

G.1 ZONALFUNCTIONBENCH: CONTROLLED OBJECTIVES ON THE SPHERE

  • ZonalFunctionBench uses random Gegenbauer features to construct rotationally invariant sphere objectives with controlled non-affine structure.Uniformly sampled feature directions avoid a preferred coordinate system or axis-aligned structure.Source: §G.1 · p. 31
  • Degrees 0 and 1 lie within the affine surrogate class, whereas degrees 2 and above add non-affine structure.The weights σ²_q control the contribution of each spherical-harmonic degree.Source: §G.1 · p. 31
  • The benchmark includes an affine control and three settings that progressively shift spectral power from degree 1 toward higher-order structure.In the severe setting, only 1/7 of degree-1–3 spectral power is assigned to the linear component; methods use the same objective draws seed-for-seed.Source: §G.1 · pp. 31–32

G.2 GENERATIVE-MODEL OPTIMISATION PROBLEMS

The experiments pair pretrained scientific generative models with black-box objectives and optimise their latent representations without retraining the models. Four problems span turbulence, antimicrobial peptides, and calorimeter showers, including a matched calorimeter pair with different discretisations.

Shared protocol

  • Generative models are used without retraining or fine-tuning, and fixed deterministic reverse sampling makes each latent objective deterministic.Diffusion-model latents serve as terminal noise; sampling uses DDIM with η = 0 or DPM-Solver++.Source: §G.2 · p. 32

Problem scope

  • The four problems span three scientific domains, with latent dimensions from 6,000 to 40,500; the calorimeter pair compares matched objectives across a 6.25-fold increase in latent voxel coordinates.Datasets 2 and 3 use the same model family, physical target, and objective, but different detector discretisations.Source: §G.2 · p. 32, Table 2 · p. 33

Lagrangian turbulence

  • The turbulence task scores agreement with direct-numerical-simulation velocity-increment statistics across five lags, using an objective that is non-differentiable.The reference distributions use 256 ground-truth trajectories and 101 fixed histogram bins.Source: §G.2 · p. 33

Antimicrobial peptides

  • The peptide task decodes 42 latent token positions into amino-acid sequences and maximises the Eisenberg mean hydrophobic moment, computed directly without an auxiliary predictor.Discrete decoding makes the latent objective piecewise constant and non-differentiable; its prior median is 0.34.Source: §G.2 · p. 33

Calorimeter showers

  • The two calorimeter tasks generate 100 GeV electromagnetic showers and maximise cosine similarity between the normalised energy-deposit distribution and a fixed target pattern.Dataset 2 has 6,480 latent coordinates; dataset 3 has 6.25 times as many voxel coordinates.Source: §G.2 · pp. 33–34

H EXPERIMENTAL AND IMPLEMENTATION DETAILS

The implementation combines precomputed exploration schedules, bounded-state recompression, and per-round fitting and acquisition procedures. These choices set numerical approximations and runtime state while retaining the full domain of admissible queries.

H EXPERIMENTAL AND IMPLEMENTATION DETAILS

  • The appendix specifies numerical implementation choices that do not alter the affine working model or acquisition definitions.It covers hyperparameter discretisation, schedule settings, bounded-state approximations, and per-round algorithms.Source: H EXPERIMENTAL AND IMPLEMENTATION… · p. 34

H.1 NUMERICAL SETTINGS

  • The finite-horizon schedule is precomputed from the evaluation horizon and reference-resolution parameter, with λref = 105 for smooth objectives and λref = 10 for quantised peptides.The schedule is independent of fitted variance scales.Source: §H.1 · p. 35

H.2 BOUNDED-STATE IMPLEMENTATION

  • Recompression retains leading Gram eigendirections and augments them with spectral-bin information, returning the live representation to O(M) rows.A trigger margin avoids immediately retriggering recompression, and discarded determinant contributions are accumulated for later hyperposterior evidence evaluations.Source: §H.2 · p. 35

H.3 COMPLETE PER-ROUND ALGORITHMS

  • Each round fits the current observation representation, selects an EI candidate at prescribed off-span norm tN, and reconstructs the query using a fresh orthogonal direction.The algorithms use the true observation count N for finite-horizon schedules; pseudo-row counts enter only bounded-state linear algebra.Source: §H.2 · p. 36, §H.3 · p. 36

I ADDITIONAL EXPERIMENTAL RESULTS

  • The additional-results appendix covers evaluation protocols, schedule-by-treatment diagnostics, long-horizon endpoints, compression experiments, and baseline configurations.Objective definitions and SABLE numerical settings are provided in Appendices G.1–G.2 and H.Source: I ADDITIONAL EXPERIMENTAL RESULTS · p. 36

I.1 EVALUATION PROTOCOL AND DIAGNOSTICS

The evaluation protocol uses paired seeds and reports objective performance alongside diagnostics of recovery, exploration, and fitted uncertainty. The paper also compares maximum-likelihood, MAP, and hyperposterior-marginalised variance-scale treatments.

I.1 EVALUATION PROTOCOL AND DIAGNOSTICS

  • Ten independent seeds pair methods on the same objective instance, and curves show means with 5th–95th percentile bands.For generative-model problems the objectives are deterministic; noisy observations use seed-determined noise, and scores are evaluated on the noiseless objective.Source: §I.1 · pp. 36–37
  • The primary metric is the best objective value observed through evaluation N, while controlled-objective tables also report attained value using the better of the best query and posterior-mean recommendation.That recommendation receives one further evaluation, which model-free baselines cannot use.Source: §I.1 · p. 37
  • Degree-one recovery, off-span norm, and fitted amplitude and discrepancy scales diagnose model recovery and exploration rather than serving as optimisation metrics.Degree-one recovery is the cosine similarity between the posterior-mean recommendation and the true coefficient direction.Source: §I.1 · p. 37

I.2 VARIANCE-HYPERPARAMETER TREATMENTS

  • The three variance-scale treatments are MLE, MAP, and an eight-atom discretised hyperposterior whose atoms are averaged in the acquisition.All treatments use the same acquisition, candidate directions, off-span complement, and recommendation readout, isolating differences to hyperparameter fitting.Source: §I.2 · p. 37

I.3 FITTED UNCERTAINTY AND OFF-SPAN EXPLORATION

The exploration ablations show that unconstrained EI can allocate too little or too much norm off-span, depending on the fitted uncertainty. A fixed reference schedule improves recovery and attained value under MAP or marginalisation on non-affine objectives, while noise changes which schedule rungs work best.

I.3 FITTED UNCERTAINTY AND OFF-SPAN EXPLORATION

  • Without a schedule, MLE drives discrepancy to its numerical floor and EI allocates mean off-span norm 0.66 on the affine control and 0.74–0.83 on the non-affine ladder, while higher-order structure can instead shrink it to 0.015–0.02.The former leaves little norm for in-span exploitation; the latter confines search to the early design span.Source: §I.3 · p. 38
  • With MAP or marginalisation, increasing the fixed reference rung improves degree-one recovery over no schedule on non-affine objectives, and attained value rises through λref = 103, with 103 and 105 statistically indistinguishable.The smallest rungs keep off-span norm high and sacrifice exploitation; the schedule cannot repair MLE's poor fit on non-affine objectives.Source: §I.3 · p. 38
  • On non-affine objectives, the model-derived reference trails λref = 103 by 0.10–0.20 in attained value because misspecification inflates discrepancy and yields an unreliable exploration ratio.On the affine control, it performs as well as the best fixed rung.Source: §I.3 · p. 38
  • With Gaussian observation noise of standard deviation σε = 0.05 at D = 1000, all treatments degrade strongly and the best fixed rungs move down to λref ∈ {100, 101}.Fine directional probes are obscured by noise; the model-derived reference becomes competitive in this noisy setting but not in the noise-free case.Source: §I.3 · p. 38

I.4 ENDPOINT VALUES AT D = N = 10,000

  • The long-horizon endpoint comparison extends the reference ladder to λref ∈ {101, 103, 105} at M = 300 with marginalised variance scales.The passage identifies the endpoint table and comparison conditions but provides no endpoint values.Source: §I.4 · pp. 38–39

I.5 LOW-RANK POSTERIOR EXPERIMENTS

The experiments test bounded-state posterior compression, compare optimisation against evaluation count, and document the baseline configurations. At D = N = 10,000 with M = 300, prescribing the off-span schedule beats every comparison on every non-affine objective.

I.5 LOW-RANK POSTERIOR EXPERIMENTS

  • Compression budgets M ∈ {10, 100, 300, 500} vary retained posterior information without restricting candidate queries to an M-dimensional subspace.The M = 10 setting is an aggressive-compression ablation; larger settings test whether acquisition-relevant information saturates below the accumulated evaluation count.Source: §I.5 · p. 39
  • Uncapped-posterior runs are compared at N = 5,000, the largest budget reached by all ten seeds, while compressed runs continue to N = D = 10,000.The uncapped posterior's per-decision cost at N = D rules out running it to the full budget in practice.Source: §I.5 · p. 39
  • At D = N = 10,000 with M = 300, prescribing the off-span schedule beats every comparison on every non-affine objective.Table 5 reports the better of the best observed point and posterior-mean recommendation, averaged over ten paired seeds.Source: Table 5 · p. 40

I.7 BASELINE CONFIGURATIONS

  • Vanilla BO, Linear BO, and TuRBO use reference BoTorch implementations and sphere decoding, but their runs are curtailed by decision cost or time limits.Vanilla BO and Linear BO cost minutes per decision by N ≈ 1,000; runs are read at 48 hours, and all three baselines are omitted on Calorimeter DS3.Source: §I.7 · pp. 44–45
  • MSR and BAxUS retain their reference initial designs and per-step refitting, while O3 searches a chart over random generative-prior latents using CMA-ES.MSR runs stop at 48 hours between N = 300 and 800 and is omitted on Calorimeter DS3; BAxUS stops between N = 700 and 1,300.Source: §I.7 · pp. 45–47

I.6 SCIENTIFIC LATENT-SPACE OPTIMISATION AGAINST EVALUATION COUNT

  • Figure 12 plots best observed value against evaluation count for Figure 4 runs, with means and 5th–95th percentile bands over ten seeds.Standard BO and Linear BO stop at the last evaluation each seed completed within 48 hours.Source: §I.6 · p. 40, Figure 12 · p. 47

Related papers

Loading 2610.31495v1…