PaperlayerView exact revision on arXiv

Source-linked AI summary

Multivariate conformal uncertainty propagation in multitask atomistic simulation: Successes and pitfalls

Katharine Fisher, Michael Herbst, James Kermode, Youssef Marzouk

arXiv:2609.31384v1physics.chem-phstat.ML

TL;DR

Reliable uncertainty quantification for multistage atomistic workflows remains challenging, particularly when multiple properties and configurations feed downstream quantities. This paper applies multivariate conformal methods to chemical properties and propagates their sets. Its results highlight how modeling correlations can capture near symmetries and error cancellation, while approximate protocols complicate guarantees.

  • Problem

    Multistage atomistic workflows need reliable uncertainty estimates for multivariate properties and downstream quantities, while realistic surrogate data may violate probabilistic assumptions.

    Source: §1 · pp. 2–3

  • Method

    The paper calibrates multivariate conformal sets using predicted covariance and evaluates their propagation to downstream properties in atomistic workflows.

    Source: §1 · p. 3, §1.1 · p. 3, §3.1 · p. 5

  • Results

    Correlation-aware multivariate sets capture near symmetries and error cancellations, and often have smaller volume than competing methods at a given coverage level.

    Source: §1 · p. 3, §6 · p. 17

  • Takeaways & Limitations

    The practical assessment of propagated conformal sets must account for the approximate transformations used in the workflow.

    Source: §5.3 · p. 16, §6 · p. 17

    The reference and predicted elastic constants use different operators, and the energy-based prediction adds an approximation stage that can lower coverage.

    Source: §5.3 · p. 16

Abstract

from arXiv · show

Machine learning has become the standard tool for the design of interatomic potentials which balance efficiency and accuracy, but uncertainty quantification remains an open problem. Multiscale simulations introduce an additional challenge: robust uncertainty quantification across scales. Even within one scale, computations are often multistage, producing a sequence of target quantities, each dependent on the previous, and each with some uncertainty. Conformal methods have emerged as a model agnostic framework for recalibrating surrogate predictions to produce sets which contain the truth at a user-specified rate. For multistage workflows, we require uncertainty calibration for multiple chemical properties and atomistic configurations, and we want to propagate uncertainty sets to downstream quantities of interest. Such propagation should capture the error cancellations which occur in many downstream targets in materials science; for instance, an approximate energy difference is often more accurate than individual energy predictions. We present the first exploration of multivariate conformal methods for chemical properties, including Bonferroni-corrected hyperrectangles, hyperellipsoidal sets based on the Mahalanobis distance, and custom loss functions within conformal risk control. Calibration is applied directly to predicted energies, atomic forces, and virial stresses, then propagated to elastic constants and vacancy formation energies employing a variety of commonly considered approximate protocols in materials modeling. We highlight the benefits of building correlation predictions into the conformal procedure, making it possible to build sets which capture near symmetries and error cancellation. We conclude with a discussion of the interplay of the employed approximate computational protocol and conformal guarantees.

1 Introduction

Reliable uncertainty quantification is needed for multistage atomistic workflows. The paper introduces correlation-aware conformal sets for multivariate chemical properties and their propagation to downstream quantities.

1 Introduction

  • Reliable uncertainty quantification is needed because realistic surrogate data can violate probabilistic assumptions, while atomistic workflows require multivariate and propagated uncertainty estimates.Prior conformal work focused on scalar quantities, whereas multistage materials workflows may propagate uncertainty from multiple systems to derived properties such as elastic constants.Source: §1 · pp. 2–3
  • Multivariate sets capture near symmetries in related properties and often achieve smaller volume than competing methods at a given coverage level.For two perturbed silicon configurations, the multivariate approach captures near symmetry in C12 elastic-constant components; univariate sets either undercover or include excessive volume.Source: Figure 2 · p. 2, §1 · p. 3

1.1 Contributions •

  • The workflow incorporates Gaussian-process covariance into conformal scores to capture relationships between quantities while maintaining frequentist validity without overly conservative sets.The paper combines Bayesian covariance predictions with frequentist conformal procedures.Source: §1.1 · p. 3
  • A multitask Gaussian process interatomic potential relates energies, forces, and stresses across bulk and vacancy atomistic systems.Source: §1.1 · p. 3
  • The test cases include conformal sets with O(100) dimensions, one to two orders of magnitude larger than typically considered in conformal literature, with few exceptions [22].Source: §1.1 · p. 3
  • Two conformal risk-control loss functions promote high component-wise coverage using covariance predictions and can avoid overly conservative sets for sufficiently high-dimensional targets.Source: §1.1 · p. 3
  • The propagation examples predict elastic constants and vacancy formation energies through workflows involving multiple energy and force evaluations, structural perturbations, and relaxations.Source: §1.1 · p. 3
  • The paper examines pitfalls that can arise when conformal sets are propagated through approximate transformations.Source: §1.1 · p. 3

2 Related work

This section reviews atomistic surrogate models and uncertainty quantification, then situates multivariate conformal sets and their propagation among existing approaches.

2.1 Machine learning interatomic potentials

  • Deep-learning interatomic potentials offer strong performance, while Gaussian approximation potentials provide probabilistic uncertainty estimates and are the focus here for implementation and interpretability.The authors note that their experiments are straightforward to port to neural-network settings.Source: §2.1 · p. 4

2.2 UQ in atomistic simulation

  • Atomistic uncertainty methods include calibrated conformal procedures and model-based estimates, but neural-network approaches lack guaranteed calibration and can be vulnerable to overconfidence.Source: §2.2 · p. 4

2.3 Multivariate conformal methods

  • Multivariate conformal methods include component-wise hyperrectangles and scores such as the Mahalanobis distance; Bayesian-predicted covariance yields hyperelliptical sets that capture correlation and support propagation.The cited Mahalanobis-score work includes [20] [21] [22] [75].Source: §2.3 · pp. 4–5

2.4 Conformal uncertainty propagation

  • Prior propagation of ellipsoidal conformal sets used toy examples or randomly sampled operators for targets of at most 16 dimensions, while this work considers atomistic workflows and higher-dimensional calibration.The closest prior work is identified as [21].Source: §2.4 · p. 5

3 Methods

The methods combine Gaussian-process surrogate predictions with conformal calibration to construct uncertainty sets for scalar and multivariate chemical properties. They also propagate sets to downstream quantities, with guarantees and sharpness shaped by the transformation used.

3.1 Gaussian approximation potential (GAP)

  • GAP maps atomistic geometries to energies, forces, or stresses, using SOAP features and a polynomial kernel whose covariances extend to forces and stresses through differentiation.The model assumes energy is additively decomposed into local contributions.Source: §3.1 · pp. 5–6

3.2 Multitask Gaussian process models

  • The multitask Gaussian process models correlations across heterogeneous training data and test predictions; secondary tasks combine a linear relation to a primary task with a residual for uncorrelated behavior.Incomplete observations across tasks or properties can still be incorporated.Source: §3.2 · pp. 6–7

3.3 Conformal prediction

  • Split conformal calibration scores held-out examples and uses their score quantile to form prediction intervals, with finite-sample coverage guarantees when calibration and test data are exchangeable.The guarantee is marginal rather than conditional on the input; the upper bound also limits excessive conservatism.Source: §3.3 · p. 7

3.4 Multivariate conformal prediction

  • Bonferroni correction adjusts coordinate-wise conformal calibration for vector coverage, while Mahalanobis scoring uses predictive covariance to form ellipsoids that can exploit correlations and accommodate varying vector sizes.Ellipsoidal sets also support sample-free propagation through linear transformations.Source: §3.4 · pp. 7–8

3.5 Conformal risk control

  • Conformal risk control calibrates sets against a chosen bounded, non-increasing loss, enabling bespoke guarantees; the introduced losses promote sparse deviations and target high but incomplete component-wise coverage.The method focuses on ellipsoidal sets for implementation and propagation.Source: §3.5 · p. 8

3.6 Conformal uncertainty propagation

  • Rectangular and ellipsoidal sets retain their shape under linear transformations, and propagated sets can be computed in closed form without sampling; sharpness translates only for invertible transformations.The conservatism of propagated sets depends on the transformation.Source: §3.6 · p. 9
  • A proposed score can pre-correct for propagation to obtain validity and sharpness guarantees, but its dependence on each downstream operator requires separate, potentially costly conformalization.The score is specific to the downstream quantity considered.Source: §3.6 · p. 9

4 Experiments

Experiments test conformal calibration for chemical-property vectors and its propagation to elastic constants and vacancy formation energies. Correlation-aware sets can be smaller, but propagation and evaluation protocols affect coverage guarantees.

4.1 Synthetic experiments

  • In synthetic propagation, higher feature correlation yields smaller multivariate sets, and reducing propagation dimension further shrinks them but tends to cause overcoverage.Correcting for propagation with score (23) prevents overcoverage; univariate set volume changes little with correlation or propagation dimension.Source: Figure 3 · p. 10, §4.1 · p. 10

4.2 Dataset and representation for chemical experiments

  • The silicon experiments use small, varied configuration sets and SOAP features, with GP training targets selected case by case from energy, force, and stress data.The dataset includes bulk configurations with 2, 16, 54, or 128 atoms and vacancy configurations with 63 or 215 atoms.Source: §4.2 · pp. 10–11

4.3 Calibration of energy and atomic forces

  • Conformal risk control produces sets consistently smaller than univariate hyperrectangles and generally smaller than multivariate hyperellipsoids, while allowing high component-wise but low full-vector coverage.The two tested losses are the 1-norm and 0.9-quantile losses.Source: Figure 5 · p. 12, §4.3 · p. 12

4.4 Uncertainty propagation to elastic constant prediction

  • For elastic constants, multivariate sets meet expected coverage after energy or stress calibration, and are consistently smaller than Bonferroni sets at similar coverage.Energy-calibrated Bonferroni coverage is only above the y = x line at lower 1 − α and plateaus when α ≥ 0.55.Source: Figure 6 · p. 13, §4.4 · pp. 13–14

4.5 Uncertainty propagation to vacancy formation energy prediction

  • At 1 − α = 0.95, all vacancy-energy methods reach complete coverage, while propagation-corrected multivariate sets remain sharpest; Bonferroni intervals are up to 2.3 times as large.Across tested tolerances, propagation-corrected intervals are consistently smaller than univariate counterparts, though only by a small factor.Source: Figure 7 · p. 15, §4.5 · p. 15

5 Discussion

The discussion weighs data and exchangeability constraints against approximate propagation, then compares conformal strategies for multistage materials workflows. Its recommendations depend on coverage goals and whether informative covariance estimates are available.

5 Discussion

  • The discussion frames its analysis around exposing workflow approximations, assumptions, and error sources to inform conformal-method choices for practical multistage problems.Source: §5 · p. 15

5.1 Limited Data

  • Fewer than one thousand configurations must be divided among hyperparameter optimization, training, calibration, and evaluation, and set sharpness depends on calibration-set size.Not all configurations are relevant to every problem.Source: §5.1 · p. 15

5.2 Exchangeability

  • The calibration and test cases differ in exchangeability: randomly selected calibration configurations do not match the correlated perturbations used downstream, and synthetic tests show multivariate overcoverage.Successful energy and force calibration is empirical evidence for exchangeability of the original diamond configurations, but not sufficient for propagation tests.Source: §5.2 · p. 16

5.3 Approximate workflow

  • Propagation can yield overly conservative sets, increasingly so as dY/dQ grows, while approximate operators can produce coverage below the expected guarantee.The energy-based elastic-constant route has lower coverage than the stress-based route; covariance-based multivariate sets differ from Bonferroni sets, showing covariance information is meaningful.Source: §5.3 · p. 16
  • Approximate workflow errors are not fully captured by calibration on GAP prediction errors, yet reasonable settings achieve expected coverage and reveal relative behavior among conformal strategies.The discussion also notes that propagated multivariate sets can be conservative, while mismatched operators may reduce coverage; observed coverage varies across settings.Source: §5.3 · p. 16

5.4 Choosing a conformal method

  • Choose component-wise univariate methods for component coverage, Bonferroni correction for full coverage without useful covariance, or Mahalanobis sets when covariance has signal.Mahalanobis sets can offer similar coverage with smaller volume and capture error cancellation; propagation may still be conservative or affected by operator approximations.Source: §5.4 · pp. 16–17
  • Conformal risk control can tailor loss functions to produce guaranteed sets with high univariate coverage and comparatively low volume, but risks overfitting a problem or dataset.The authors identify its role in materials workflows as a direction for further research.Source: §5.4 · p. 17

6 Conclusion

The paper evaluates multivariate conformal methods in multistage materials workflows and identifies both their utility and pitfalls. Correlation-aware ellipsoidal sets can reduce volume and capture error cancellation, while propagated coverage depends on workflow approximations.

6 Conclusion

  • Mahalanobis-based multivariate sets leverage correlations across configurations and properties, sometimes capturing near symmetries and reducing set volume while representing error cancellations in relative properties.The paper tests propagation to elastic constants and vacancy formation energies and discusses how computational approximations affect validity.Source: §6 · p. 17
  • The paper identifies distribution-shift-robust conformal methods, class-conditional or relative uncertainty representations, and new risk-control guarantees as directions for future work.The experiments calibrate Gaussian process surrogates, while conformal prediction is model agnostic.Source: §6 · p. 17

Funding

The authors acknowledge support from Swiss and U.S. research programs and fellowships.

Funding

  • MFH acknowledges SNSF and NCCR MARVEL support; KEF acknowledges the Don Loomis Galusha Fellowship and an NSF Graduate Research Fellowship; KEF and YMM acknowledge DOE NNSA PSAAP-III and PSAAP-IV support.Source: Funding · p. 17

A Hyperparameter settings

The workflow uses literature- and experience-based SOAP settings, small optimization sets, and line searches to select surrogate, multitask, finite-difference, and conformal parameters while aiming to avoid overfitting.

A Hyperparameter settings

  • SOAP uses a 5-angstrom cutoff, Gaussian variance 0.25, nmax = 8, and ℓmax = 6, selected using literature best practices and prior experience.Source: A Hyperparameter settings · p. 25
  • For the polynomial-kernel Gaussian process, a 20-configuration leave-one-out search over γ²/(cv) selects a = −8, with performance evaluated on energy prediction.The kernel exponent is 4; conformal rescaling makes predictions insensitive to the absolute scale of Σ.Source: A Hyperparameter settings · p. 25
  • The multitask search uses 20 bulk and 10 vacancy configurations, selecting vp/vs = 1 and ϱs = 0.55 to avoid overfitting.The search finds a narrow low-error basin at ϱs ≥ 0.95 with large vp/vs, and a wider one near ϱs = 0.5 and vp/vs ≈ 1.Source: A Hyperparameter settings · p. 25
  • Finite-difference tests use a central-difference step size of 0.05, selected by leave-one-out stress-prediction error; Appendix D also reports 0.01.The step-size search spans 10⁻⁴ to 0.5, and the authors note that the elastic constant itself cannot be used for optimization because it is the target.Source: A Hyperparameter settings · p. 25
  • Conformal set character is controlled by tolerance α, and conformal risk control may add loss-function parameters such as upper bound B, often chosen as a calibration-loss quantile.The authors test a range of conformal parameters and present representative results.Source: A Hyperparameter settings · p. 25
  • Across hyperparameter categories, the stated aim is to use minimal data and avoid overfitting.Source: A Hyperparameter settings · p. 25

B Additional results of synthetic experiment

Synthetic propagation tests show multivariate conformal sets maintain target coverage, although basic propagated sets become more conservative as output dimension decreases. Propagation correction restores both coverage and sharpness guarantees when its operator is known during calibration.

B Additional results of synthetic experiment

  • As propagated dimension d_Q decreases, multivariate sets retain target coverage but become increasingly conservative.Component-wise sets often miss the desired 1 − α coverage, particularly with lower covariate correlation or larger α.Source: B Additional results of synthetic… · p. 26, Figure 8 · p. 26
  • Propagation-corrected multivariate sets satisfy both coverage and sharpness guarantees across the tested output dimensions.The correction requires knowing the propagation operator at calibration time.Source: B Additional results of synthetic… · p. 26, Figure 9 · p. 27
  • With uncorrelated calibration covariates and test covariates correlated at ρ = 0.99, both multivariate approaches remain over-conservative.The covariance mismatch reduces the performance difference between basic and propagation-corrected methods.Source: B Additional results of synthetic… · p. 26

C Additional results of energy and force calibration

The energy-and-force experiments verify conformal risk control’s loss bounds for two loss choices. Results are also reproduced using a lower calibration-based upper bound on the loss function.

C Additional results of energy and force calibration

D Additional results of elastic constant prediction

The elastic-constant experiments compare energy- and stress-based finite-difference workflows and show that coverage depends on target dimension and approximation settings. Multivariate sets can be smaller while covering more than univariate sets, but six-dimensional guarantees are not consistently attained.

D Additional results of elastic constant prediction

  • The Gaussian-process elastic-constant predictions are evaluated for workflow adequacy, not as a state-of-the-art prediction method.The comparison uses energy-based second derivatives and stress-based first derivatives, with errors reported against DFT references.Source: D Additional results of elastic… · p. 27, Table 1 · p. 29
  • In the energy-based six-dimensional experiment, multivariate sets have greater coverage than univariate sets while also having smaller volume, but multivariate and Bonferroni methods miss guaranteed coverage for some α.The multivariate method uses covariance information to identify symmetries such as C12 = C21; the reported size ratio is dimension-standardized, with true volume ratios reaching O(1000).Source: D Additional results of elastic… · pp. 27–28, Figure 13 · p. 30
  • Projected univariate coverage is close to 1 − α for the stress-based workflow, but falls short for the energy-based workflow; multivariate coverage is consistently high.The projection evaluates only [C11, C12, C44], excluding the zero components C45 and C54.Source: D Additional results of elastic… · p. 28, Figure 14 · p. 30
  • With a 0.01-angstrom energy-based finite-difference step, multivariate-set coverage is reduced relative to the other experiments and the expected guarantee.Most results use a 0.05-angstrom step; the associated coverage and volume comparisons are shown for three- and six-component predictions.Source: D Additional results of elastic… · p. 28, Figure 15 · p. 31, Table 2 · p. 31
  • For Si54, six-dimensional multivariate sets remain below guaranteed coverage but are considerably smaller than univariate sets; three-component sets meet the guarantee for almost all α.The authors attribute the dimension-dependent difference to difficulty covering zero components C45 and C54.Source: D Additional results of elastic… · p. 29, Table 3 · p. 31, Figure 16 · p. 32

E Additional results of vacancy formation energy prediction

The vacancy-formation-energy appendix repeats the conformal comparisons with fewer training configurations and reports similar qualitative behavior. The workflow calibrates on bulk and vacancy energies before propagating to their scaled difference.

E Additional results of vacancy formation energy prediction

  • With 150 training configurations instead of 350, conformal methods show similar qualitative behavior for vacancy-formation-energy prediction.The comparison uses 25 random training and calibration draws for each of 55 silicon configurations.Source: E Additional results of vacancy… · p. 33, Figure 17 · p. 33
  • Calibration is performed on bulk-configuration and vacancy energies, then propagated to their scaled difference.Table 4 reports relative prediction-error distributions across silicon configurations with 2, 16, 54, and 128 atoms.Source: Table 4 · p. 33
  • The replicated comparison plots coverage and interval- or set-volume ratios for univariate, Bonferroni-corrected, multivariate, and propagation-corrected methods.The figure also reports the interquartile ribbons across the random draws.Source: Figure 17 · p. 33
Loading 2609.31384v1…