PaperlayerView exact revision on arXiv

Source-linked AI summary

Conformal Uncertainty Quantification Guarantees for Neural Operators

Tom Stent, Nicolas Boullé

arXiv:2608.28515v1math.NAcs.AImath.PR

TL;DR

Neural operators lack reliable uncertainty quantification with coverage guarantees. The paper develops split conformal calibration that scales an estimated error using spatial residual quantiles, yielding finite-sample marginal guarantees for continuum and discrete domains; conditional coverage follows a Beta distribution under additional assumptions.

  • Problem

    Neural-operator uncertainty estimates generally depend on modelling assumptions, training objectives, or data distributions and lack coverage guarantees.

  • Method

    The method computes spatial (1−γ)-quantiles of normalized residual fields and calibrates a single scaling factor on held-out data for continuum or discrete evaluations.

  • Results

    The framework provides finite-sample marginal coverage under exchangeability, while conditional coverage follows a Beta distribution under i.i.d. atomless-score conditions.

  • Takeaways & Limitations

    The construction provides theoretically justified prediction bands with controlled spatial failure fractions for neural-operator outputs.

  • Takeaways & Limitations

    Guarantees are marginal over exchangeable calibration and test inputs, do not cover distribution shift or every realized calibration set, and discrete guarantees depend on the calibration grid.

Abstract

from arXiv · show

Neural operators provide fast surrogate models for approximating operators between function spaces, but their predictions often lack uncertainty quantification. We develop a split conformal framework to guarantee that a calibrated pointwise band around the neural operator output contains the true solution on at least a $1-γ$ fraction of the evaluation domain, with probability at least $1-α$ over test and calibration inputs, where $α,γ\in(0,1)$. Our method reduces a normalized residual field to its spatial $(1-γ)$-quantile and computes a scaling factor using a held-out calibration dataset. We prove marginal coverage guarantees for measurable residual fields defined on arbitrary probability spaces, covering both continuum domains and fixed discretizations. Under mild assumptions on the data distribution, we show that the coverage conditional on the calibration set follows a Beta distribution, which we verify with numerical experiments on Darcy flow and Navier--Stokes equations, where our calibration yields bands consistently tighter than existing corrections while retaining the target coverage.

1 Introduction

Neural operators approximate operators between function spaces, but existing uncertainty estimates generally lack coverage guarantees. This work introduces split conformal calibration for neural operators and establishes finite-sample guarantees under broad conditions.

  • Neural operators approximate nonlinear operators, including PDE solution operators, between infinite-dimensional function spaces.
  • Existing neural-operator uncertainty methods often depend on modelling assumptions, training objectives, or data distributions and generally lack coverage guarantees.
  • Cellwise conformal methods control the average fraction of covered solution-location pairs rather than domain coverage for each individual solution.
  • The proposed split-conformal framework uses measurable residual fields and applies to continuum and discretized settings under exchangeability.
  • Under i.i.d. inputs with continuous conformal-score distributions, conditional coverage follows a Beta distribution that quantifies calibration-set variability.

2 Calibration Set and Coverage Guarantees

The framework constructs a prediction band by scaling an estimated pointwise error so that the true solution is contained over at least 1−γ of the domain with probability at least 1−α. It provides marginal guarantees for both continuum and fixed-grid evaluations.

  • The prediction band centers on the neural-operator output and scales the pointwise error estimator by λ.
  • The target is containment on at least 1−γ of domain points, with probability at least 1−α over an input drawn from µ.
  • A generic measurable residual-field argument reduces spatial coverage to a scalar score defined by the smallest threshold covering at least 1−γ of the evaluation space.
  • The calibration scaling factor is the kth smallest calibration score, where k=⌈(1−α)(n+1)⌉, or ∞ when k>n.
  • Under exchangeability, continuum and discrete bands achieve marginal coverage at least k/(n+1)≥1−α.
  • For i.i.d. inputs and atomless scores, conditional coverage follows Beta(k,n+1−k), while discrete spatial scores are computed exactly by sorting grid residuals.

3 Function-Valued Split Conformal Calibration

The practical procedure separates training, residual-estimation, and calibration data, then calibrates a single scaling factor from spatial residual quantiles. It applies to continuum and discrete evaluations and characterizes conditional-coverage variability.

  • The data are split into disjoint training, residual, and calibration sets before fitting and freezing the prediction operator and error estimator.A typical split uses around 60% training, 30% residual, and 10% calibration data.
  • For each calibration input, the procedure computes a normalized residual field and its spatial (1−γ)-quantile score.
  • The scaling factor is the kth order statistic of calibration scores, with k=⌈(1−α)(n+1)⌉, and defines the final prediction band.
  • In the discrete setting, the spatial infimum becomes the qth order statistic of M residual values, with q=⌈(1−γ)M⌉.
  • Conditional coverage follows a Beta distribution, and its standard deviation is O(n−1/2), decreasing as calibration-set size increases.

4 Numerical Experiments

Experiments on Darcy flow and Navier–Stokes show that functional conformal calibration produces tight pointwise bands while retaining the target containment coverage. Compared with Ma et al. (2024), the method uses smaller scaling factors and avoids their conservative coverage behavior.

  • Experimental setup: The experiments compare functional conformal calibration with Ma et al. (2024) using identical trained operators and calibration data.Both methods produce pointwise bands of the form ˆGM(a)(xj) ± ˆλ ˆEM(a)(xj).
  • Experimental setup: For γ = α = 0.1, coverage measures whether at least 90% of grid points are contained over 90% of calibration or test draws.The study uses 3000 random disjoint calibration/test re-splits and reports empirical distributions of ˆλ and coverage.
  • Darcy flow: 2.37× at N = 12 to 1.16× at N = 84: Ma et al. (2024) produces larger Darcy-flow scaling factors than the proposed method.The proposed calibrated factors remain consistently smaller across the reported spatial resolutions.
  • Darcy flow: At N = 60, the proposed method’s empirical coverage closely matches the predicted Beta-Binomial law and targets 1 −α = 0.9.Ma et al. (2024) concentrates near 0.98, ensuring validity but being substantially more conservative than the target.
  • Resolution transfer: Under zero-shot super-resolution, coverage is zero at all calibration resolutions for both methods; spectral interpolation gives near-zero coverage for N ≤42.Coverage increases with spatial resolution under spectral interpolation, revealing a transfer limitation to fine grids.
  • Navier–Stokes: The estimated Navier–Stokes error field tracks the spatial error pattern, enabling tight pointwise bands while maintaining containment fraction 1 −γ.The visualization uses N = 64 and shows the input vorticity, solution at t = 21, FNO prediction, pointwise error, and estimated error.

5 Conclusion

The paper develops split conformal uncertainty quantification for neural operators, constructing calibrated bands with controlled spatial failure. Its guarantees are marginal and grid-dependent, with limitations under distribution shift, realized calibration sets, and continuum transfer.

  • Conclusion: The framework calibrates prediction bands from a held-out calibration set to contain the true solution over a controlled fraction of the spatial domain.It combines a spatial quantile of a measurable residual field with a single calibrated scaling factor.
  • Limitations: The guarantees are marginal over calibration and test inputs and require their exchangeability.They do not ensure coverage under distribution shift or conditional on every realized calibration set.
  • Limitations: The discrete guarantee is tied to the calibration grid and is not automatically invariant to spatial resolution.Lifting it to the continuum requires a problem-dependent regularity-based score that may produce conservative bands.

A.1 Independence of the representatives

The appendix establishes that the containment-set measure is independent of the chosen measurable representatives of the operators. This follows because representative differences occur only on a common null set.

  • Result: The containment-set measure does not depend on the representatives chosen for G†, ˆG, and ˆE.This is the stated representative-independence result for the operators and error estimator.
  • Proof: Product-measurable representatives yield measurable a-sections, so the containment sets belong to B(D) and have defined measure ν.The proof uses measurability of each factor in the defining inequality.
  • Proof: After excluding a common ν-null set, the defining inequalities coincide, making the symmetric difference of the containment sets ν-null.Therefore the two containment sets have equal ν-measure.

A.2 Conformal calibration for a generic residual field

The framework converts a measurable residual field into a spatial quantile score and calibrates its order statistic with split conformal prediction. Under exchangeability it guarantees marginal containment on at least 1−γ of the evaluation space with probability at least 1−α, while i.i.d. atomless scores yield a Beta-distributed conditional coverage.

  • Generic framework: The method applies to any nonnegative jointly measurable residual field over a probability space, including continuum and discrete evaluation settings.Its guarantees are agnostic to the model, PDE, or whether the evaluation space is the physical domain.
  • Score construction: The scalar score is the smallest threshold whose residual-field containment fraction reaches 1−γ.Measurability, monotonicity, and right-continuity establish score-containment equivalence.
  • Calibration: The calibrated scaling factor is the kth smallest calibration score, with k=⌈(n+1)(1−α)⌉ and an infinite fallback when k>n.This order-statistic construction is the split-conformal calibration step.
  • Conditional variability: With i.i.d. inputs and an atomless score distribution, the conditional coverage F(λ̂r) follows Beta(k,n+1−k).The marginal guarantee averages conditional coverage over calibration sets rather than requiring the bound for every realized set.
  • Dominating scores: Calibrating a computable score that upper-bounds the ideal score preserves coverage but can make the resulting uncertainty band conservative.Because domination is one-sided, coverage may exceed 1−α by more than 1/(n+1), and band width is not controlled by the theorem.

B.1 Implementation Details

The experiments use released Darcy-flow and Navier–Stokes datasets, fixed train/calibration/test partitions, and paired prediction and error operators. Calibration rules are compared on identical trained models and calibration sets, with repeated resampling to assess validity and efficiency.

  • Data: Darcy flow uses 421×421 steady-state data, reduced to 420 points per axis after removing the duplicated final row and column.The setup uses constant forcing f≡1 and zero Dirichlet boundary conditions.
  • Data: Navier–Stokes uses 10,000 trajectories at 64×64 plus 20 trajectories at 256×256 as a fine reference grid.The learned map advances vorticity from t=11 to t=21, with working resolutions formed by strided subsampling.
  • Data partitioning: Both datasets are partitioned once with fixed seed 42 into four disjoint subsets.Partitions are sampled uniformly over sample indices.
  • Architecture and training: The prediction and error operators share the same architecture, while the error operator adds a final Softplus activation for nonnegative outputs.The prediction operator is frozen after relative L2 training; the error operator uses pinball loss at level 1−γ=0.9.
  • Calibration and evaluation: Both calibration rules use the same trained operators and calibration set, so reported differences are attributable to the rules alone.Validity and efficiency experiments pool calibration and test data, then re-split them 3,000 times using identical permutations.
  • Compute: All training and evaluation runs on a single 2024 MacBook Pro with an Apple M4 chip and 24GB memory, without GPU or cluster resources.Training time ranges from approximately 5 minutes at the coarsest resolution to 90 minutes at the finest.

B.2 Additional Numerical Results

Additional experiments show that the method remains calibrated near the target across resolutions while producing tighter bands than Ma et al. (2024), but cross-grid transfer is not guaranteed.

  • Scope: On each fixed grid, Theorem 2.5 is the relevant validity statement for the reported resolution-specific results.The appendix uses N as the resolution per spatial direction and M = N^2 as the number of 2D grid points.
  • Darcy flow: Our bands are uniformly tighter than Ma et al. (2024), with width ratios from 2.37× at N = 12 to about 1.16–1.17× at the finest resolutions.The per-resolution width ratio is smoother than mean bandwidth because both methods use the same trained operators and residual fields.
  • Coverage distributions: Across Darcy and Navier–Stokes resolutions, our coverage distributions centre near the target, whereas Ma et al. (2024) remains consistently above target.The upward shift indicates conservative validity for the comparator.
  • Cross-grid transfer: On the fine reference grid, Ma et al. (2024) reaches the target only at N = 84 after increasing width by up to 2.4×, while zero-shot coverage is 0 at all resolutions.This experiment evaluates cross-grid transfer, outside the fixed-grid guarantee of Theorem 2.5.
  • Navier–Stokes: For Navier–Stokes, efficiency ratios are 1.78×, 1.33×, and 1.23×, while resampled coverage tracks the theorem-predicted Beta-Binomial reference.The comparator’s distribution is shifted toward higher coverage with wider bands.
Loading 2608.28515v1…