PaperlayerView exact revision on arXiv

Source-linked AI summary

Dimension Bridging for 3D RANS with Neural Network Accelerated Gaussian Functional Regression

Wesley Lao, Thomas Scott, Tan Bui-Thanh, Paul Laiu, Matt Bement

arXiv:2608.27639v1cs.CE

TL;DR

The paper addresses the limited availability of high-fidelity data for fitting response surfaces in many-query tasks. It introduces a dimension-bridging kernel using low-dimensional solutions and a neural-network surrogate, yielding improved regression and real-time prediction capability.

  • Problem

    High-fidelity models are too costly for many-query tasks, limiting the availability of high-fidelity data for directly fitting data-driven surrogates.

  • Method

    The paper introduces a warped inner-product kernel using low-dimensional primal and adjoint solutions and trains a neural network to approximate its featurized Mercer kernel.

  • Results

    The resulting kernel better captures design-space variation, while the neural-network-accelerated approach substantially speeds kernel evaluation and enables real-time predictions.

  • Takeaways & Limitations

    Dimension-bridging Gaussian Functional Regression can improve response-surface regression and make computational predictions feasible within a real-time budget.

Abstract

from arXiv · show

In many computational science and engineering problems, repeatedly solving fully resolved physics-based models to design for a quantity of interest (QoI) can quickly become intractable, requiring the use of low-fidelity models to predict the same QoI but introduce errors where some features are neglected or are otherwise inaccurately resolved. We use Gaussian Functional Regression (GFR) to learn a correction to a 2D Reynolds-Averaged Navier-Stokes (RANS) model to predict the aerodynamic coefficients from a 3D RANS model. This model pair has a disparity in the governing physics from the reduced dimensionality, a previously unexplored application for GFR. Empirically, our results show that with a proper choice of low-dimensional (LD) model, the proposed kernel allows for the use of fewer high-dimensional (HD) evaluations to regress a response surface to the same level of accuracy as standard stationary kernels. Moreover, the new kernel provides more informative uncertainty quantification, which we show is advantageous when used to drive an adaptive sampling algorithm. Finally, we propose a novel neural network accelerated kernel, which we show offers predictions in good agreement while speeding up evaluations by millions of times in wall clock measurements, bringing the computational budget within the real-time regime.

I. Introduction

The paper addresses costly high-fidelity design queries by using low-dimensional physics to inform Gaussian Functional Regression corrections for high-dimensional response surfaces. It develops dimension bridging and neural-network acceleration to improve sampling efficiency, uncertainty quantification, and inference speed.

  • Motivation: High-fidelity models are too costly for many-query design tasks, limiting the data available for regressing quantities of interest.The paper therefore combines lower-cost model predictions with high-fidelity corrections.
  • Gaussian Functional Regression: The method uses a low-fidelity response surface as a Gaussian-process prior mean and learns a correction from high-fidelity data.This correction is introduced through a Gaussian process or forcing term so the posterior approximates the high-fidelity response.
  • Featurized kernel: The low-fidelity model acts as a feature map, allowing the kernel to identify regions of higher sensitivity and provide a basis for regressing the response surface.When the LD geometry cannot capture some design-parameter changes, an auxiliary LD model produces a separable featurized kernel.
  • Dimension bridging: GFR is extended to nonlinear low-fidelity models and nonlinear observation operators in lower-dimensional physical spaces, creating a dimension-bridging model.The approach distinguishes models by physical dimensionality—low-dimensional (LD) and high-dimensional (HD)—rather than relative fidelity.
  • Adaptive sampling: Adaptive sampling driven by the featurized kernel’s uncertainty quantification converges faster than sampling driven by standard stationary kernels.The comparison concerns convergence of the regressed surface.
  • Neural-network acceleration: A neural network learns an arbitrary Mercer kernel and removes additional RANS solves during online inference, yielding wall-clock speedups on the order of millions.The neural-network approach is intended to bring kernel evaluation and prediction into the real-time regime while maintaining good accuracy.

II. Background and Relevant Work

The background introduces Gaussian Process Regression as a probabilistic response-surface model and Gaussian Functionals as its extension to functionals. In this framework, adjoint solutions provide feature maps that turn functional observations into a parameter-space Gaussian process.

  • Gaussian Process Regression: Gaussian Process Regression models a function as a realization of a Gaussian process characterized by a mean function and a symmetric positive-definite covariance kernel.It produces predictions and uncertainty through the joint distribution of training and test evaluations.
  • Gaussian Process Regression: Observed noisy evaluations are conditioned to predict the function at test points, with the posterior computed through a Schur complement.The training covariance matrix needs to be inverted only once before predicting at multiple test points.
  • Gaussian Functionals: Gaussian Functionals extend Gaussian Processes from functions evaluated at points to functionals that map functions to scalars.The paper uses this framework to represent forcing corrections in a low-fidelity model.
  • QoI construction: For a given parameter configuration, the quantity of interest is obtained by evaluating the forcing functional on the corresponding adjoint solution.This allows the QoI response to be represented as a Gaussian process over parameter space.
  • QoI construction: The low-dimensional adjoint solution serves as a feature map, and composing the kernel with it defines the covariance of the parameter-space Gaussian process.The resulting process is over parameter configurations rather than directly over functions.

C. Sum and Product Kernels

The paper constructs sum and product kernels to model correlations across parameter and function spaces, including physics-informed nonstationarity and scalable kernel approximations.

  • Sum kernels: Sum kernels combine individually positive-definite kernels with nonnegative weights to infer structure such as regularity from data.The mixture hyperparameters include each component's hyperparameters and associated weight.
  • Product kernels: For Cartesian-product design spaces, factorized kernels can support efficient inversion and matrix-vector products through Kronecker products on regular grids.The construction applies when the input set is a regular Cartesian grid in R^P.
  • Product kernels: Product kernels correlate evaluations only when all factor kernels indicate similarity, enabling separate treatment of function-space and parameter-space factors.The paper uses products to require similar parameters and similar sensitivity under the LD model.
  • Nonstationarity through warping: The proposed kernel combines parameter-to-adjoint input warping with PDE-informed output warping and uses a function kernel derived from the GF correction's dual space.This construction is presented as distinct from related warped-kernel forms.
  • Nonstationarity through warping: Nonstationary kernels represent spatially varying behavior but are harder to parameterize and optimize than stationary kernels.The paper associates larger shape parameters with quickly varying regions and smaller ones with slowly varying regions.

III. Problem Statement

The paper seeks to predict high-dimensional QoIs from limited high-dimensional data by correcting a cheaper low-dimensional model with a physics-aware Gaussian Functional Regression surrogate.

  • Problem statement: Fully resolved high-dimensional solves are infeasible under limited computational budgets, motivating prediction of new QoIs with cheaper but inaccurate low-dimensional models.The target QoI depends on design variables and physical parameters, including geometry or nondimensional quantities.
  • Problem statement: The objective is to regress a corrective forcing term that reduces the error between corrected low-dimensional and high-dimensional QoIs.The correction acts through the low-dimensional model rather than directly modeling only the observed QoI error.
  • Problem statement: A direct Gaussian process on the high-dimensional-minus-low-dimensional error is purely data-driven and does not encode the underlying problem structure.The numerical examples report that this approach could require more data to reach a desired accuracy.
  • Physics-aware correction: The physics-aware approach postulates a corrective forcing g in the dual test space to produce a corrected low-dimensional solution and account for missing QoI-relevant physics.The forcing term is introduced in the low-fidelity model and is informed by the available physics solver.
  • Physics-aware correction: A Gaussian Functional prior on the corrective forcing induces a Gaussian-process prior over the QoI surface with low-dimensional mean and configuration-to-adjoint covariance features.Conditioning on observations yields posterior mean and covariance for a dimension-bridging surrogate and its uncertainty.
  • Auxiliary LD models: For low-dimensional-invisible geometry, the primary low-dimensional solution and adjoints remain unchanged, so geometry variation enters through the geometry kernel.An auxiliary low-dimensional model can featurize the geometry, and forward primal solutions can remain beneficial even when that auxiliary model is unnecessary.

C. Multiple QoIs

The paper treats multiple QoIs independently and identifies practical issues in shared covariance modeling and hyperparameter optimization.

  • Multiple QoIs: Different kernel parameterizations are learned independently for each QoI, using the appropriate adjoint when the k_V factor is included.The examples concern quantities such as lift, drag, and pitching moment coefficients.
  • Multiple QoIs: Shared integral-kernel cross-covariance produced poor results because stronger sensitivities tended to dominate less sensitive QoIs.The regression of less sensitive quantities could ignore observations in favor of predicted covariances with more sensitive quantities.
  • Hyperparameter optimization: The hyperparameters are inferred by minimizing the Gaussian negative log-likelihood over the data's marginal distribution.The covariance matrix K_TT and prior mean enter the likelihood formulation.
  • Hyperparameter optimization: A regularization term penalizes prior variances that differ substantially from observed squared errors, interpreted as maximum a posteriori estimation.The hyperparameter prior assigns a Gamma distribution whose mode equals the mean observed squared error.
  • Hyperparameter optimization: L-BFGS-B optimization uses multiple randomized restarts and rejects numerically singular or zero-valued results.This procedure addresses local minima and numerical failures during hyperparameter fitting.

V. Adaptive Sampling

The paper uses physics-aware uncertainty to adaptively select expensive high-dimensional samples and proposes a neural-network kernel to make online evaluations fast.

  • Adaptive sampling: Expensive high-dimensional solvers make it difficult to determine sufficient samples or configurations for an accurate response surface.Adaptive sampling addresses this by iteratively acquiring samples only when needed.
  • Adaptive sampling: Stationary kernels typically sample far from existing points, whereas the nonstationary adjoint-inner-product kernel can cluster samples where more information is needed.In this setting, those regions correspond to rapidly varying low-dimensional solutions and parameter sensitivities.
  • Adaptive sampling: The acquisition rule selects the point of maximal posterior variance, or the variance sum weighted by measurement precision for multiple QoIs.The candidate and training sets are defined at each iteration, with measurement noise represented by σ_n.
  • Adaptive sampling: The acquisition function is V-optimal and other optimal-design criteria, including A-optimality and I-optimality, are left for future work.The paper explicitly limits its present acquisition strategy to the variance-based form.
  • Neural-network kernel: The neural-network kernel avoids additional forward and adjoint low-dimensional solves during online inference for arbitrary new parameters.This supports large or distribution-defined test sets when fast online inference is desired.
  • Neural-network kernel: The proposed neural-network approach approximates arbitrary symmetric positive-definite kernels with a finite sequence of learned functions and log-weights.Its form is motivated by Mercer’s theorem and neural-network universal approximation.

C. Evaluation

The evaluation workflow combines hyperparameter optimization, optional neural-network kernel training, Gaussian Functional Prediction, and adaptive high-dimensional sampling. Neural-network evaluation scales with O(N_test), enabling efficient posterior computation.

  • Neural-network acceleration: The neural network constructs N_train × N_test kernel-evaluation matrices for Gaussian-process posterior computations.Evaluating at combined training and test inputs can supply the needed feature values, while broadcasting and reduction perform the final kernel computation.
  • Neural-network acceleration: The dominant neural-network cost scales with O(N_test), making online kernel evaluation efficient for unseen parameters.The implementation can evaluate only N_test inputs when training and test parameter vectors coincide.
  • Pre-correction: The method rescales and shifts low-dimensional QoIs using empirical high-dimensional statistics before kernel construction.The scaling can be reconciled with the observation functional because its effect on adjoints is canceled by trained kernel scaling factors.
  • Gaussian Functional Regression: The Gaussian Functional Regression algorithm uses high-dimensional QoI samples together with low-dimensional parameters and kernel samples for training.When a neural network is trained, it evaluates the kernel directly; otherwise, low-dimensional simulations and any needed adjoints are run first.
  • Evaluation workflow: The workflow optimizes kernel hyperparameters, optionally trains a neural network, predicts the QoI posterior, and adaptively samples new high-dimensional data.Acquisition selects the next query by maximizing posterior variance, then reruns optimization until the iteration limit.

VIII. Results

The test problem uses compressible RANS models for an ONERA M6-based continuous blended winglet, with a 2D auxiliary model representing the wingtip plane. Geometry varies through the normalized winglet-height parameter ξ.

  • Model pair: The study corrects aerodynamic coefficients from a 2D simulation to better model values from a full-wing 3D simulation.The 2D auxiliary solve is performed around the winglet-tip airfoil because the root airfoil does not change with ξ.
  • Physics and solvers: The governing equations are compressible Reynolds-Averaged Navier-Stokes equations solved with SU2 and the SST-2003 two-equation linear eddy-viscosity model.The 3D and 2D conservative states contain six and five components, respectively.
  • Winglet geometry: The 3D geometry is based on the ONERA M6 wing and uses ξ to parameterize a continuous blended winglet.ξ is the wingtip-chord height above the ONERA M6 chord plane, normalized by the root chord length C.
  • Geometry behavior: The geometry parameterization has two changing-behavior regions: geometry changes beyond the original wingtip after full cant, and ξ=0 produces an infinite rate of change.Before full cant, only the winglet tip grows; after full cant, taper and sweep are modified to control wingtip size.
  • Auxiliary model: The auxiliary 2D domain is the plane containing the wingtip, changing from span-normal at ξ=0 to a 100° cant as ξ increases.Freestream conditions are projected onto in-plane components, and the 2D wall uses a slip boundary condition consistent with the 3D root-plane symmetry.

1. Varying Winglet Size

The experiments compare PDE-informed Gaussian Functional kernels with stationary Matérn mixtures for varying winglet size and flight condition. The PDE kernels improve regression accuracy and uncertainty guidance, while the neural-network kernel greatly reduces evaluation cost.

  • Adaptive regression: The PDE kernel outperforms a Matérn kernel on the same PDE-acquired dataset and achieves better accuracy for a given number of adaptive iterations.Adaptive acquisition concentrates samples near high-uncertainty regions associated with large adjoint sensitivity.
  • Adaptive regression: Stationary kernels initially miss surface features, whereas PDE kernels resolve them quickly through uncertainty-guided sampling.The stationary kernels become sensitive to shorter length scales only after enough samples have been acquired.
  • Varying flight condition: The adjoint inner product kernel captures rapid variation around the critical Mach number and slower variation beforehand, unlike a stationary kernel.It also produces increased posterior uncertainty near the critical Mach number, which the stationary kernel does not reveal alone.
  • Cost-benefit analysis: 1.93e6 is the reported speedup for evaluating the three QoIs at a new parameter with the neural-network kernel.The comparison accounts for reusing the primal solution across QoIs.
  • Varying flight condition: The Mach-varying PDE kernel follows the high-dimensional QoI more closely and provides more informative uncertainty than the stationary kernel at identical parameters.This strengthens its usefulness for adaptive sampling applications.
  • Cost-benefit analysis: The cost savings and regression benefits depend on the case: more expensive 3D solvers increase benefits, while poorer 2D solvers diminish them.The authors characterize the demonstrated savings as applying to expensive many-query applications.

B. Neural Network Accelerated Kernel

The paper accelerates its PDE-informed Gaussian Functional Regression kernel with a neural network surrogate, preserving kernel and posterior accuracy while enabling real-time evaluation. The resulting construction combines nonlinear model and observation-function features with auxiliary low-dimensional models for neglected geometry effects.

  • Neural network kernel: A standard deep fully connected neural network uses Gaussian Fourier features to learn the kernel over scaled parameters.The final architecture has 3 hidden layers of 512 neurons, 52 series terms, and 8 random Fourier features.
  • Neural network kernel: The neural network adequately replicates the Gaussian-process posterior when used for all kernel evaluations.Small training-matrix errors can significantly affect solved coefficients, so posterior fidelity requires particularly small approximation error.
  • Neural network kernel: Millions-fold speedup makes the neural-network kernel suitable for real-time applications, although it may be unnecessary for a few unconstrained low-dimensional evaluations.The speedup is reported in order-of-millions terms, while the limitation concerns settings without demanding time constraints.
  • PDE-informed kernel: The proposed warped inner-product kernel uses low-dimensional primal and adjoint solutions as parameter feature maps, producing a non-stationary covariance.The construction attenuates covariance and increases variance where the low-dimensional model has higher sensitivity.
  • Auxiliary models: Auxiliary low-dimensional models featurize geometry parameters neglected by the primary model, allowing the product kernel to capture variation across the design space.This extends the method to symmetry-breaking or otherwise neglected parameters.
  • Online acceleration: Neural-network surrogates eliminate repeated forward and adjoint low-dimensional solves during online inference, enabling low-dimensional-physics-informed real-time predictions.The paper presents neural networks as an acceleration strategy for the featurized Mercer kernel.
  • PDE-informed kernel: The PDE-informed kernel captures rapid aerodynamic-forcing changes near critical Mach number and small winglet height that stationary kernels miss without further sampling.These features motivate comparing the product kernel with standard stationary kernels.

Appendix

The appendix formulates Gaussian Functional Regression for correcting low-dimensional quantity-of-interest errors using adjoint-based observations and a Gaussian-process-style posterior. The derivation relies on linearization and becomes exact for linear forward operators and quantities of interest.

  • Adjoint derivation: Adjoint equations are derived from a Lagrangian optimal-recovery problem to obtain the test function required by functional Gaussian-process regression.The forward and adjoint equations are stated in weak form before deriving the quantity-of-interest error estimate.
  • Functional regression formulation: The correction problem is posed as a constrained least-squares or optimal-recovery problem in the kernel-induced reproducing-kernel Hilbert space.The formulation can also be interpreted as a maximum-a-posteriori point, although the paper seeks a constructive regression procedure.
  • Adjoint derivation: The quantity-of-interest error estimate is linearized about the low-dimensional solution and ignores second-order terms, making low-dimensional-model selection a strong assumption.The paper emphasizes balancing low-dimensional cost against accuracy.
  • Adjoint derivation: When the forward operator and quantity of interest are linear, the adjoint-based derivation has no approximation errors.This exactness statement is limited to the linear case.
  • Online prediction: At unseen parameter values, evaluating the required inner products requires computation despite the absence of high-dimensional observations.This motivates the subsequent kernel-acceleration strategy.
  • Posterior construction: Gaussian Functional Regression models the posterior of the correction function from noisy high-dimensional-minus-low-dimensional quantity-of-interest errors.Training observations are treated analogously to standard Gaussian-process regression, with independent observation noise and correction function.
  • Posterior construction: The posterior is defined jointly over training and test adjoint-parameter pairs, yielding a posterior mean and covariance for new quantity-of-interest predictions.The construction uses finite evaluations and a multivariate Gaussian conditional distribution.

XI. Derivation of the Adjoint Inner Product Kernel

The adjoint inner-product kernel converts covariance between functional evaluations into a kernel construction over the associated functions, with adjoint states representing low-dimensional model-error structure. The paper combines GP kernels, matrix-valued covariance parameterizations, and stationary LMC components while distinguishing physical-domain from function-space stationarity.

  • Adjoint feature maps: Evaluating the functional kernel on low-dimensional adjoint states predicts covariance in low-dimensional-model errors.The adjoint solution serves as a feature map for the Gaussian-process covariance.
  • Functional kernel construction: The Gaussian Functional covariance kernel can be constructed from an ordinary Gaussian-process kernel through a Hilbert-Schmidt integral kernel.Mercer-theorem conditions ensure the resulting functional covariance is symmetric positive definite.
  • Matrix-valued covariance: For vector-valued functions, the integral kernel is a symmetric positive-definite matrix, permitting nonstationary covariance through an LU-based parameterization.The parameterization permits asymmetric cross-covariance matrices while preserving positive semidefinite collocated covariances.
  • Stationarity distinction: Stationary Gaussian-process covariance is stationary in the physical domain, whereas stationary Gaussian-Functional covariance would depend on distance in function space.A stationary GP representation can therefore assign smaller covariance to translated functions even when their function-space distance is unchanged.
  • Stationarity distinction: The paper uses a stationary GP covariance with Linear Model for Coregionalization, producing a physically stationary diagonally weighted matrix kernel.This choice is made despite the distinction between stationary GP and stationary GF behavior.
  • Matrix-valued covariance: A lower-triangular factorization reduces the number of parameters needed to represent the full-rank covariance matrix.The factorization is equivalent to absorbing the diagonal scaling into a stationary Cholesky factor.
Loading 2608.27639v1…