Source-linked AI summary
The Random Feature Model for Input-Output Maps between Banach Spaces
Nicholas H. Nelsen, Andrew M. Stuart
TL;DR
The paper addresses the challenge of learning PDE solution operators between infinite-dimensional function spaces from limited data. It develops a random feature emulator and demonstrates accurate, scalable approximation for viscous Burgers’ and variable-coefficient elliptic equations.
Problem
Scientific applications often provide limited high-resolution simulation or experimental data, motivating fast approximate emulators for full-order models.
Method
The paper formulates random feature learning between Banach spaces, reducing training to finite-dimensional convex optimization through operator-valued kernels.
Results
The random feature model accurately approximates nonlinear solution operators for viscous Burgers’ and variable-coefficient elliptic equations while supporting discretization-robust deployment.
Takeaways & Limitations
Random features provide a non-intrusive, scalable emulator for infinite-dimensional PDE input-output maps when suitable computable bases are unavailable.
Takeaways & Limitations
The method has an additional approximation gap when the target map does not belong to the assumed operator-valued RKHS.
Abstract
from arXiv · showhide
Well known to the machine learning community, the random feature model is a parametric approximation to kernel interpolation or regression methods. It is typically used to approximate functions mapping a finite-dimensional input space to the real line. In this paper, we instead propose a methodology for use of the random feature model as a data-driven surrogate for operators that map an input Banach space to an output Banach space. Although the methodology is quite general, we consider operators defined by partial differential equations (PDEs); here, the inputs and outputs are themselves functions, with the input parameters being functions required to specify the problem, such as initial data or coefficients, and the outputs being solutions of the problem. Upon discretization, the model inherits several desirable attributes from this infinite-dimensional viewpoint, including mesh-invariant approximation error with respect to the true PDE solution map and the capability to be trained at one mesh resolution and then deployed at different mesh resolutions. We view the random feature model as a non-intrusive data-driven emulator, provide a mathematical framework for its interpretation, and demonstrate its ability to efficiently and accurately approximate the nonlinear parameter-to-solution maps of two prototypical PDEs arising in physical science and engineering applications: viscous Burgers' equation and a variable coefficient elliptic equation.
1. Introduction.
The paper develops a data-driven random feature methodology for learning black-box input-output maps between infinite-dimensional function spaces, motivated by limited high-resolution scientific data. It positions this non-intrusive approach as a surrogate for PDE solution operators, distinct from pointwise PDE-solving neural networks and methods with only one high-dimensional space.
- Motivation: Limited high-resolution data from simulations or experiments motivates fast approximate solvers that emulate full-order models efficiently and accurately.
- Contributions: The methodology learns black-box input-output maps between Banach spaces directly from data, with a reproducing kernel Hilbert space as the natural hypothesis space.It formulates the approximation problem as supervised learning in infinite dimensions and targets maps from a Banach space to a separable Hilbert space.
- Related work: Unlike point-collocation neural networks that solve prescribed initial-boundary value problems, the approach emulates the infinite-dimensional map from input parameters to PDE solutions or quantities of interest.
- Contributions: The random feature model is a simple randomized-basis expansion and is proposed here as a non-intrusive function-space learning method for settings where only data, not a known model, is available.
2. Methodology.
The methodology formulates PDE solution-map approximation as supervised learning between Banach spaces and constructs an infinite-dimensional random feature model using operator-valued reproducing kernels. Random features yield a finite-dimensional, convex penalized least-squares method while avoiding the computational costs of full kernel regression.
- The method learns a parametric approximation F(·,α) to a possibly nonlinear map F†:X→Y from input-output training pairs in Banach spaces.The framework accommodates PDE-generated outputs, model or discretization error, and potentially noisy evaluations.
- Although Monte Carlo approximation achieves O(m−1/2), its direct coefficient formula is impractical because the target map, kernel, and RKHS are unknown and the target may lie outside the RKHS.The practical method therefore estimates parameters from data rather than evaluating ideal coefficients analytically.
- The random feature model expands outputs in randomized basis functions, with the feature map implicitly defining the operator-valued kernel rather than being derived from a preselected kernel.This construction defines a random finite-dimensional hypothesis space H_k(m) and estimates coefficients directly from data.
- Training the random feature model with an L2-type penalized least-squares loss is convex and requires solving only a finite-dimensional linear system.The resulting hypothesis class is equivalent to kernel ridge regression restricted to the random-feature space H_k(m).
- The random-feature formulation avoids full kernel regression, whose operator-valued representation can require prohibitively expensive block linear solves in high-dimensional PDE applications.This computational bottleneck motivates learning coefficients using only the available random features.
3. Application to PDE Solution Maps.
The paper constructs PDE-specific random feature maps for viscous Burgers’ and Darcy solution operators, using priors and structural knowledge to guide data-driven surrogate design. The resulting features exploit smoothing, filtering, or predictor-corrector structure while remaining non-intrusive.
- 3. Application to PDE Solution Maps: The section develops random feature maps for two PDE parameter-to-solution operators: viscous Burgers’ evolution and Darcy coefficient-to-solution mappings.The feature-map and measure choices determine the induced kernel and strongly affect reconstruction quality, although the overall methodology is data-driven.
- 3.1 Viscous Burgers’ Equation: For viscous Burgers’, Fourier-space features apply filtered random convolutions to initial data, reshaping low-to-medium wavenumbers and suppressing high frequencies.The Burgers solution map is smoothing and nonlinear; the hand-tuned filter reflects this prior structure.
- 3.1 Viscous Burgers’ Equation: ELU activation outperforms ReLU, tanh, sigmoid, sin, SELU, and softplus in the random-feature framework for the Burgers’ problem.Sobolev embedding ensures pointwise ELU evaluation is well defined in the chosen output space for sufficiently large regularity.
- 3.2 Darcy Flow: For Darcy flow, predictor-corrector features use thresholded Gaussian fields and a smoothed coefficient to approximate the nonlinear coefficient-to-solution map.The coefficient prior generates two-valued, high-contrast permeability fields with random geometry, while smoothing makes ∇a usable in feature evaluation.
- 3.2 Darcy Flow: The Darcy feature map is a randomized perturbation of one predictor-corrector iteration, but may require full PDE knowledge and cost as much as solving the original equation.Removing its random terms yields a remarkably accurate surrogate, while larger enclosing domains could enable FFT-based acceleration.
4. Numerical Experiments.
Numerical experiments on Burgers’ equation and Darcy flow show that the random feature model accurately approximates PDE solution operators while transferring across mesh resolutions. The method achieves low test error, captures sharp solution interfaces, and exhibits mesh-resolution invariance consistent with its function-space formulation.
- Burgers’ equation: The Burgers’ RFM remains accurate after three compositions, with relative error increasing by only around 0.04 over 1.5 time units beyond the training time.The model learns time evolution without explicitly time-stepping the PDE, demonstrating out-of-distribution time-upscaling performance.
- Burgers’ equation: Training and testing across mesh resolutions preserves low error for Burgers’ equation, while learned coefficients converge as resolution increases, supporting mesh-invariant function-space approximation.The lowest error occurs when training and testing resolutions match at very low resolution, where discretization error dominates; beyond that regime, error is approximately invariant.
- Burgers’ equation: 0.0303 is the smallest expected relative test error reported for the Burgers’ solution operator, matching the order of magnitude of prior results.The comparison is made with a Neural Operator study using deep learning and non-convex stochastic optimization.
- Darcy flow: For Darcy flow, predictor-corrector random features capture steep coefficient-induced interfaces with slight smoothing, while test error improves with more random features and approaches a constant under refinement.The model can be trained and tested at different resolutions, and its basis functions permit direct evaluation on any desired mesh resolution after training.
5. Conclusions.
The paper introduces a non-intrusive random feature methodology that reduces infinite-dimensional operator learning to finding m real numbers while remaining scalable, discretization-robust, and continuum-consistent. Numerical experiments support these properties, and future work targets harder scientific applications, inverse and design problems, and data-adaptive random features.
- Conclusions: The random feature model approximates maps between infinite-dimensional Banach spaces by reducing the learning problem to finding m real numbers.This formulation is presented as a dimension-reduction approach for infinite-dimensional learning.
- Conclusions: The non-intrusive algorithm is scalable, continuum-consistent, robust to discretization, and flexible in practical use, with benefits verified on two nonlinear PDE forward operators.The experiments involve one operator based on a semigroup and another operator described in the supplied passage.
- Future work: Future work will apply random feature methods to challenging scientific problems, design and inverse problems, and the development of random features adapted to data.The proposed settings include climate modeling and material modeling, with the random feature model serving as a cheap emulator for design and inverse problems.
Appendix A. Proofs of Results.
Appendix A proves the paper’s key RKHS results by constructing the relevant inner product, identifying the image space with the induced RKHS, and establishing finite-feature representation properties. The proofs also use linear independence to guarantee uniqueness and derive the RKHS norm in coefficient form.
- Proof of Result 2.5: The image space im(A) is shown to equal the RKHS Hkµ induced by kµ through a constructed inner product and reproducing property.The argument views A as a bijection between its coimage and image spaces and verifies the reproducing identity.
- Proof of Result 2.6: Result 2.6 is proved by applying the preceding RKHS identification to the finite-dimensional feature-map setting, assuming the feature functions are linearly independent.
- Proof of Result 2.8: Result 2.8 establishes that finite-feature representations are unique under linear independence and expresses the corresponding RKHS norm in terms of the coefficient vector.The proof uses the reproducing kernel property and substitutes the coefficient-based norm into the stated equivalence.
Appendix B. Further Remarks on Integral Representation of RKHS.
This appendix clarifies the integral representation of RKHS elements through the operator A. It identifies the unique orthogonal coefficient representation and shows that A and its adjoint provide square-root representations of the integral operator.
- Operator representation: A is shown to be a square root of the integral operator T_kµ, with H_kµ represented as the image of square roots of T_kµ.The appendix establishes AA*=T_kµ and relates H_kµ to the corresponding operator images.
- Integral representation: Each F ∈ H_kµ has a unique coefficient c_F in ker(A)⊥, making Equation (2.12) formal notation for the restricted inverse Ã^-1F.The decomposition L2_µ = ker(A) ⊕ ker(A)⊥ separates the nonunique kernel component from the unique representing coefficient.
- Adjoint calculation: The adjoint formula for A is derived using Fubini–Tonelli, which also verifies the identity AA*=T_kµ.The appendix uses Fubini–Tonelli in the adjoint calculation and in the final operator identity.