Operator Learning: Algorithms and Analysis

1. Introduction.

• The paper reviews operator learning algorithms and analysis, focusing primarily on neural operators while also covering learned linear operators, Gaussian processes, and random features.Its stated perspective and literature review are followed by an overview of the paper's remaining sections.Source: §1 · p. 1

1.1. High Dimensional Vectors Versus Functions.

• Viewing discretized data as functions can capture intrinsic problem properties rather than pixellation details, allowing learned models to transfer across discretization levels.The introduction illustrates this perspective with images and other fields represented as functions.Source: §1 · p. 1

1.2. Literature Review.

• The literature spans continuum algorithms, supervised learning between function spaces, universal approximation, and complexity analysis for neural operators and learned linear operators.The review also notes related work on PDEs, Koopman operators, and time-series from a continuous viewpoint.Source: §1.2 · pp. 2–4

1.3. Overview of Paper.

• The paper proceeds from operator-learning formulation and architectures to universal approximation and complexity questions, then concludes.The review asks how many parameters or samples are required for a specified approximation accuracy and what operator properties can reduce complexity.Source: §1.3 · p. 4

2.1. Supervised Learning.

• Supervised learning seeks an underlying map from input-output samples, and the article extends this setup beyond finite-dimensional regression or classification.The data model based on a measure over inputs is presented as a sufficient special case for exposition.Source: §2.1 · p. 4

2.2. Supervised Learning of Operators.

• In scientific and engineering applications, operator learning generalizes supervised learning to separable Banach spaces of vector-valued functions.The operator is approximated from data by a parameterized family.Source: §2.2 · pp. 4–5

2.3. Training and Testing.

• Training selects model parameters by empirical risk, while testing evaluates error on a measure supported on the input distribution.The population risk requires knowledge of the entire input probability measure; test measures may also be chosen to assess robustness.Source: §2.3 · p. 5

2.4. Finding Latent Structure.

• Learned encoders map input functions to finite-dimensional latent spaces, and decoders map latent outputs back to functions, reducing operator approximation to a finite-dimensional problem.The section illustrates the latent structure with Figure 3.Source: §2.4 · p. 5, Figure 3 · p. 6

2.5. Example (Fluid Flow in a Porous Medium).

• The porous-medium example maps permeability to piezometric head through the Darcy law, with positive, bounded permeability inputs required for a well-defined solution.The input operator maps a subset of L∞(D) into H¹₀(D), and the input measure is chosen to ensure positivity and boundedness.Source: §2.5 · p. 6

3.1. PCA-Net.

• PCA-Net uses data-driven PCA encoders and decoders around a finite-dimensional neural network to approximate an operator between Hilbert spaces.Its input and output PCA dimensions are architecture hyperparameters.Source: §3.1 · p. 7

3.2. DeepONet.

• DeepONet combines a linear input encoder, a finite-dimensional branch network, and a neural-network-basis decoder called the trunk network.The encoding may use PCA coefficients or pointwise sensor observations.Source: §3.2 · pp. 7–8

3.3. FNO.

• FNO generalizes neural-network layers directly to function spaces, using integral-kernel convolutions evaluated through Fourier multipliers and a finite Fourier cutoff.Its architecture composes input, hidden, and output layers; practical implementations commonly use an equidistant grid and FFT.Source: §3.3 · pp. 8–9

3.4. Random Features Method.

• Random feature models fix randomly drawn operator features and optimize only their coefficients, yielding a convex optimization problem with conventional loss functions.The review notes that suitable random features are problem-dependent and describes Fourier-space features used in the original work.Source: §3.4 · pp. 9–10

3.5. Discussion.

• PCA-Net and DeepONet outputs lie in a fixed finite-dimensional linear subspace, whereas FNO and random-feature outputs can lie on a nonlinear manifold.The encoder-decoder structure parallels numerical schemes but replaces hand-crafted algorithms and discretization choices with data-driven weights and encoding; operator benefits of nonlinear approximation remain sparse, with some specific cases observed.Source: §3.5 · p. 10
• Random features permit error and convergence analysis accounting for finite samples, parameters, and optimization, but designing efficient operator-learning features remains largely unresolved.The discussion also identifies active work beyond neural-network approaches, including nonlinear reduced-order models, Koopman methods, solver augmentation, closure modeling, and time-series.Source: §3.5 · p. 11

4.1. Encoder-Decoder-Nets.

• For separable Banach spaces with the approximation property, encoder-decoder nets approximate continuous operators to arbitrary accuracy on compact input sets.The approximation property holds for many function spaces used in PDE theory and numerical analysis.Source: §4.1 · p. 12

4.1.1. Operator Network and DeepONet.

• DeepONet is universal for continuous operators on compact sets of continuous functions, and the result extends to broader operator domains and deep branch- and trunk-nets.The stated result uses a non-polynomial activation and a compact domain with Lipschitz boundary; error is measured in the supremum norm over a compact input set.Source: §4.1.1 · p. 13

4.1.2. PCA-Net.

• PCA-Net approximates measurable operators between separable Hilbert spaces in expected Bochner L2(μ) error, under the theorem’s moment conditions.The expectation includes iid samples that determine the empirical PCA encoder and reconstruction.Source: §4.1.2 · p. 14

4.2.1. Averaging Neural Operator.

• The averaging neural operator removes non-essential FNO components for universal approximation and still approximates continuous operators on compact sets to arbitrary accuracy.Its layers use an integral average; the corollary extends universality to neural-operator architectures that can represent constant kernels, including the FNO.Source: §4.2 · p. 15, §4.2.1 · p. 15

4.2. Neural Operators.

• Neural-operator implementations vary in their parametrized integral kernels, and architectures that can represent every constant matrix-valued kernel satisfy the stated universality criterion.The cited examples use Fourier, wavelet, and spherical-harmonic bases, among other kernel choices.Source: §4.2 · p. 14, §4.2.1 · p. 15

4.2.1. Averaging Neural Operator.

• A shallow averaging neural operator can be viewed as a spatial-averaging encoder into a finite-dimensional latent space followed by a nonlinear decoder.This encoder-decoder interpretation provides a path for analyzing its universality.Source: §4.2.1 · p. 16

• Quantitative operator-learning theory seeks explicit error and complexity estimates to determine model size or sample count for a target accuracy.The section contrasts these estimates with universality, which is inherently qualitative and cannot explain practical efficiency.Source: §5 · p. 17

• Learning a linear operator on a separable Hilbert space is formulated as a linear inverse problem and studied through its eigenvalues in a known Schauder basis.The authors use this linear setting to analyze complexity and shed light on the more general operator-learning problem.Source: §5.1 · p. 17
• The analysis assumes Gaussian input with positive covariance diagonalizable in the known basis, and noisy observations; the cited work also treats cases without simultaneous diagonalizability.The noise makes data requirements meaningful in this setup; without noise, the eigenvalues can be recovered from one data point under the known-basis assumption.Source: §5.1 · p. 17
• The input observations include noise, which can model round-off or discretization errors even when practical observations are not noisy.Without noise, the known-basis eigenvalues can be recovered from a single data point in the stated setup.Source: §5.1 · p. 17
• A Bayesian posterior for the eigenvalue sequence yields a Gaussian-mean estimator, with the truth’s smoothness specified through its eigenvalues.The estimator is diagonalized in the same basis.Source: §5.1 · pp. 17–18
• Theorem 5.1 gives an asymptotic error bound under specified covariance and noise decay and smoothness conditions, and the resulting sample rate quantifies data needed for a target error.The rate’s dependence on truth, input, prior, and error-metric smoothness parameters informs estimator design.Source: §5.1 · p. 18

5.2. Holomorphic Operators.

• Holomorphic operators are studied mainly through parameterized inputs represented by coefficients in a basis or frame expansion, including the elliptic Darcy flow example.With sufficiently fast coefficient decay, the solution operator in this setting has a holomorphic extension.Source: §5.2 · pp. 18–19
• In parameterized settings, holomorphic operators admit convergent multivariate polynomial expansions whose basis elements depend on finitely many input components.This provides the basis for efficient approximation; practical parametric differential equations can give rise to (b, ε)-holomorphic operators.Source: §5.2 · p. 19
• Best n-term polynomial approximation converges at rates n^(1−1/p) in the sup norm and n^(1/2−1/p) in Bochner L2(µ), independent of finite input dimension d.Under the stated assumptions, these dimension-independent rates can overcome the curse of dimensionality.Source: §5.2 · p. 19
• DeepONets can approach rates n^(1−s) in worst-error supremum norm and n^(1/2−s) in Bochner L2(µ), with n parameters and s determined by input-frame coefficient decay.The review relates these rates to best n-term approximation rates under a loose identification s ∼ 1/p.Source: §5.2 · p. 19
• For (b, ε)-holomorphic operators in a Bochner L2 setting, N-sample methods cannot exceed the rate N^(1/2−1/p), and this optimal rate is achievable up to logarithmic terms [6] [7].These sample-complexity results rely on N-width theory.Source: §5.2 · p. 20

5.3.1. Error Decomposition

• Encoder–decoder-net error splits into input encoding, latent neural-network approximation and output reconstruction contributions, E ≲ EU + Eψ + EV.The method encodes the input and output function spaces into finite-dimensional latent spaces and approximates the resulting encoded operator with a neural network.Source: §5.3.1 · pp. 20–21
• For discontinuous PDE outputs, nonlinear reconstruction overcomes limitations of linear reconstruction: FNO and shift-DeepONet achieved higher accuracy than vanilla DeepONet theoretically and experimentally.The section also notes related nonlinear manifold decoder work.Source: §5.3.1 · p. 21
• For W^k,∞([0, 1]^d), a ReLU network reaches uniform accuracy ε with O(ε^-d/k log(ε^-1)) parameters; for an operator’s latent input dimension dU, the stated componentwise scaling is O(dVε^-dU/k).The review cautions that architectures reducing parameter counts below ε^-d/k must require greater parameter precision, keeping total storage above the model-class entropy limit.Source: §5.3.1 · p. 22

5.3.2. Upper Complexity Bounds

• Work on upper complexity bounds includes error estimates for operator networks and a mimetic architecture whose error analysis separates numerical approximation, stability, training and quadrature contributions.The section also surveys analyses of PCA-Net, DeepONet, basis operator networks and convolutional networks, among others [47].Source: §5.3.2 · p. 23

5.3.3. Lower Complexity Bounds

• The continuous nonlinear n-widths for the functional class studied in decay only poly-logarithmically, implying a lower bound on latent parameters for continuous encoder–decoder approximation.The comparison to a neural-network lower bound assumes a fixed architecture and continuous assignment of optimal weights; that continuity may fail under gradient-based optimization.Source: §5.3.3 · p. 25
• Standard architectures including DeepONet, FNO and PCA-Net cannot overcome parametric complexity on the full class of Lipschitz or Fréchet-differentiable operators.Analogous lower complexity bounds apply to PCA-Net, FNO and other architectures, with more general results for Sobolev input functions [78].Source: §5.3.3 · pp. 25–26
• Non-standard architectures let DeepONets achieve algebraic approximation-rate bounds for general Lipschitz and Hölder-continuous operators, breaking the parametric-complexity curse [123].These architectures use non-standard activations or three-dimensional connectivity and reduce the neural-network approximation error with complexity growing more slowly than the rough standard scaling.Source: §5.3.3 · p. 26

5.3.4. Sample Complexity Results

• For sample complexity, derives non-asymptotic generalization-error upper bounds for empirical risk minimizers in a Bochner L2(µ) setting, combining best approximation estimates with statistical learning theory.The analysis covers general, fixed-basis and PCA encoder–decoder pairs; the supplied passage introduces a representative fixed trigonometric-basis result but does not provide its bound.Source: §5.3.4 · pp. 26–27

5.3.4. Sample Complexity Results.

• The error bound separates encoding and reconstruction errors from combined neural-network approximation and generalization errors.Here N denotes the number of samples, s = k + α is smoothness on the input and output spaces, and ψ is a ReLU network of depth L and width p.Source: §5.3.4 · p. 27
• Choosing dU,dV ∼ ε^-d/s and N ∼ ε^-(2+dU)/2 to reach error E ≤ ε yields an exponential sample-complexity curse as dU depends on ε.The resulting analysis also gives a lower bound on the neural network size Lp.Source: §5.3.4 · p. 27
• Work suggests these Lipschitz-operator results are unlikely to improve substantially, while low-dimensional latent input structure can yield fast-rate error bounds under suitable conditions.Source: §5.3.4 · p. 27
• Work [62] estimates FNO Rademacher complexity to derive generalization bounds and compares the resulting insights with empirical generalization error and proposed FNO capacity in numerical experiments.Work [13] discusses out-of-distribution risk bounds for neural operators, focusing on the Helmholtz equation.Source: §5.3.4 · pp. 27–28
• With noisy measurements, work [100] shows that accurate DeepONet approximation requires the number of trunk basis functions dV ≳ √n.Source: §5.3.4 · p. 28

5.4. Structure Beyond Smoothness.

• For general Lipschitz or Fréchet differentiable operators, standard architectures may require exponentially many parameters, whereas holomorphic operators admit algebraic complexity bounds.The section contrasts these settings with exploiting structure beyond smoothness.Source: §5.4 · p. 28
• Operator Barron spaces identified in form a general class, distinct from holomorphic operators, permitting efficient approximation by shallow neural operators; a priori, practical-interest membership is unclear, leaving theory-to-practice efficiency links tenuous.Source: §5.4 · p. 28
• Leveraging representations or methods that are efficiently approximated by classical numerical schemes can let operator-learning frameworks overcome parametric complexity for specific operators.The review notes that an abstract characterization of exploitable structure is not yet available.Source: §5.4 · p. 29
• For Darcy flow, Navier–Stokes, reaction-diffusion, and inviscid Burgers operators, emulating numerical methods yields algebraic, and in some settings logarithmic, model-size scaling in ε.The cited results concern specific PDE solution operators.Source: §5.4 · p. 29

5.4.1. Discussion.

• Neural-network emulators add translation-to-weights overhead to the complexity of the numerical algorithm they emulate, motivating a general theory of efficient algorithm emulation.The review presents this as a conjectural direction and an open formalization problem.Source: §5.4.1 · p. 29

6. Conclusions

• Universal approximation results for many neural-operator architectures establish approximation of wide classes of infinite-dimensional operators, but do not ensure practically realistic model sizes.The review treats this as a qualitative guarantee rather than a feasibility result.Source: §6 · p. 30
• For general Lipschitz or Fréchet differentiable operators, standard architectures require exponentially many parameters for error ε, while holomorphic operators admit algebraic model-size and sample-complexity bounds.The review identifies these as distinct settings in quantitative complexity analysis.Source: §6 · p. 30
• Beyond smoothness, operator-learning frameworks can achieve algebraic convergence by leveraging intrinsic operator structure, but current results rely on case-by-case analyses and often emulate numerical methods.The review identifies general characterization of exploitable structure as a major challenge.Source: §6 · p. 30