PaperlayerView exact revision on arXiv

Source-linked AI summary

Fourier Neural Operator for Parametric Partial Differential Equations

Zongyi Li, Nikola Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, Andrew Stuart, Anima Anandkumar

arXiv:2010.08895v3cs.LGmath.NA

TL;DR

Repeatedly solving parameterized PDEs is computationally demanding, so the paper introduces a Fourier neural operator that learns solution operators across function spaces. Across Burgers’ equation, Darcy flow, and Navier–Stokes, it outperforms existing deep-learning methods and supports zero-shot super-resolution, while offering substantially faster inference than traditional solvers.

  • Problem

    Learning an operator for an entire PDE family is more challenging than solving one PDE instance, while repeated fine-grid simulations remain slow.

  • Method

    The Fourier neural operator learns mappings between function spaces by parameterizing a convolutional integral operator through a linear transformation in the Fourier domain.

  • Results

    It reduces error by 30% on Burgers’ equation, 60% on Darcy flow, and 30% on turbulent Navier–Stokes at fixed 64×64 resolution, while enabling zero-shot super-resolution.

  • Takeaways & Limitations

    On a 256 × 256 grid, inference takes 0.005s versus 2.2s for the pseudo-spectral Navier–Stokes solver without accuracy degradation in a Bayesian inverse problem.

  • Takeaways & Limitations

    Training data can be expensive to generate, requiring 10,000 numerical-solver training pairs for turbulent Navier–Stokes at viscosity ν = 1e−4.

Abstract

from arXiv · show

The classical development of neural networks has primarily focused on learning mappings between finite-dimensional Euclidean spaces. Recently, this has been generalized to neural operators that learn mappings between function spaces. For partial differential equations (PDEs), neural operators directly learn the mapping from any functional parametric dependence to the solution. Thus, they learn an entire family of PDEs, in contrast to classical methods which solve one instance of the equation. In this work, we formulate a new neural operator by parameterizing the integral kernel directly in Fourier space, allowing for an expressive and efficient architecture. We perform experiments on Burgers' equation, Darcy flow, and Navier-Stokes equation. The Fourier neural operator is the first ML-based method to successfully model turbulent flows with zero-shot super-resolution. It is up to three orders of magnitude faster compared to traditional PDE solvers. Additionally, it achieves superior accuracy compared to previous learning-based solvers under fixed resolution.

1 INTRODUCTION

Traditional PDE solvers face costly resolution–accuracy trade-offs, while existing neural methods are discretization- or instance-dependent. The Fourier neural operator addresses these limitations with a resolution-invariant operator and demonstrates strong accuracy, super-resolution, and speed results.

  • Motivation: Traditional PDE solvers require repeated fine discretizations, making complex systems slow and computationally challenging.Coarse grids are faster but less accurate, whereas fine grids are more accurate but slower.
  • Related approaches: Classical neural networks learn solutions tied to specific discretizations, whereas neural operators use one parameter set across different discretizations.Finite-dimensional approaches cannot generally query solutions at new domain points or resolutions.
  • Our Contributions: The Fourier neural operator learns mappings between infinite-dimensional function spaces by instantiating a convolutional integral operator as a Fourier-domain linear transformation.Its architecture combines global integral operations with nonlinear local activations.
  • Our Contributions: The method is the first to learn a resolution-invariant solution operator for turbulent Navier–Stokes equations, where previous graph-based neural operators do not converge.Its shared parameters enable zero-shot super-resolution from lower-resolution training data to higher-resolution evaluation.
  • Our Contributions: 30% lower error rates are achieved on Burgers’ Equation and Navier Stokes, while Darcy Flow achieves 60% lower error rates than existing deep learning methods at 64×64 resolution.For entire time-series mapping, error is < 1% at viscosity ν = 1e−3 and 8% at viscosity ν = 1e−4.
  • Our Contributions: 0.005s inference time is achieved on a 256 × 256 grid, compared with 2.2s for the pseudo-spectral Navier–Stokes method, without downstream accuracy degradation.The comparison is reported for solving a Bayesian inverse problem.

2 LEARNING OPERATORS

The methodology learns nonlinear solution operators between infinite-dimensional function spaces from finite observed input-output pairs. It uses parametric neural operators, including Fourier layers, to approximate PDE solution maps more efficiently than solving individual instances.

  • Learning the Operator: The framework approximates a typically nonlinear map G†: A → U between separable Banach spaces of functions using finite-dimensional parameters θ.The target maps arise as solution operators of parametric PDEs, and the approximation is selected so Gθ† ≈ G†.
  • Learning the Operator: 0: the method learns operators from finite collections of observed input-output pairs, with inputs sampled independently and outputs potentially corrupted by noise.The observations satisfy aj ∼ µ and uj = G†(aj), subject to possible noise.
  • Learning the Operator: The operator is trained and evaluated through data-driven empirical approximations to a cost functional in a test-train setting.Existence of minimizers in the infinite-dimensional formulation remains an open problem, motivating empirical approximations.
  • Learning the Operator: Approximating an operator is typically more challenging than solving one PDE instance, while classical solvers and PINNs target individual solutions.The paper motivates operator learning for applications requiring solutions across many parameter instances.
  • Fourier Layers: Fourier layers transform inputs, apply learned linear transforms to lower Fourier modes while filtering higher modes, then invert the transform and add a local linear transform.The neural-operator architecture applies integral-operator and activation layers before projecting back to the target dimension.
  • Discretization: The approach can produce u(x) at arbitrary domain points, including points outside the discretization used for observed functions.This discretization-invariant property is presented as desirable for numerical work with pointwise function evaluations.

3 NEURAL OPERATOR

The neural operator iteratively lifts input functions into higher-dimensional representations and updates them through learned kernel integral operators, linear maps, and nonlinear activations. Under translation-invariant kernels, this framework becomes convolutional and motivates Fourier-space parameterization with FFT computation for a fast PDE architecture.

  • Neural operator: The architecture lifts a function-valued input a into a higher-dimensional representation v0 before applying iterative updates v0 7→ v1 7→ . . . 7→ vT.The lifting map is v0(x) = P(a(x)), with P usually parameterized by a shallow fully-connected neural network.
  • Neural operator: Each update combines a learned operator K(a; φ), a linear transformation W, and a component-wise nonlinear activation σ.K maps to bounded linear operators on U(D; Rdv), while W : Rdv → Rdv and σ : R → R.
  • Neural operator: The operator K(a; φ) is implemented as a kernel integral transformation whose kernel function κφ is learned from data through a neural network.Together with nonlinear activations, these integral operators generalize neural networks to infinite-dimensional spaces and can represent highly nonlinear operators.
  • Fourier parameterization: Removing dependence on a and imposing κφ(x, y) = κφ(x−y) converts the kernel integral operator into a convolution operator.This translation-invariant form is motivated by fundamental solutions.
  • Fourier parameterization: Fourier-space parameterization of κφ combined with the Fast Fourier Transform efficiently computes the convolution operator and yields a fast architecture with state-of-the-art PDE results.The paper develops this approach in the following section.

4 FOURIER NEURAL OPERATOR

The Fourier neural operator replaces the kernel integral operator with a convolution whose kernel is parameterized directly in Fourier space. Truncated Fourier modes yield an efficient, discretization-invariant architecture capable of zero-shot super-resolution.

  • Fourier integral operator: The proposed operator defines convolution in Fourier space and directly parameterizes the kernel’s Fourier transform with complex-valued weights.The construction uses the convolution theorem and represents the truncated modes as a learnable tensor.
  • Mode truncation: The Fourier representation is finite-dimensional by truncating the Fourier series to a maximal set of modes kmax.The parameter tensor has shape kmax × dv × dv and stores the retained complex Fourier modes.
  • Implementation: Choosing kmax,j = 12 yields kmax = 12d parameters per channel and was sufficient for all considered tasks.The retained modes correspond to the corners of the implementation tensor, enabling parallel matrix-vector multiplication.
  • Invariance to discretization: Fourier layers are discretization-invariant because functions can be evaluated on arbitrary discretizations by projecting onto globally defined Fourier basis functions.This property enables zero-shot super-resolution and produces consistent error across resolutions.
  • Quasi-linear complexity: Truncation reduces the Fourier-transform cost to O(nkmax), while the FFT costs O(n log n), giving the architecture quasi-linear computational complexity.The inner multiplication costs O(kmax), so Fourier transforms dominate the computation.

5 NUMERICAL EXPERIMENTS

The numerical experiments compare FNO with finite-dimensional and operator-based methods on Burgers, Darcy Flow, and Navier–Stokes equations. FNO achieves strong accuracy, resolution invariance, zero-shot super-resolution, and substantial evaluation-speed advantages.

  • Burgers’ equation: FNO obtains the lowest relative error on Burgers while maintaining resolution-invariant error, unlike convolutional neural network methods whose error grows with resolution.It is also more accurate and computationally efficient than GNO and MGNO, which use Nyström sampling in physical space.
  • Darcy Flow: Nearly one order of magnitude lower relative error than every benchmark is achieved on Darcy Flow, with error again invariant to resolution.FNO learns the nonlinear operator mapping the diffusion coefficient to the solution despite the PDE itself being linear.
  • Navier–Stokes equation: FNO-3D performs best for Navier–Stokes when sufficient data are available, while all methods exceed 15% error in data-insufficient configurations and FNO-2D performs best there.The benchmark comparison is reported at 64 × 64 spatial resolution because the competing methods are designed for that resolution.
  • Resolution transfer: Training on 64 × 64 × 20 data and evaluating at 256 × 256 × 80 demonstrates zero-shot space-time super-resolution without higher-resolution training data.The neural operator is mesh-invariant, and FNO is the only model among the listed benchmarks reported to demonstrate this capability.
  • Computational efficiency: 0.005s per instance for FNO versus 2.2s for the optimized traditional solver reduces the reported MCMC runtime from over 18 hours to 2.5 minutes.Both methods recover almost the same posterior mean, which reproduces the late-time Navier–Stokes dynamics when pushed forward.

6 DISCUSSION AND CONCLUSION

The method’s data requirements can be substantial, especially for challenging PDEs, motivating combinations of neural operators with numerical solvers. The discussion also identifies recurrent structure as a consideration.

  • Requirements on Data: N = 10000 training pairs are required to learn the Navier-Stokes equation with viscosity ν = 1e−4.These pairs, {a_j, u_j}, must be generated with a numerical solver.
  • Requirements on Data: Generating even a few training samples can be very expensive for more challenging PDEs.The passage attributes this limitation to the data-driven method’s dependence on data quality and quantity.
  • Requirements on Data: Combining neural operators with numerical solvers is proposed as a future direction to reduce data requirements.The passage frames this combination as a way to alleviate the need for generated training data.
  • Recurrent structure: The discussion also introduces recurrent structure as a consideration for neural operators.The supplied passage begins a subsection titled “Recurrent structure” but does not provide its full explanation.

A APPENDIX · A.1 TABLE OF NOTATIONS

Appendix A.1 provides a table of notations, identified as Table 2.

  • A.1 TABLE OF NOTATIONS: Table 2 is the paper’s table of notations.The passage explicitly states that a table of notations is given in Table 2.

A.2 SPECTRAL ANALYSIS

The spectral analysis shows turbulence-consistent decay in Navier–Stokes data, while the energy spectrum remains stable over time. Across Burgers, Darcy, and low-viscosity Navier–Stokes equations, the Fourier neural operator recovers high-frequency modes despite truncation at kmax = 12.

  • Spectral decay: k−5/3 is the Navier–Stokes spectrum slope, matching the energy spectrum in the turbulence region.The observed decay is consistent with turbulent-flow energy scaling.
  • Spectral decay: The Navier–Stokes energy spectrum does not decay along with time.
  • Truncation analysis: kmax = 12 truncation still allows the Fourier neural operator to recover high-frequency modes across Burgers, Darcy, and Navier-Stokes with ν ≤1e−4.All three equation families exhibit high-frequency modes in the truncation-based spectral analysis.

A.3 DATA GENERATION

This section describes the data generator used for the three equations in Section 5 and presents spectral-decay and truncation-error diagnostics. The diagnostics use wavenumber and truncation mode as their x-axes.

  • The section provides data-generation details for the three equations used in Section 5.
  • Figure 4 presents spectral decay for the Navier–Stokes equations against the wavenumber |k| = k1 + k2.
  • The normalized truncation error is plotted against the truncation mode kmax, as illustrated in Figure 5.

A.3.1 BURGERS EQUATION … A.5 BAYESIAN INVERSE PROBLEM

The paper specifies benchmark PDE settings for Burgers, Darcy flow, and Navier–Stokes, then reports benchmark results and a Bayesian inverse-problem comparison. The Fourier neural operator surrogate matches the traditional solver in the reported Navier–Stokes inverse problem.

  • A.3.1 BURGERS EQUATION: Burgers’ equation is posed on the one-dimensional unit torus with periodic initial data and viscosity ν = 0.1.The equation uses nonlinear advection and diffusion with initial condition u(x, 0) = u0(x).
  • A.3.1 BURGERS EQUATION: The Burgers dataset is generated at spatial resolution 213 = 8192 using a split-step Fourier solver and subsampled to other resolutions.The initial condition follows µ = N(0, 625(−∆+ 25I)−2), and the heat-equation component is solved exactly in Fourier space.
  • A.3.2 DARCY FLOW: Darcy flow is modeled as a two-dimensional second-order linear elliptic equation with spatially varying coefficients and fixed forcing f(x) = 1.The coefficient field is generated by a push-forward Gaussian construction with zero Neumann boundary conditions.
  • A.3.3 NAVIER-STOKES EQUATION: The Navier–Stokes benchmark uses the two-dimensional viscous, incompressible vorticity equation on the unit torus with fixed forcing.The forcing is f(x) = 0.1(sin(2π(x1 + x2)) + cos(2π(x1 + x2))).
  • A.4 RESULTS OF BURGERS’ EQUATION AND DARCY FLOW: Burgers’ equation and Darcy flow error rates are reported in Table 3 and Table 4.The cited passage identifies these tables as the locations of the detailed error-rate results.
  • A.5 BAYESIAN INVERSE PROBLEM: The Bayesian inverse problem for Navier–Stokes is evaluated using results shown in Figure 6.The figure reports the Bayesian inverse-problem results for the Navier–Stokes equation.
  • A.5 BAYESIAN INVERSE PROBLEM: The Fourier neural operator surrogate performs as well as the traditional solver on the Navier–Stokes Bayesian inverse problem.This comparison is stated directly for the results shown in Figure 6.
Loading 2010.08895v3…