PaperlayerView exact revision on arXiv

Source-linked AI summary

Multiwavelet-based Operator Learning for Differential Equations

Gaurav Gupta, Xiongye Xiao, Paul Bogdan

arXiv:2109.13459v2cs.LGmath.AP

TL;DR

Neural operators can struggle with resolution changes and efficient kernel representation when learning PDE solution maps. This paper proposes a multiwavelet-based operator model that sparsely represents kernels across scales, reports higher accuracy than existing neural operators, and predicts high-resolution outputs from lower-resolution training data.

  • Problem

    Existing neural networks may be tied to a training resolution, while direct PDE-solution architectures can require retraining when PDE coefficients change; efficiently representing the complete operator kernel remains challenging.

  • Method

    The MWT model uses multiwavelet filters and projections onto polynomial bases to represent integral-operator kernels across multiple scales, reusing kernel networks within a resolution-independent recurrent structure.

  • Results

    The model achieves higher accuracy than existing neural operators, including an order-of-magnitude relative L2 error improvement on one-dimensional KdV and Burgers equations, and predicts at finer resolutions after lower-resolution training.

  • Takeaways & Limitations

    Multiwavelets provide a sparse kernel basis for efficient operator learning and support solution prediction across resolutions, with potential applications to systems modeled by complex PDEs.

  • Takeaways & Limitations

    The decimation and reconstruction modules require carefully constructed fixed filter matrices; randomly selected matrices do not enable learning.

Abstract

from arXiv · show

The solution of a partial differential equation can be obtained by computing the inverse operator map between the input and the solution space. Towards this end, we introduce a \textit{multiwavelet-based neural operator learning scheme} that compresses the associated operator's kernel using fine-grained wavelets. By explicitly embedding the inverse multiwavelet filters, we learn the projection of the kernel onto fixed multiwavelet polynomial bases. The projected kernel is trained at multiple scales derived from using repeated computation of multiwavelet transform. This allows learning the complex dependencies at various scales and results in a resolution-independent scheme. Compare to the prior works, we exploit the fundamental properties of the operator's kernel which enable numerically efficient representation. We perform experiments on the Korteweg-de Vries (KdV) equation, Burgers' equation, Darcy Flow, and Navier-Stokes equation. Compared with the existing neural operator approaches, our model shows significantly higher accuracy and achieves state-of-the-art in a range of datasets. For the time-varying equations, the proposed method exhibits a ($2X-10X$) improvement ($0.0018$ ($0.0033$) relative $L2$ error for Burgers' (KdV) equation). By learning the mappings between function spaces, the proposed method has the ability to find the solution of a high-resolution input after learning from lower-resolution data.

1 Introduction

The paper targets data-efficient learning of PDE operator maps, motivated by complex PDE-governed systems and limitations of resolution-specific or data-intensive approaches. It proposes multiwavelets to sparsify operator kernels and reports strong accuracy and resolution generalization.

  • PDEs model complex dynamics in systems including aerospace, complex fluids, and cyber-physical processes.
  • Neural-network PDE solvers may be tied to a training resolution, requiring retraining for different discretizations.
  • Learning PDE operators from scarce and noisy data motivates exploiting fundamental operator properties for data-efficient representations.
  • Multiwavelets sparsify smooth operator kernels through vanishing moments, making kernel learning in the multiwavelet domain feasible and data efficient.
  • The proposed model incorporates multiwavelet filters and reports order-of-magnitude relative L2 error improvements for nonlinear KdV and Burgers equations.

2 Operator Learning using Multiwavelet Transform

The method represents integral-operator kernels in multiscale multiwavelet bases, using fixed filters to decompose and reconstruct coefficients. A recurrent neural architecture learns across scales while reusing networks, supporting compact and resolution-independent operator learning.

  • 2.1 Problem Setup: Operator learning maps an input function a to an output function u through T, with T: A → U between Sobolev spaces.
  • 2.1 Problem Setup: Learning the complete integral kernel is generally numerically infeasible, motivating compact subspaces and projection tools that exploit operator properties.
  • 2.2 Multiwavelet Transform: Multiwavelets combine orthogonal-polynomial and wavelet ideas, using vanishing moments and orthogonality to create fine-grained coordinates for operator representations.
  • 2.2 Multiwavelet Transform: Arbitrary-measure multiwavelet filters transform fine-scale coefficients to coarse-scale coefficients and support orthogonal-polynomial bases such as Chebyshev and Gegenbauer.
  • 2.2 Multiwavelet Transform: The non-standard representation untangles scale interactions and yields a sparse, banded kernel representation associated with smoothness.
  • 2.3 Multiwavelet-based Model: The MWT model maps input multiwavelet coefficients through a ladder-down decomposition and ladder-up reconstruction across scales.
  • 2.3 Multiwavelet-based Model: Reusing the same kernel networks across scales makes the model resolution independent and reduces the number of learned parameters.

3 Empirical Evaluation

The MWT model is evaluated across several PDE datasets and input conditions, including varying resolutions, fluctuations, and multiwavelet bases. It consistently achieves strong accuracy and can learn mappings from lower-resolution data.

  • Experimental setup: The MWT model is evaluated against neural-operator benchmarks on Burgers, Darcy Flow, KdV, and related PDE settings.The experiments use Legendre and Chebyshev operator bases and compare against GNO, MGNO, LNO, and FNO.
  • KdV equation: MWT Leg achieves the lowest relative error for KdV across input resolutions and remains effective at resolution 64.The reported result indicates that coarse-resolution data can suffice for learning the function mapping.
  • KdV equation: MWT consistently outperforms recent baselines across varying KdV input fluctuations, including highly fluctuating signals.The comparison varies λ; lower λ corresponds to more frequent fluctuations, while MWT performs well with low Fourier modes in its CNNs.
  • Theoretical properties validation: For the Euler-Bernoulli validation, relative errors become similar across epochs when the number of operator-basis functions satisfies k ≥ 3.For k < 3, the multiwavelets cannot annihilate the kernel diagonal, preventing sparsification and slowing learning.
  • Burgers’ equation: MWT Leg obtains the lowest relative error on Burgers’ equation across resolutions and is an order of magnitude lower than the state-of-the-art.It maintains a very low error even at low resolution, supporting learning from finite-resolution mappings.
  • Darcy Flow: MWT Leg also obtains the lowest relative error on Darcy Flow at various resolutions, while randomly replacing the filters prevents effective operator learning.MWT Rnd performs worse than all compared models, highlighting the importance of the filter matrices.
  • Additional experiments: The MWT model can predict outputs at resolution s = 2048 after training at lower resolution, including Burgers training at s = 256 with relative error 0.0226.Training and testing directly at s = 2048 yields a relative error of 0.00189.

4 Conclusion

The paper addresses data-driven learning of operators between function spaces and develops multiwavelet representations that exploit kernel structure for sparse, efficient learning. It also establishes the polynomial and wavelet preliminaries underlying the proposed models.

  • The work addresses learning operators that map between two function spaces.
  • Multiwavelets provide a sparse kernel basis by combining wavelet locality with orthogonal-polynomial representations.They use vanishing moments, orthogonality, and finite-interval polynomial bases.
  • Wavelet bases are generated through dilation and translation of a mother function across finitely many scales.The work restricts attention to compactly supported wavelets on finite intervals, especially [0,1].
  • The measure-version multiwavelets extend the approach to non-uniform measures and families of finite-domain orthogonal-polynomial bases.The paper gives Legendre and Chebyshev examples and notes extensions to Gegenbauer polynomials.
  • The paper develops multiwavelet transforms and filter operations for decomposing and reconstructing operator representations across scales.The technical development covers orthogonal-polynomial bases, multiwavelet filters, and pseudo-differential operators.

C Derivations for Multiwavelet Filters

This section derives measure-dependent multiwavelet filters that transform coefficients between scales and provides reconstruction formulas and polynomial-basis implementations. It also identifies floating-point precision as a practical constraint on filter computation.

  • Multiwavelet filters transform coefficient representations between fine and coarse scales.The derivation treats decomposition from higher to lower scales and reconstruction in the reverse direction.
  • Measure-variant filters introduce correction terms when basis functions at different scales use different measures.The construction uses tilted bases and Radon-Nikodym derivatives to handle non-uniform measures.
  • Filter coefficients are obtained as projection coefficients between polynomial and multiwavelet subspaces.The derivation uses inner products and orthonormality relations for the relevant bases.
  • The filter matrices admit an inverse form, enabling reconstruction from lower-resolution to higher-resolution coefficients.The reconstruction equations follow by inverting the scale-transformation relations.
  • Gaussian quadrature computes the required polynomial inner products efficiently and exactly under the stated degree conditions.A k-point quadrature is sufficient for the specified products of degree-restricted basis functions.
  • For k > 20, floating-point errors make computed filter matrices diverge from the mathematical constraint, while k in [1,6] is useful in the experiments.The limitation is attributed to numerical precision rather than the underlying mathematics.

D Additional Results

The additional-results section evaluates the proposed multiwavelet models on Navier-Stokes and examines prediction at finer resolutions using models trained on lower-resolution data.

  • The additional evaluation includes a Navier-Stokes dataset.
  • The section also evaluates prediction at finer resolutions from lower-resolution trained models.
  • Additional experiments assess multiwavelets on pseudo-differential equations.

D.1 Navier-Stokes Equation

The Navier-Stokes experiments learn time-dependent vorticity mappings across several viscosities and dataset sizes. The proposed MWT Leg model outperforms existing neural operators and other deep-learning benchmarks, while the chosen architecture leaves room for recurrent variants.

  • The task maps an initial vorticity history through time 10 to the later vorticity trajectory.Experiments use two-dimensional incompressible Navier-Stokes flow on the unit torus.
  • The experiments vary viscosity, final time, and training-pair count across four configurations.Data are generated on a 256 × 256 grid and subsampled to 64 × 64.
  • MWT Leg outperforms existing neural operators and other deep neural-network benchmarks in Table 3.The table validates models at various viscosities and separates the proposed methods from prior methods.
  • The Navier-Stokes MWT models use a two-dimensional multiwavelet transform with k = 3 and three-dimensional convolutions for time-correlated kernels.
  • A time-recurrent MWT variant could improve L2 error in lower-data settings, but the experiments use only the three-dimensional-convolution version.

D.2 Prediction at higher resolutions

The MWT Leg model is designed to extend predictions beyond the resolution used during training. Experiments on Burgers’ equation test this resolution-extension capability.

  • D.2 Prediction at higher resolutions: The MWT Leg model trained at lower resolutions can predict outputs at higher resolutions.The experiment evaluates whether the model generalizes beyond its training discretization.
  • D.2 Prediction at higher resolutions: 2.26% relative L2 error is achieved when training at resolution s = 256 and predicting at resolution s = 2048.This corresponds to a 10X increase in resolution.
  • D.2 Prediction at higher resolutions: 4.56% relative L2 error is achieved when training at s = 128 and predicting at s = 8192.The prediction is made at 26 times the training resolution.
  • D.2 Prediction at higher resolutions: Figure 8 presents the resolution-extension pipeline, down-sampled training functions, and two test input/output examples.The examples distinguish the test samples by blue and red colors.

D.3 Pseudo-Differential Equation

The pseudo-differential-equation experiment studies learning the map from an external function to the PDE solution. It examines how the number of operator basis functions affects multiwavelet kernel annihilation.

  • D.3 Pseudo-Differential Equation: The experiment modifies the Euler–Bernoulli beam setup to define a third-order pseudo-differential PDE.The equation is used as a canonical PDE for simulation, without a stated physical interpretation like Euler–Bernoulli.
  • D.3 Pseudo-Differential Equation: Figure 11 shows two input functions in red and blue alongside their corresponding outputs in matching colors.The figure provides sample input/output pairs for the PDE.
  • D.3 Pseudo-Differential Equation: The task is to learn the map from the external function f to the solution u.A sample force function and corresponding solution are shown in Figure 11.
  • D.3 Pseudo-Differential Equation: For k ≥ 2, MWT Leg models show similar relative error across epochs.This behavior agrees with the stated property that k > T − 1 suffices to annihilate the kernel away from the diagonal when T = 3.
  • D.3 Pseudo-Differential Equation: Figure 9 compares relative L2 error across epochs for MWT Leg models using different numbers of operator basis functions.The comparison focuses on the effect of k in the pseudo-differential-equation experiment.

D.4 Korteweg-de Vries (KdV) Equation

Additional KdV experiments examine squared-exponential-kernel inputs and robustness when training and test samples use different random sampling schemes.

  • D.4 Korteweg-de Vries (KdV) Equation: The KdV experiments evaluate inputs sampled from a squared exponential kernel.The corresponding benchmark results are reported for different input resolutions.
  • D.4 Korteweg-de Vries (KdV) Equation: The experiments also test neural operators when training and test samples come from different random sampling schemes.This evaluates behavior under a change in the sampling procedure between training and testing.
  • D.4 Korteweg-de Vries (KdV) Equation: Table 5 reports KdV benchmarks across input resolutions, with proposed methods listed above previous neural-operator methods.The table separates the authors’ methods from prior neural-operator results.

D.4.1 Squared Exponential Kernel

The squared-exponential-kernel experiment samples KdV inputs from a periodic Gaussian-process kernel and compares MWT models with existing neural operators across resolutions.

  • D.4.1 Squared Exponential Kernel: KdV input functions are sampled from a periodic squared exponential kernel because of the periodic boundary conditions.The kernel is used to generate random input functions for the experiment.
  • D.4.1 Squared Exponential Kernel: The kernel uses domain length P = 1 and smoothing parameter L = 0.5 to avoid sharp peaks in sampled functions.The random input function is sampled from N(0, K_m), where K_m is the kernel matrix.
  • D.4.1 Squared Exponential Kernel: MWT models perform better than existing neural operators at all tested resolutions.The comparison is reported in the KdV benchmark table.

D.4.2 Training/Evaluation with different sampling rules

The experiments typically use the same sampling rule for training and testing, but this is not a general function-space evaluation setting. Performance varies with input fluctuation strength, which may matter more than the generating rule.

  • Train and test samples in the neural-operator experiments are generated by sampling input functions with the same rule.This setting produces similar smoothness, periodicity, and peak patterns across training and evaluation data.
  • The same-rule setting is useful when future evaluation functions have patterns similar to those seen during training.
  • This evaluation protocol is not general for learning operators between function spaces.
  • Changing input fluctuation strength affects neural-operator performance, including when training and testing distributions differ.The additional experiment trains on periodic squared-exponential samples and tests on random-field samples with controllable fluctuation parameter λ.

D.5 Burgers Equation

This section provides the Burgers’ equation numerical results and organizes validation across input resolutions, alongside a comparison with previous neural-operator methods.

  • Burgers’ equation numerical values from Figure 6 are reported in Table 7.
  • Table 6 describes neural-operator performance when training on squared-exponential random inputs and testing on smooth random functions controlled by λ.In the test data, λ is inversely proportional to fluctuation sharpness for KdV inputs.
  • Table 7 evaluates Burgers’ equation validation at various input resolutions s.
  • The table places the proposed methods in the top section and previous neural-operator works in the bottom section.
Loading 2109.13459v2…