Source-linked AI summary
Euclidean Fourier Neural Operators
Nathanael Bosch, Niklas Frederik Schmitz, Michael F. Herbst
TL;DR
FNOs are grid-resolution independent but their mode-indexed weights change physical meaning across periodic domains, creating a domain-transfer gap. The paper introduces EFNOs, which parameterize the spectral kernel continuously in physical wavevectors, and demonstrates consistent transfer across domains in heat-equation and materials-science tasks.
Problem
FNO weights are tied to domain-dependent physical wavevectors, so applying a trained model to another domain changes the represented operator and limits cross-domain transfer.
Method
EFNOs parameterize the spectral kernel as a continuous function of physical wavevectors rather than as a lookup table over Fourier mode numbers.
Results
EFNOs generalize to unseen grid sizes and domains across heat-equation and exchange-correlation-potential tasks, while heat-equation transfer achieves EFNO relative L2 error below 10^-5%.
Takeaways & Limitations
EFNOs provide operators that can be applied consistently across periodic domains of varying shape and size.
Abstract
from arXiv · showhide
Fourier neural operators (FNOs) provide an efficient framework for learning mappings between function spaces as they are, by construction, independent of the grid resolution at which they are trained and evaluated. However, FNOs are not independent of the periodic domain they are applied to: their discrete spectral weights are indexed by integer Fourier mode numbers, which correspond to physical wavevectors. When applied to a different domain, the same trained weights act at different wavevectors, and the FNO silently represents a different operator. This makes FNOs unsuitable for tasks where transfer across domains is crucial. We propose Euclidean Fourier neural operators~(EFNOs) as a domain-independent alternative to FNOs. By parameterizing the spectral kernel as a continuous function of the physical wavevector, the EFNO can learn operators that act consistently across periodic domains of varying shape and size. We evaluate the EFNO on a simple heat equation and on a practically relevant materials science task of learning exchange-correlation potentials across different crystal structures, and demonstrate that the EFNO is able to generalize to unseen grid sizes and domains.
1 Introduction
FNOs learn grid-resolution-independent mappings but remain dependent on the periodic domain, limiting transfer across domains. EFNOs address this by parameterizing the spectral kernel using physical wavevectors and generalize across unseen grid sizes and domains.
- FNOs are independent of discretization resolution but must transfer across domains of varying shape and size.
- Discrete FNO weights are indexed by Fourier modes that correspond to domain-dependent physical wavevectors, so changing domains changes the represented operator.
- EFNOs parameterize the spectral kernel as a continuous function of physical wavevectors to learn operators applicable consistently across domains.
- Experiments on a heat equation and crystal-structure exchange-correlation potentials show EFNO generalization to unseen grid sizes and domains.
2 Fourier neural operators and their domain dependence
FNOs implement nonlocal operators through Fourier-domain convolution with trainable mode-indexed weights. Because physical wavevectors depend on the periodic lattice, applying the same weights across domains changes the operator.
- Neural operators combine lifting, hidden transformations, nonlocal kernel operators, nonlinearities, and projection to learn mappings between function spaces.
- FNOs represent convolution in Fourier space, where applying the operator becomes point-wise multiplication of Fourier coefficients.
- FNO spectral coefficients are trainable lookup-table weights indexed by Fourier modes up to a cutoff M.
- Although FNOs tolerate changes in grid resolution when retained modes are resolved, their physical wavevectors depend on the domain lattice.
- Applying one trained FNO to different domains assigns the same weights to different wavevectors, causing the applied operator to differ by domain.
3 Euclidean Fourier neural operators
EFNOs replace mode-number lookup tables with continuous physical-wavevector symbols, removing implicit periodic-domain dependence. The approach supports domain transfer while retaining FFT-based computation and allowing physical inductive biases.
- A Euclidean convolution operator is characterized by a symbol that multiplies Fourier coefficients at any physical wavevector.
- EFNOs directly parameterize a continuous symbol so it can be evaluated at whatever wavevectors a domain requires.
- Uniform-grid discretization permits FFT computation, while grid resolution truncates the Fourier series and eliminates the need for a fixed mode cutoff.
- In the domain-transfer test, FNO error grows with supercell size while EFNO error remains constant.
- Continuous-symbol parameterization can encode decay, smoothness, isotropy, and Hermitian symmetry as physical inductive biases.
4 Experiments
The experiments test EFNO and FNO transfer across periodic domains and resolutions using heat-equation and exchange-correlation-potential tasks. EFNO generalizes across unseen domains and grid sizes, while FNO performance degrades when physical domains change.
- Experimental scope: The evaluation covers heat-equation transfer across periodic cells and exchange-correlation-potential learning across crystal structures of varying shape and size.The materials task maps PBE electron densities to RPA exchange-correlation potentials.
- Heat equation: ∼90% relative L2 error is reached by FNOs on larger heat-equation supercells, while EFNO error remains below 10−5%.The FNO is fitted on a small rhombic cell and evaluated on an equivalent rectangular cell and larger supercells.
- Exchange-correlation potentials: The FNO extrapolates poorly from smaller training cells to larger test cells, whereas the EFNO generalizes well with WRMSE errors an order of magnitude below the PBE baseline.Figure 2 reports WRMSE against cell volume for the exchange-correlation-potential tasks.
- Evaluation protocol: The materials experiments train on smaller structures and evaluate on larger ones using density-weighted WRMSE as the reported metric.The datasets include diamond and liquid-water structures with different atom counts and sizes.
5 Conclusion
EFNOs replace FNO lookup tables with continuous functions of physical wavevectors, removing implicit domain dependence. Heat-equation and materials-science experiments demonstrate consistent application across domains of varying shape and size, while broader evaluations remain future work.
- EFNOs parameterize the Fourier symbol continuously in the physical wavevector, enabling consistent operators across domains of varying shape and size.
- The approach is demonstrated on a heat equation and on exchange-correlation-potential learning across crystal structures.
- Future work includes more expressive symbol parameterizations, additional baselines, and complex PDEs from other fields.
A Related work
Related work extends FNOs to irregular geometries or continuous spectral representations, but these approaches generally use mode numbers or additional position dependence. EFNO instead uses the physical wavevector while retaining a position-independent convolution.
- Neural operators on varying geometries: Geo-FNO, DNO, and DAFNO extend FNOs to irregular or varying geometries through domain deformation, reference-domain mapping, or characteristic-function encoding.
- Learned continuous spectral symbols: AM-FNO and SirenFNO replace discrete weight tables with functions of frequency variables, while PDNO and KANO use position-dependent pseudo-differential symbols.
- Learned continuous spectral symbols: Existing continuous spectral approaches use mode numbers as frequency arguments, so their represented operators remain tied to domain-dependent physical scales.
- Parameterization comparison: EFNO alone among the compared parameterizations uses the physical wavevector, the quantity that changes with the domain.
- Applications to electronic structure: Electronic-structure neural operators include fixed-box molecular density prediction and crystal density prediction using combined orbital and plane-wave bases.
- Kernel properties: Table 3 organizes equivalences between real-space kernel properties and corresponding spectral properties.
- Gaussian basis parameterization: The EFNO symbol is parameterized with Gaussian basis functions in |k|^2, producing a smooth, decaying, isotropic, real-valued function of k.
D.1 Experiment 1: Gaussian smoothing across periodic cells
The experiment tests whether operators fitted on one periodic cell transfer to rectangular and enlarged supercells. FNO errors grow on changed cells, whereas EFNO remains at its fitting residual.
- Models and fitting: Both models are single spectral convolution layers, differing only in their parameterization of the convolution symbol.The FNO uses 12 modes per dimension, while the EFNO uses 25 Gaussian-basis coefficients fitted to the exact symbol.
- Evaluation: Errors are measured as relative L2 error against the exact smoothed field.The input is a periodic array of Gaussian sources, exactly representable on every supercell.
- Results: On the fitting cell, both models are near-exact, but on changed cells FNO error becomes large while EFNO error remains at the fitting residual.Table 4 characterizes the FNO as near-exact on the fitting cell but failing to transfer, while EFNO remains at its basis-approximation residual on every cell.
D.2.1 Architecture and training
The materials-science models learn mappings from PBE electron densities to RPA exchange-correlation potentials across structures of different sizes. Training uses gauge-invariant density-weighted error and a common architecture with different spectral parameterizations.
- Dataset and splits: The dataset contains diamond and liquid-water structures, with training and validation on smaller structures and testing on larger structures.Diamond uses 20 eight-atom training structures and 20 sixteen-atom test structures; water uses 32 eight-molecule training structures and 39 larger test structures.
- Dataset and splits: Training and validation structures are Fourier-resampled to common grids, while test structures retain their original grids.The common grids are 48^3 for diamond and 80^3 for water, enabling batching while preserving grid variation at evaluation.
- Loss and evaluation: The loss is a gauge-invariant density-weighted mean squared error, reported as WRMSE in meV/e−.Subtracting the density-weighted mean removes the constant gauge freedom of the exchange-correlation potential.
- Model and training: Each model uses lifting, two spectral blocks, and projection, with 8 channels; EFNO blocks use 16 basis functions and the FNO baseline uses 4 modes.Both models are trained with Adam at learning rate 10−2, using the checkpoint with the lowest validation WRMSE.
D.2.2 Visualizations of the predicted potentials
The visualizations compare PBE, FNO, and EFNO predictions with RPA reference potentials for representative training and test structures. Both models fit training structures, but EFNO remains close to the reference on tests while FNO errors grow.
- Visualization setup: Figures 3 and 4 show RPA potential predictions for representative diamond and liquid-water training and test structures.PBE electron density is used as input, with prediction differences shown relative to the RPA reference.
- Training structures: On training structures, both FNO and EFNO reproduce the RPA potential with small errors.This behavior is shown for both the diamond and liquid-water examples.
- Test structures: On test structures, FNO errors grow substantially, whereas EFNO predictions stay close to the RPA reference.For the diamond example, FNO errors are larger than the PBE baseline, while EFNO remains close to the reference.