Source-linked AI summary
Neural Network Preconditioning: A Case Study for the Solution of the Parametric Helmholtz Equation
Luc Giraud, Carola Kruse, Paul Mycek, Maksym Shpakovych, Yanfei Xiang
TL;DR
For 2D parametric Helmholtz systems, the paper investigates preconditioning where no generic efficient option exists. It trains a neural network without reference solutions to approximate the operator inverse and uses it within FGMRES, reporting improved convergence and robustness beyond training conditions.
Problem
No generic, efficient preconditioner currently exists for the linear systems arising from the targeted 2D parametric Helmholtz discretization.
Source: §1 · p. 1Method
A self-supervised neural network approximates the system-matrix inverse using a residual-based loss, then acts as a nonlinear preconditioner within FGMRES.
Source: §2.2 · p. 3, §3.2 · p. 6Results
The neural preconditioner significantly accelerates FGMRES convergence and remains effective for problems beyond the training distribution, while the network alone lacks accuracy and robustness as a solver.
Source: §1 · p. 2, §4.1.2 · p. 9, §6 · p. 15Takeaways & Limitations
The hybrid approach combines neural-network generalization and speed with the numerical reliability of classical iterative methods.
Source: §6 · p. 15The study is limited to 2D cases and reports no computational times, with no effort made to optimize the implementation.
Source: §4.2.3 · p. 14
Abstract
from arXiv · showhide
This work presents a hybrid numerical approach for solving linear systems arising from the discretization of the two-dimensional parametric Helmholtz equation. A convolutional neural network based on the U-Net architecture is trained in an self-supervised manner (i.e., without explicitly generating reference solutions) to approximate the inverse of the discretized Helmholtz operator, using a loss function involving the residual norm of the linear system. The trained network is used as a nonlinear preconditioner within the flexible GMRES (FGMRES) algorithm. Numerical experiments show that while the neural network is not accurate enough to act as a standalone solver, it significantly improves the convergence of FGMRES when employed as a preconditioner. The neural preconditioner demonstrates robust performance and generalization capabilities with respect to variations in the velocity field and the domain size. Comparisons with algebraic preconditioners, based on LU factorizations of approximations of the original system matrices, indicate the superior efficiency of the neural approach under equivalent conditions. We believe that the proposed method is not tied to a specific neural architecture and can be extended to other parametric PDEs.
1 Introduction
The paper investigates neural networks as nonlinear preconditioners for parametric Helmholtz systems, where no generic efficient preconditioner currently exists. It trains a network without reference solutions and uses FGMRES to combine its approximate inverse with a classical solver.
1 Introduction
- No generic, efficient preconditioner currently exists for the targeted 2D parametric Helmholtz systems, motivating investigation of a machine-learning preconditioner.Unlike PINNs, which aim to find solutions directly, this hybrid approach uses a subspace method to improve on the network's predicted solution.Source: §1 · p. 1
- The network is trained without explicit reference solutions, using a loss based on the scaled residual norm for a class of parameter values.The network approximates the system-matrix inverse and is used within FGMRES.Source: §1 · pp. 1–2
- FGMRES enables 64-bit computation for ill-conditioned problems while accommodating a nonlinear network typically trained in 32-bit arithmetic.The authors cite prior flexible-preconditioning work [30, Chapter 5] and.Source: §1 · p. 1
- The trained network serves as a nonlinear FGMRES preconditioner and is reported to generalize robustly across larger domains and parameter values or structures outside the training scope.The paper presents the approach as applicable beyond its chosen network architecture and suggests extending it to other linear systems.Source: §1 · p. 2
- Related work includes neural operators, data-driven preconditioners, GNN-based methods, hybrid DeepONet strategies, and multigrid or residual-network approaches.The authors note that their references are not an exhaustive state-of-the-art overview.Source: §1 · p. 2
2 Background
The background formulates the two-dimensional Helmholtz problem and its discretization, then describes how flexible GMRES accommodates nonlinear preconditioning. The PML approximately enforces boundary conditions on the truncated computational domain.
2.1 The Helmholtz equation
- The unbounded Helmholtz domain is truncated to a square grid, Fourier differentiation discretizes the derivatives, and PMLs approximately enforce boundary conditions in the outer region.With N points in each direction, the discretization has n = N^2 degrees of freedom.Source: §2.1 · p. 2
- The discretized problem is a linear system A(c)x = b, with the velocity field c determining the operator and b and x representing the source and solution.The Helmholtz equation is subject to the Sommerfeld radiation condition.Source: §2.1 · p. 2
2.2 Subspace solver with nonlinear preconditioner
- GMRES builds a Krylov subspace and minimizes the residual norm; right preconditioning retains the residual of the original system.Iterative methods offer moderate memory use and allow stopping when solution quality matches uncertainty in matrix entries or right-hand sides.Source: §2.2 · p. 3
- Flexible GMRES permits a different preconditioner at each iteration, extending the Arnoldi relation to nonlinear operators such as the trained network.The network's nonlinear response and 32-bit application within otherwise 64-bit computation are both sources of nonlinearity.Source: §2.2 · p. 3
3 Neural network components
The U-Net maps discretized source and velocity fields to a predicted Helmholtz solution, and is trained without explicit solution labels using a residual-based loss. Training uses synthetic inputs for a specified discretization and velocity range.
3.1 Network architecture
- The U-Net combines source and velocity fields as a 3 × N × N input tensor and outputs a 2 × N × N tensor representing a predicted complex solution field.The source’s real and imaginary parts form two channels; the real-valued velocity field forms one.Source: §3.1 · p. 4
- The convolutional U-Net has 831,650 trainable parameters, independent of the discretized system size n.Its downscaling and upscaling convolutions halve and double the image size, respectively.Source: §3.1 · p. 4
3.2 Training of the network
- The network is trained self-supervised: its loss uses the linear-system residual rather than comparisons with explicitly generated solution labels.The residual also represents a discretized PDE residual, including boundary conditions.Source: §3.2 · p. 6
- Training the network to reduce the residual makes A(c)Nθ(c) approximate the identity operator, an alternative interpretation of its loss.The authors describe this as similar to an autoencoder, with A(c) acting as the decoder.Source: §3.2 · p. 6
- Training and validation use N = 64, L = 20, and ℓ = 4, with source entries drawn from N(0, 1) and velocity entries from U([1]).For this velocity range, the stated wavelength interval is [2π, 4π], corresponding to 10–20 discretization points per wavelength.Source: §3.2 · p. 7
4 Numerical experiments
The experiments show that the network’s direct predictions are too inaccurate for standalone solution, but as an FGMRES preconditioner it converges quickly and compares favorably with algebraic alternatives. Tests also examine transfer across velocity fields, source structure, and domain sizes.
4 Numerical experiments
- Two networks with similar validation loss were evaluated first as standalone solvers and then as nonlinear FGMRES preconditioners.Both validation losses reached about 0.004 after a similar number of gradient steps.Source: §4 · p. 7, Figure 4 · p. 8
4.1 Numerical behavior of the trained network as a standalone solver
- The experiments test the trained networks as standalone solvers on training-like systems and on changed velocity distributions and larger grids.Solution quality is evaluated using normwise backward error.Source: §4 · p. 7, §4.1 · p. 8
4.1.1 Generalization capabilities with respect to the parameter c
- For standalone predictions, backward error ranges from 0.1 to 0.8 and worsens as the velocity range moves farther beyond training; the two networks show similar trends.The best performance occurs for the training distribution, c iid ∼U([1]).Source: §4.1.1 · pp. 8–9
4.1.2 Generalization capabilities with respect to the domain size
- On larger grids, standalone predictions remain very poor despite a nonsignificant apparent quality improvement; their variance appears to decrease, for which the authors have no explanation.The tested grid sizes correspond to α ∈{2, 4, 8}, with the velocity distribution held fixed.Source: §4.1.2 · p. 9, Figure 6 · p. 9
4.2 Numerical behavior of the trained network as nonlinear preconditioner
- The numerical study uses the trained network as a nonlinear preconditioner to accelerate FGMRES, then tests transfer across velocity parameters and domain sizes.The comparison with algebraic preconditioning is presented before the generalization tests.Source: §4 · p. 7, §4.2 · p. 9
4.2.1 Comparison with an algebraic preconditioner
- NN-FGMRES reached ηb = 10^-10 in fewer than 20 iterations and outperformed every tested algebraic implicit preconditioner; unpreconditioned GMRES did not reach that tolerance in 1,024 iterations.The comparison covered 100 systems with random velocity fields; the best algebraic preconditioner used ϵ = 0.005 and retained nearly 5% of matrix entries.Source: Figure 7 · p. 10, §4.2.1 · p. 11
- The NN preconditioner also outperformed system-dependent CSL at β = 0.25 and average-based CSL even at β = 0; average-based CSL was slower for β > 0.For system-dependent CSL, smaller β accelerated convergence, but even the smallest tested value remained slower than the NN preconditioner.Source: §4.2.1 · p. 11, Figure 8 · p. 12
4.2.2 Generalization capability with respect to the parameter c
- As overlap with training range [1, 2] shrinks, velocity-range transfer degrades and varies more, though it still improves on no preconditioner.For the structurally different square-field/source test, ηb reached 10^-10 in fewer than 30 iterations for (c0, c1) = (1.25, 1.75), and in 85–90 iterations for (1, 2).Source: §4.2.2 · pp. 12–13, Figure 9 · p. 13, Figure 10 · p. 13
5 Further discussion
The study reports U-Net results, while preliminary 1D work suggests other architectures can achieve similar outcomes with different preconditioner efficiency. Architecture selection remains open, and representational capacity matters for FGMRES.
5 Further discussion
- Only U-Net is evaluated here; preliminary 1D results suggest other architectures trained with the same loss can achieve similar outcomes but differ in preconditioner efficiency.Identifying the optimal architecture is outside this paper's scope.Source: §5 · p. 14
- The proposed strategy combines learned preconditioning with classical linear algebra to retain solution quality while benefiting from neural-network acceleration, and is described as applicable in other contexts.Source: §5 · p. 14
- An architecture with insufficient representational capacity may produce a low-rank preconditioner and potentially cause FGMRES breakdown, as illustrated by the DeepONet example.For DeepONet, prediction-space dimension corresponds to the trunk network's embedding size.Source: §5 · p. 14
6 Concluding remarks
Neural networks used as nonlinear preconditioners within FGMRES accelerate convergence on parametric Helmholtz problems, including cases far beyond the training distribution. The authors present the hybrid approach as combining neural-network generalization and speed with classical iterative-method reliability.
6 Concluding remarks
- Integrating trained neural networks as nonlinear preconditioners within FGMRES significantly accelerates convergence even for problems far beyond the training distribution, although standalone accuracy and robustness are limited.Source: §6 · p. 15
- The framework is not tied to U-Net; other architectures may work as preconditioners if they retain sufficient representational capacity.U-Net was chosen for its convolutional structure and efficient use of parameters.Source: §6 · p. 15
- Future work includes discretization-independent PINNs or neural operators trained across resolutions, and extensions to three-dimensional problems and other PDEs.The proposed framework is described as extendable beyond Helmholtz to other classes of linear systems.Source: §6 · p. 15