PaperlayerView exact revision on arXiv

Source-linked AI summary

Variationally correct loss functions for nonlinear parametric problems and their neural approximations

Pablo Cortés Castillo, Wolfgang Dahmen, Jay Gopalakrishnan

arXiv:2610.19296v1math.NAcs.NA

TL;DR

Approximating parameter-to-solution maps for nonlinear parametric PDEs can facilitate inference on system states, but requires losses that assess approximation error. The paper develops residual-based, variationally correct losses and shows that their certification properties differ between two computational approaches.

  • Problem

    Approximating parameter-to-solution maps can facilitate inference on physical system states, motivating the need to assess approximation error for nonlinear parametric problems.

    Source: §1.1 · p. 1, §2.1 · p. 8

  • Method

    An abstract framework measures residuals in computable dual norms, yielding NFOSLS with an L2-type test space and NDPG with element-local test-space problems.

    Source: §1.3 · p. 4, §1.4 · p. 5

  • Results

    NFOSLS certifies arbitrary H(div) × ˚ H1-conforming neural network outputs without mesh restrictions, while the NDPG bound uses a weaker ultraweak norm and requires a mesh, lowest-order trial functions, and piecewise constant parameters.

    Source: §3.6 · p. 29

  • Takeaways & Limitations

    The error estimator and neural-network training loss are the same variationally correct functional, so it can both train models and certify their predictions.

    Source: §1 · p. 1

    The residual-error ratio check is only a plausibility check, not a bound, because Csat is unknown and the check covers only bQsub.

    Source: §5.4 · p. 43

Abstract

from arXiv · show

We construct computable a posteriori error estimators for parametric families of nonlinear elliptic problems by measuring the residual of a variational formulation in the dual norm of a closed subspace of the test space. A single abstract theorem shows that the resulting estimator is bounded above and below by the error in the trial norm, i.e., it is a variationally correct loss function. Two instances are covered. The first considers an $L_2$-like test space, yielding a nonlinear first-order system least-squares loss function that certifies arbitrary, possibly meshless, approximations. The second breaks the test space, yielding a nonlinear discontinuous Petrov-Galerkin loss function computed by element-local problems. All assumptions are verified for quasilinear diffusion, including monotone fluxes and semilinear problems, and a nonlocal Kirchhoff model. Numerical studies train neural networks approximating the parameter-to-solution maps with the latter loss and measure the ratio of loss to squared error.

1. Introduction

The paper develops computable residual-based error estimators that also serve as variationally correct losses for neural network approximations of nonlinear parametric PDE solution maps. It motivates this approach, defines the required error equivalence, contrasts existing losses, and presents a general theorem with two computational instances and applications.

1. Introduction

  • The paper’s a posteriori error estimators and neural-network losses are the same dual-norm residual construction, used for adaptive refinement or training and error certification.The framework also applies directly to adaptive finite element methods for nonlinear elliptic problems.Source: §1 · p. 1, §1.4 · p. 5

1.1. Learning parameter-to-solution maps.

  • Residual-based operator learning avoids high-fidelity solves for many parameter samples, but non-convex training leaves the loss as the only available accuracy assessment at termination.The target is the parameter-to-solution map, which is useful for inference on uncertain physical systems.Source: §1.1 · pp. 1–2

1.2. Variational correctness.

  • A variationally correct loss is uniformly equivalent to squared trial-norm error at every hypothesis, with constants independent of the parameter and discretization.For nonlinear problems, stability may be only local, while exact dual-norm evaluation is generally intractable.Source: §1.2 · p. 2

1.3. Which losses are variationally correct?

  • Classical strong and localized residual losses need not certify arbitrary trial functions; adding the discrete Galerkin violation restores equivalence but requires a global solve per sample.The energy gap is variationally correct but its unknown offset prevents its raw value from serving as a computable error certificate.Source: §1.3 · pp. 3–4
  • NFOSLS measures a first-order-system residual in L2, while NDPG breaks the test space so its dual norm reduces to element-local problems.NFOSLS controls an H(div) × H1 graph error and needs a conforming flux; NDPG controls lower-regularity interior-variable error at the cost of local solves.Source: §1.3 · pp. 4–5

1.4. Results of this paper.

  • A single abstract theorem gives a two-sided error bound for computable residual surrogates under differentiability, derivative-bound, nonlinear inf-sup, and Fortin conditions.NFOSLS supports arbitrary, including meshless, trial classes; NDPG uses hybrid trial spaces with interface traces for local computability.Source: §1.4 · p. 5
  • The assumptions are verified for quasilinear diffusion, including monotone-flux and semilinear cases, and for a nonlocal Kirchhoff model.The framework also yields computable two-sided error control for adaptive finite element refinement.Source: §1.4 · p. 5

1.5. Relation to other work.

  • The paper’s bounds apply to the nonlinear functional itself, unlike cited analyses that establish equivalence only for linearized objectives or near-solution approximations.For the semilinear example, the two-sided bound comes with a smallness condition relating lower-order nonlinearity Lipschitz constants to the diffusion coefficient.Source: §1.5 · pp. 6–7

2. Nonlinear parametric PDEs and residual loss functions

This section develops computable residual losses for nonlinear parametric problems by measuring residuals in dual norms of closed test-space subspaces, with a theory connecting the losses to trial-space error. It covers continuous least-squares and broken-test-space DPG constructions, including an element-local computation of the DPG dual norm.

2.1. Problem setting and computable residuals.

  • The target is the parameter-to-solution map across a potentially high- or infinite-dimensional parameter set, while this section analyzes individual fiber problems.The parameter may encode physical properties such as material coefficients; a parameter-independent problem is included by taking the parameter set to be a singleton.Source: §2.1 · p. 8
  • Measuring residuals in a closed test-space subspace unifies two computable losses: NFOSLS uses an L2-type test space, while NDPG uses broken tests for element-local dual-norm computations.NFOSLS moves derivatives onto trial functions through a first-order system reformulation, requiring a stronger trial norm and extra trial variables; NDPG retains the weaker trial norm and decouples computations by element.Source: §2.1 · p. 8, §2.2 · p. 9

2.2. The residual loss function and its variational correctness.

  • Under local nonlinear inf-sup and Fortin assumptions, the residual loss has local reliability and efficiency bounds, yielding variational correctness when data oscillation vanishes and Fortin norms remain bounded.The bounds hold on a local admissible trial set; globally, the result applies throughout the trial class when the nonlinear inf-sup condition holds globally.Source: §2.2 · pp. 10–11

2.3. The least-squares instance.

  • With a global nonlinear inf-sup condition, the NFOSLS loss gives two-sided error bounds for any trial class, including meshless neural-network hypotheses.The continuous residual is directly evaluable when the test space is L2-type and identified with its dual; first-order system reformulations provide the stated NFOSLS construction.Source: §2.3 · p. 14

2.4. The DPG instance.

  • The DPG construction enlarges the trial space with interface unknowns so the test space can be broken and its Riesz map becomes element-local.The supplied passage describes a conforming simplicial mesh from a uniformly shape-regular mesh hierarchy.Source: §2.1 · p. 8, §2.4 · p. 15
  • Although the broken test space makes the trial space mesh-dependent through skeleton components, the estimates remain mesh-independent.The broken space has no interelement continuity across element interfaces.Source: §2.4 · p. 15
  • The squared dual norm decouples into independent element-by-element computations, each solving a small symmetric positive definite system; the resulting loss is called the NDPG loss function.Each system uses the Gram matrix of the element test-space inner product.Source: §2.4 · p. 15
  • The broken test space must be large enough for its residual norm to approximate R(α, wh), with the two factors ∥Π∥L(Vα) and osc(ℓ) supplied by Assumption 2.3 through Theorem 2.4.Concrete realizations are given in §3.3 and §4.3.Source: §2.4 · p. 15

3. Quasilinear diffusion with nonlinear fluxes and lower-order terms

This section develops quasilinear diffusion applications with nonlinear fluxes and lower-order reactions, introducing NFOSLS and broken ultraweak NDPG formulations. It establishes variational correctness and two-sided a posteriori error bounds under stated hypotheses for nonlinear-flux and semilinear models.

3. Quasilinear diffusion with nonlinear fluxes and lower-order terms

  • The model covers nonlinear leading-order fluxes without lower-order terms and semilinear problems with a linear leading operator and nonlinear additive lower-order terms.The flux may depend nonlinearly on both the solution gradient and its value.Source: §3 · p. 15

3.1. A model class of quasilinear problems.

  • Under Hypothesis 3.1, the inverse flux is jointly C1 and the weak quasilinear boundary-value problem has a unique solution.The hypothesis includes uniform monotonicity and Lipschitz conditions on the flux, plus a smallness condition on lower-order nonlinearities.Source: §3.1 · pp. 16–17

3.2. The NFOSLS loss function.

  • The NFOSLS loss is globally variationally correct on all Uα = H(div) × ˚H1 for any α and k ≥ 0, and requires only quadrature to evaluate.It uses σα rather than its inverse ρα; the trial space may also include neural networks with outputs in H(div) × ˚H1.Source: §3.2 · pp. 19–21

3.3. The NDPG loss function.

  • The NDPG formulation integrates the first-order equations by parts elementwise and introduces boundary traces as unknowns in broken test and trial spaces.Its forms require the inverse flux ρα; the discrete setup uses piecewise polynomial spaces and trace subspaces.Source: §3 · p. 16, §3.3 · pp. 21–22
  • Under Hypothesis 3.1 and condition (79), the NDPG loss is globally variationally correct for every mesh and every discrete trial function.When f is piecewise constant, the oscillation term vanishes, yielding bounds with mesh- and parameter-independent constants.Source: §3.3 · p. 25

3.4. Examples of parametric nonlinear diffusive fluxes.

  • Under profile conditions (81), both NFOSLS and NDPG losses are variationally correct, with NDPG requiring piecewise-constant α and f.The p-Laplacian for p ≠ 2 is not covered by these profile conditions; moreover, Aα is not Fréchet differentiable unless ρα is affine.Source: §3.4 · pp. 26–28

3.5. Semilinear parametric examples.

  • For the semilinear model, NFOSLS is variationally correct under Corollary 3.14’s coefficient and nonlinearity assumptions, while NDPG additionally requires κα and f to be elementwise constant.Source: §3.5 · p. 28

3.6. Discussion.

  • NFOSLS certifies arbitrary conforming neural-network outputs without restrictions on α or a mesh, whereas NDPG uses a weaker ultraweak norm but requires a mesh, lowest-order trials, and piecewise-constant parameters.The discussion notes condition (79) is easy to check for the nonlinear-flux and semilinear examples.Source: §3.6 · p. 29
  • Unlike [29], the method introduces no auxiliary L2 unknowns, keeps the nonlinearity inside Aα, and uses the trial space of the corresponding linear problem.The scheme in [29] adds unknowns for gα(u) and γα(u), making the remaining equations linear and adding squared L2 mismatches as least-squares penalties.Source: §3.6 · p. 29
  • Theorems 3.6 and 3.9 provide computable two-sided error estimators for arbitrary trial-class members, unlike [29]’s C´ea-type estimate, which requires an exact solution and an Aubin–Nitsche-type closeness hypothesis.The paper notes that certifying predictions beyond the exact minimizer matters for neural-network applications; [29] proves no two-sided residual-to-error bound.Source: §3.6 · p. 29
  • The approach requires the smallness condition (89), which [29] does not, and uses an ultraweak rather than [29]’s primal DPG formulation.Source: §3.6 · p. 29

4. The stationary Kirchhoff model

This section establishes globally reliable and efficient FOSLS and primal DPG error estimators for the stationary Kirchhoff model, whose nonlocal diffusion coefficient depends on the solution’s global Dirichlet energy. It also shows that the Kirchhoff NDPG loss is globally variationally correct under a piecewise-constant coefficient condition, with estimates uniform in the parameter and mesh, for every polynomial degree and without restriction on the nonlinearity.

4.1. The parametric Kirchhoff boundary value problem.

  • Each Kirchhoff solution is a rescaling of a linear diffusion solution, with the scale determined by a single scalar equation.The exact flux equals the flux of the linear problem, and the scalar root is bounded independently of the parameter.Source: §4.1 · p. 31

4.2. The NFOSLS loss function.

  • The NFOSLS loss is globally variationally correct on the entire trial space, which may contain finite-element or meshless approximations.Its trial class enters neither the loss nor its analysis.Source: §4.2 · pp. 32–33

4.3. The NDPG loss function.

  • The primal DPG loss is computed through element-local small linear symmetric positive definite systems, with nonlinearity entering through one scalar.For piecewise-constant diffusion coefficients, the global energy is computed in one mesh pass using quadrature exact for degree 2k.Source: §4.3 · p. 34
  • The Kirchhoff DPG formulation satisfies the estimator assumptions globally, with the stability estimate holding for all trial functions.The verification gives δ = +∞.Source: §4.3 · pp. 34, 36
  • The alternate (b′)–(c′) route gives only local variational correctness, whereas Proposition 4.7(c) yields the stronger global result.Its neighborhood radius is chosen so the linearization error absorbs half the inf-sup constant.Source: §4.3 · p. 36
  • The Kirchhoff NDPG loss is globally variationally correct for every mesh and trial function when κα is piecewise constant, with constants independent of α and the mesh.If f belongs to P_max(k−1, 0), then osc(ℓ) = 0.Source: §4.3 · p. 36

4.4. Discussion.

  • Unlike the quasilinear NDPG result, the Kirchhoff NDPG loss requires no restriction on the nonlinearity and applies at every polynomial degree.Both NDPG results require a piecewise-constant coefficient; the Kirchhoff Fortin property needs no adaptation to Mα because it enters the form only through a scalar multiplier.Source: §4.4 · p. 37

5. Application to neural approximations

The section formulates neural operator learning as minimizing a computable loss over parameterized trial functions. Its experiments use finite-dimensional, cellwise parameters and compare network predictions with finite-element solutions and a high-fidelity solution.

5. Application to neural approximations

  • Experiments compare network predictions with finite-element solutions in the predicted space and with a high-fidelity solution in a richer space, and assess loss relative to squared error.The loss-to-squared-error ratio tests the two-sided bounds from variational correctness.Source: §5 · p. 37

5.1. The learning problem.

  • For both models, global variational correctness with α- and mesh-independent constants and zero oscillation yields an error bound applicable to any trained network output.The loss is used in place of the uncomputable error.Source: §5.1 · p. 38
  • Training minimizes an empirical loss over i.i.d. parameter samples without solving the PDE, but the error bound does not control the gap between training and population losses.An independent test sample estimates population loss with root mean squared sampling error of order ntest^-1/2 when the trained network has finite loss variance.Source: §5.1 · p. 38
  • Parameters are represented by cell values on uniform n × n tilings for n = 2, . . . , 5; adjacent-cell jumps make the model a nonlinear interface problem.Every mesh used resolves the tiling, as required by the cited theorems.Source: §5.1 · p. 38
  • At one sample parameter, the trained DPG network prediction is visually indistinguishable from the nonlinear DPG solution in the same trial space.Source: §5.1 · p. 38
  • The hypothesis class combines a finite-element trial space with a neural network that predicts coefficients in a fixed basis.Source: §5.1 · p. 38

NDPG-NN NDPG-FEM

The section evaluates NDPG neural networks against finite element minimizers and tests how network depth, parameter dimension, training-set size, and mesh refinement affect their loss. Across these experiments, the network often approaches the loss attainable in its trial space, while the loss–error relation remains stable across tested parameter dimensions and contrasts.

NDPG-NN NDPG-FEM

  • The NDPG neural prediction and nonlinear DPG minimizer are visually indistinguishable in the illustrated traces for the selected coefficient.The example uses a 5 × 5 tiling, contrast κ = 20, 4096 training samples, and 20,000 epochs.Source: Figure 1 · p. 39

5.2. General settings.

  • Experiments compare a quasilinear flux law with the saturated Carrier Kirchhoff model using low-rank residual networks; high-fidelity references are computed only for evaluation, and both models share test parameters for each tiling and contrast.All networks use N = 128 and r = 32 and Adam with learning rate 10^-4 and batch size 32; unless stated otherwise, they train for 10,000 epochs and are evaluated on an independent 1,024-parameter test set.Source: NDPG-NN NDPG-FEM · p. 39, §5.2 · pp. 40–41

5.3. Network complexity.

  • Deeper networks reach a given cumulative minimum loss in fewer epochs, and the study fixes depth l = 5 thereafter.Depth was varied at fixed width and rank in the 5 × 5 tiling, contrast κ = 30, with Figure 2 showing the epoch curves.Source: §5.3 · p. 41, Figure 2 · p. 41

5.4. Residual-error ratios.

  • As contrast increases, the network loss exceeds the finite element loss by at most 30%, while the loss–error ratio grows by less than twofold for the flux model and about tenfold for Kirchhoff.At κ = 1, the losses agree in both models; the ratios are measured over the 2 × 2 tiling.Source: Table 1 · p. 42, §5.4 · p. 43

5.5. Dimension of the parameter domain.

  • As the tiling grows from 2 × 2 to 5 × 5, the loss–error relation remains insensitive to parameter-domain dimension, while the network-to-FEM loss ratio stays below 1.4.The parameter dimension increases from 4 to 25; the network loss rises by 27% in the flux model and 50% in Kirchhoff.Source: §5.5 · p. 43, Table 2 · p. 43

5.6. Balancing the training set and the trial space.

  • As the training set grows, the network-to-FEM loss ratio approaches 1 in both models; smaller sets leave a deficit that persists with further training.The finite element loss is independent of training, while the network approaches a positive expected loss for any finitely parametrized approximation.Source: §5.6 · p. 44, Table 3 · p. 44
  • With 4096 training samples, both losses decay like h^2, and the network exceeds the finite element loss by at most about 2% on the finest flux-model mesh.The network keeps pace despite a roughly sixfold drop in weights per degree of freedom; persistence under further refinement was not tested.Source: §5.6 · p. 44, Table 4 · p. 44
  • On finer meshes, the cumulative-minimum training curves reach their trial-space FEM loss levels later.Figure 3 compares training-set sizes at fixed mesh size and mesh sizes at fixed training-set size.Source: §5.6 · p. 45, Figure 3 · p. 45
Loading 2610.19296v1…