PaperlayerView exact revision on arXiv

Source-linked AI summary

Optimal Recovery for Solving Variational Problems

Ting Wang, Gideon Simpson, Jaroslaw Knap

arXiv:2609.30049v1math.NA

TL;DR

Variational problems can be difficult to reformulate, and conventional methods can struggle with low regularity, complex physics, or noisy observations. This paper develops a two-step optimal-recovery method in an RKHS to minimize variational energies while incorporating constraints and data. It establishes convergence results and reports numerical evidence, including improved enforcement of a divergence-free constraint with a divergence-free kernel as the collocation grid densifies.

  • Problem

    Directly solving variational problems is motivated when weak or strong reformulations are infeasible for low-regularity solutions or complex physics, and when conventional methods struggle with scattered noisy data.

    Source: §1.3 · p. 2

  • Method

    The paper introduces a two-step optimal-recovery procedure for variational problems that incorporates solution constraints, using an RKHS-based formulation.

    Source: §3 · p. 5, §3.1 · p. 6

  • Results

    With a divergence-free kernel, the divergence-free constraint is well enforced as the collocation grid densifies, while the L2-norm and H1-seminorm of the approximation decrease with more collocation points.

    Source: §6.5 · p. 25

  • Takeaways & Limitations

    The variational formulation uses only first derivatives and avoids the misspecification issue described for the Euler–Lagrange-based kernel method in the semilinear elliptic example.

    Source: §6.3 · p. 21

    For Stokes’ problem, finite collocation constraints on the divergence-free space show non-convergent behavior, which the authors suspect is caused by instability in approximating that space.

    Source: §6.5 · p. 25

Abstract

from arXiv · show

In science and engineering, many physical laws and scientific principles naturally arise as variational problems of minimizing an energy functional over an appropriate functional space. In many cases, it is more advantageous to directly discover the minimizer of the energy functional than to solve the associated Euler-Lagrange equations. Conventional numerical solvers, such as the finite-element method (FEM), often lack the flexibility to incorporate prior knowledge or noisy observational data. In recent years, machine learning approaches, particularly kernel-based methods, have attracted growing attention in scientific computing. Unlike FEM, which requires domain discretization and mesh generation, kernel-based methods construct solutions from scattered nodes, thereby avoiding the complexity of meshing, especially in high-dimensional or geometrically complex domains. This works presents a two-step procedure for variational energy minimization based on the optimal recovery formulation in a reproducing kernel Hilbert space (RKHS), providing a unified framework for seamlessly incorporating both physical constraints and noisy data. On the computational side, we employ the sparse Cholesky decomposition for Matérn kernels to alleviate the well-known cubic complexity bottleneck of kernel methods. On the theoretical side, we rigorously establish the existence and convergence theory of the proposed method using $Γ$-convergence theory. Numerical experiments on benchmark problems demonstrate the efficiency, robustness and accuracy of the approach.

1. Introduction

The introduction motivates direct kernel-based minimization of variational problems where reformulation, meshing, or handling noisy data can be difficult. It presents optimal recovery as a framework for combining physical constraints and observations, with computational and theoretical advances and benchmark evaluation.

1.1. Variational energy minimization

  • Variational energy minimization concerns minimizing an energy functional over admissible functions, with additional physics constraints such as volume preservation or incompressibility.The formulation also allows finitely many noisy observations of the solution.Source: §1.1 · p. 1

1.2. Related research

  • FEM uses domain discretization and meshing, whereas mesh-free methods use scattered points; kernel methods offer an interpretable non-parametric alternative but face high computational costs and ill-conditioned Gram matrices.The introduction contrasts these methods with DNN approaches, which it says may lack interpretability and rigorous convergence guarantees.Source: §1.2 · p. 2

1.1. Variational energy minimization

  • Variational formulations can preserve problem properties when weak or strong reformulations are infeasible, including for low-regularity solutions or complex physics.The introduction also notes that traditional methods such as FEM can struggle to incorporate scattered, noisy experimental data.Source: §1.3 · p. 2

1.3. Main contributions

  • The proposed optimal recovery formulation combines physical constraints with noisy data, while sparse Cholesky for Matérn kernels mitigates kernel methods’ cubic complexity with a controllable efficiency–accuracy trade-off.The paper also uses Γ-convergence to establish existence and convergence results.Source: §1.3 · pp. 2–3
  • The introduction reports fast convergence and robust performance on scalar- and vector-valued variational problems.The paper’s roadmap places computational techniques in Section 4, theoretical foundations in Section 5, and numerical benchmarks in Section 6.Source: §1.3 · p. 3, §1.4 · p. 3

1.4. Organization of the paper

  • The paper organizes its development from kernel methods for PDEs, through variational optimal recovery and sparse Cholesky computation, to theory and numerical benchmarks.The introduction describes the method as a direct approach to the variational problem.Source: §1.3 · p. 2, §1.4 · p. 3

2. Optimal recovery method for solving strong form PDEs

The section formulates optimal recovery in a function-space setting and applies it to linear PDEs through collocation constraints. It relates the recovery solution to Gaussian-process regression, whose uncertainty and probabilistic interpretation also motivate sparse computation.

2.1. The abstract duality framework for optimal recovery

  • With a bijection from the dual space U* to a separable Banach space U, the induced inner product and quadratic norm make U an RKHS when the operator comes from a continuous, symmetric positive-definite kernel.For well-behaved observations in L2(Ω), the kernel action reduces to a Fredholm integral operator.Source: §2.1 · p. 3
  • Optimal recovery seeks the minimum-norm function in U satisfying M observational constraints [ϕi, u] = zi, and the representer theorem gives a unique solution under suitable conditions.The observations are functionals in U*, and point evaluation is one such functional.Source: §2.1 · p. 4

2.2. An application of optimal recovery to linear PDEs

  • For linear PDEs, collocation points are split between the interior and boundary, where bounded linear observation functionals encode the operators and their prescribed data.The setup assumes the true solution lies in the RKHS U.Source: §2.2 · p. 4
  • The PDE representer uses kernel evaluations acted on by the differential and boundary operators, with Gram-matrix blocks for interior and boundary observations.The section denotes action on the second kernel argument using L′ and B′ and notes extension to nonlinear PDEs.Source: §2.2 · pp. 4–5
  • For radial kernels, kernel interpolation is equivalent to RBF collocation, while optimal recovery supplies a function-space framework and a unified treatment of physical constraints and data.The text identifies these features as important to the method developed in the paper.Source: §2.2 · p. 5
  • The optimal recovery mean equals the Gaussian-process posterior mean, and the posterior variance gives prediction uncertainty at a test point.The recovery problem is also equivalent to a maximum a posteriori estimator conditioned on the constraints.Source: §2.2 · p. 5
  • The Gaussian-process perspective provides the probabilistic basis for the sparse Cholesky decomposition used to reduce computational complexity in the variational method.The PDE setup partitions its collocation locations into interior and boundary point sets.Source: §2.2 · pp. 4–5

3. An application of optimal recovery to variational problems

The section develops optimal recovery for variational minimization as a two-step procedure that accommodates constraints. It also discusses when restricting the problem to an RKHS can be ill posed.

3.1. Optimal recovery formulation

  • The method first solves an optimal recovery problem using interior observation functionals and boundary point evaluations, yielding a unique solution by the representer’s theorem.The boundary condition is automatically enforced by the interpolant.Source: §3.1 · p. 6
  • Noisy observations and non-local integral observation functionals can be incorporated into the recovery formulation, extending it beyond interior collocation.The integral functionals are induced by linearly independent test functions.Source: §3.1 · p. 6
  • The second step minimizes an objective IM over latent variables z; Newton’s method is used, with Gauss–Legendre quadrature evaluating the integrals.The quadrature mesh is used only for integral evaluation.Source: §3.1 · pp. 6–7
  • Inverting the Gram matrix is the algorithm’s main computational bottleneck, requiring O(M^3) floating-point operations.The paper points to sparse approximations as a way to facilitate computation.Source: §3.1 · p. 7
  • For a quadratic functional, IM is quadratic and its unique minimizer can be found in one Newton iteration; the formulation also extends to vector-valued solutions.The vector-valued extension uses matrix-valued kernels and is illustrated in numerical examples.Source: §3.1 · pp. 7–8

3.2. A discussion on the kernel misspecification issue

  • Point evaluations may be discontinuous for H1 functions, including when Ω is a subset of R3; suitable non-local observations can address this issue.The paper assumes thereafter that the RKHS embeds continuously in C0 and contains the solution, ensuring point evaluations are defined and avoiding misspecification.Source: §3.2 · p. 8
  • If the true minimizer is outside the specified RKHS, the restricted problem may lack a minimizer even when the RKHS is dense in the underlying Sobolev space.The paper describes a regularized variational problem that is coercive in the RKHS norm; its solution converges to the true solution in a suitable sense.Source: §3.2 · p. 8

4. Sparse approximation

The section uses Matérn-kernel screening and maximin ordering to sparsify Cholesky factors for the Gram matrix and Hessian, reducing computational cost while controlling approximation accuracy.

4. Sparse approximation

  • The sparse Cholesky approach uses Matérn-kernel screening and extends the factorization strategy from the Gram matrix to the Hessian.The factors can be used directly in the solver or as preconditioners for conjugate gradients.Source: §4 · p. 9, §4.2 · p. 12

4.2. Sparse Cholesky decomposition

  • Maximin ordering places points far from previously selected points, and the resulting off-diagonal Cholesky entries decay exponentially with distance, motivating their truncation to zero.The ordering proceeds coarse to fine, allowing nearby observations to screen correlations among finer-scale points.Source: §4.2 · p. 10
  • The Gram-matrix factor shows a clear sparsity–accuracy trade-off, with nearly exact approximation attainable using few nonzeros for moderately large ρ.The comparison uses 61 × 61 equidistant collocation points.Source: Figure 2 · p. 11, §4.2 · p. 11

4.3. Sparse decomposition of the Hessian matrix

  • Under strong convexity of G in (u, ξ), the kernel induced by the Hessian is positive definite on the finite set of interior values z_i.This supports applying sparse Cholesky decomposition to the inverse Hessian.Source: §4.3 · p. 12
  • At similar sparsity, the inverse-Hessian decomposition has larger relative error than the inverse-Gram decomposition, so Vρ is proposed as a preconditioner for conjugate gradients.The Hessian’s screening effect may be diminished because its kernel combines operations on the Matérn kernel.Source: §4.3 · p. 13

5. Theoretical foundation

The theoretical section establishes existence for the finite-dimensional minimization problems and convergence of their minimizers to the variational problem’s minimizer, using Γ-convergence under stated assumptions.

5. Theoretical foundation

  • The functionals FM are designed so their minimizers converge to the unique minimizer of the infinite-dimensional variational problem.The section identifies Γ-convergence as the basis for this convergence result.Source: §5 · p. 14

5.1. Existence of minimizers

  • For each fixed M, the finite-dimensional minimization problem admits a minimizer when the RKHS contains the unique variational minimizer and is continuously embedded in W1,p(Ω).The stated assumptions also require a positive definite kernel.Source: §5.1 · p. 15

5.2. Convergence of minimizers

  • The Γ-convergence result assumes a bounded, connected Lipschitz domain with an interior cone condition, p > d, a sufficiently smooth positive definite kernel, and an RKHS containing the minimizer and embedded in W1,p(Ω).Under these hypotheses, the minimizers converge weakly in W1,p(Ω).Source: §5 · p. 14, §5.2 · p. 16

Assume that the fill distances

The convergence argument establishes strong convergence of the minimizers and uses Γ-convergence with equi-coercivity to connect the finite-dimensional problems to the variational problem.

Convergence of minimizers

  • The minimizers converge strongly to u*.The section’s convergence statement also gives weak convergence in W1,p(Ω), with a particular case noted for p = 2.Source: Assume that the fill distances · p. 16

Assumptions

  • For Matérn kernels with integer smoothness parameter α + d/2, the RKHS-to-W1,p(Ω) continuous embedding required by the result is automatic.The passage describes this embedding as automatic for most popular kernels.Source: Assume that the fill distances · p. 16

Convergence proof

  • The proof strategy establishes Γ-convergence of FM to F and equi-coercivity of the sequence of functionals.It combines a liminf inequality with construction of a recovery sequence.Source: Assume that the fill distances · pp. 17–18

6. Numerical Experiments

The experiments apply the proposed solver to Poisson, semilinear elliptic, elasticity, and constrained Stokes problems, reporting convergence, accuracy, and computational trade-offs.

6.1. Poisson equation

  • For Poisson, sparse-Cholesky preconditioning reduces PCG iterations even at small ρ where CG fails, while Matérn-7/2 converges faster than Matérn-5/2.The fill-distance comparison is reported for the Poisson problem; Table 1 reports approximation errors and PCG iterations.Source: Table 1 · p. 19, §6.1 · p. 19, Figure 4 · p. 20

6.2. Semilinear elliptic problem

  • In semilinear elliptic tests, Matérn-5/2 is roughly one order faster in L2 convergence, while Matérn-3/2 is slightly faster in CPU time.The comparison uses the sparse solver with ρ = 8.Source: §6.2 · p. 20, Figure 6 · p. 20
  • For smooth p-Laplace solutions with p = 6, 8, and 10, approximation error and iteration counts rise with p, but all solutions converge within 30 iterations.The experiment uses a Matérn-7/2 kernel and equidistant collocation points.Source: §6.3 · p. 21, Figure 7 · p. 21
  • For the p = 3, β = 1.1 numerical demonstration, correctly specified Matérn-1/2 captures the solution’s cusp at the origin, whereas smoother Matérn-5/2 oversmooths it; Matérn-1/2 gives a rough boundary approximation.The regularized variational problem uses γ = 10^-6.Source: §6.3 · p. 21

6.4. Vector-valued case: linear elasticity with the equation of state

  • For κ0 = 2 nonquadratic elasticity, global convergence is not guaranteed and an appropriate initial guess is needed; the sparse solver trades accuracy for efficiency.The sparse decomposition is applied separately to the Gram matrix's block-diagonal submatrices.Source: §6.4 · p. 23, Figure 11 · p. 24

6.5. Constrained case: Stokes’ problem

  • With collocation-only incompressibility constraints, div u's L2 error does not converge; a divergence-free kernel enforces the constraint as grids densify while L2 and H1 errors decrease.The collocation approach leaves div u uncontrolled between collocation points.Source: §6.5 · p. 25, Table 2 · p. 25

7. Conclusions

The paper introduces a kernel-based optimal recovery framework for minimizing physical variational functionals while incorporating physical constraints and observation data. It establishes convergence via Γ-convergence and reports that sparse Cholesky techniques reduce computational cost, with experiments demonstrating efficiency and accuracy.

7. Conclusions

  • The framework minimizes physical variational functionals through kernel-based optimal recovery and integrates physical constraints with observation data.The method is variational and requires lower regularity than approaches based on strong-form Euler–Lagrange equations.Source: §7 · p. 25
  • Table 3 reports approximation error for the Stokes problem using a divergence-free kernel.The conclusion describes an extensive set of numerical experiments as demonstrating efficiency and accuracy.Source: Table 3 · p. 26, §7 · p. 26
  • The numerical solution converges to the true solution in an appropriate sense via Γ-convergence, while sparse Cholesky techniques reduce the cubic-growth computational bottleneck of kernel methods.The paper reports that numerical experiments demonstrate efficiency and accuracy.Source: §7 · p. 26

Appendix A. Matrix-valued kernel and optimal recovery

Appendix A reviews matrix-valued kernels and their use in vector-valued interpolation and regression, then describes kernel choices and an optimal recovery formulation. The choices include one that decouples vector problems and another that ensures a divergence-free interpolant.

Appendix A. Matrix-valued kernel and optimal recovery

  • A matrix-valued kernel is a symmetric, positive semi-definite function from pairs of domain points to d × d matrices, and it induces a vector-valued RKHS.The appendix reviews matrix-valued kernels for vector-valued interpolation and regression, also called multi-output learning.Source: Appendix A · p. 26
  • The vector-valued RKHS is constructed by closing a pre-Hilbert space of kernel-based functions, and observations at a finite set of points define the optimal recovery problem over that space.The stated representer theorem gives a solution expressed using kernel evaluations at the collocation points.Source: Appendix A · pp. 26–27
  • The appendix represents kernel evaluations across the collocation points as a block matrix, with each block formed from a matrix-valued kernel evaluated at a point pair.It also gives the evaluation row K(x, x) across the M points.Source: Appendix A · p. 27
  • The appendix presents two kernel constructions: a scalar kernel times the identity that reduces the vector problem to d independent scalar problems, and a radial-kernel construction that ensures a zero-divergence interpolant.The latter uses a scalar-valued radial kernel ψ(r).Source: Appendix A · p. 27

Appendix B. Γ–convergence theory

Appendix B reviews weak convergence, coercivity, and Γ-convergence in reflexive Banach spaces. Its fundamental theorem states that equi-coercive functionals Γ-converging to a functional with a unique minimizer have minimizers converging weakly to that minimizer.

Appendix B. Γ–convergence theory

  • Weak convergence is defined by convergence under every continuous linear functional in the dual space; the appendix also gives corresponding characterizations for Lp and W1,p spaces.These characterizations are specified for 1 < p < ∞.Source: Appendix B · p. 27
  • Γ-convergence requires a liminf inequality for every weakly convergent sequence and a weakly convergent recovery sequence for every point.The appendix states this definition using functionals on a space X.Source: Appendix B · pp. 27–28
  • Coercivity means each functional sublevel set is bounded, while equi-coercivity requires the union of sublevel sets across the sequence to be bounded.Both definitions are given for functionals on a reflexive Banach space.Source: Appendix B · p. 28
  • For equi-coercive functionals that Γ-converge to a functional with a unique minimizer, every sequence of minimizers converges weakly to that minimizer.The theorem also states that the limit functional is coercive.Source: Appendix B · p. 28
Loading 2609.30049v1…