PaperlayerView exact revision on arXiv

Source-linked AI summary

Convex Equilibrium Potentials for Monge--Amp\`ere: Curvature Bounds and A Posteriori Evaluation

Edison Vázquez, Yingdong Lu, Tomasz Nowicki

arXiv:2610.31386v1math.NAcs.LGcs.NA

TL;DR

Monge–Ampère applications of equilibrium networks must contend with curvature growth and the fact that their potential is evaluated only approximately. The paper derives curvature and residual-based certificates for nonnegative softplus equilibrium networks, then applies them to trained models on the unit disk. All six models meet the stated residual and boundary-error targets, while the guarantees remain limited to evaluation of fixed trained networks.

  • Problem

    Equilibrium networks used for Monge–Ampère problems raise questions about depth-related curvature and how to distinguish approximate evaluation from PDE residual and solution error.

    Source: §1 · p. 2

  • Method

    The paper combines positivity-based curvature analysis with residual-based estimates and interval methods to certify fixed-point evaluation and PDE error.

    Source: §1 · p. 2, §6.1 · p. 9

  • Results

    All six trained networks on two manufactured unit-disk problems are certified with residual below 0.25 and boundary error below 0.01.

    Source: §1 · p. 3

  • Takeaways & Limitations

    The results establish uniform residual and solution-error bounds for frozen models, while float64 adaptive evaluation saves time and certified adaptive evaluation remains costlier than fixed depth.

    Source: §10 · p. 18

    The guarantees cover evaluation of frozen networks, not training convergence, approximation or optimization errors, or comparisons with explicit ICNNs and other PDE solvers.

    Source: §9 · p. 17

Abstract

from arXiv · show

Equilibrium neural networks define their hidden state as the fixed point of a contraction, so their effective depth is set at evaluation time rather than by the architecture. Using them for fully nonlinear PDEs raises two issues that explicit networks avoid: second derivatives may grow with the effective depth, and the potential is only ever evaluated approximately. We study nonnegative softplus equilibrium networks with contraction factor $q<1$, a convex class suited to the Monge--Amp\`ere equation. A positivity cancellation in the resolvent shows that the Hessian grows at most like $(1-q)^{-2}$, rather than the $(1-q)^{-3}$ of a direct estimate, and an explicit one-neuron family shows that this exponent is sharp; third derivatives satisfy a sharp $(1-q)^{-3}$ bound. Residual-based a posteriori estimates then control the state, its input derivatives and, in two dimensions, the Hessian determinant at an approximate equilibrium. Combined with interval arithmetic on complete covers of the domain, they yield rigorous uniform bounds on the Monge--Amp\`ere residual and, through a comparison argument, on the solution error of a fixed trained network. On two manufactured Dirichlet problems on the unit disk, all six trained models are certified with residual below $0.25$ and boundary error below $0.01$. Adaptive stopping halves float64 evaluation time at best, whereas certified adaptive evaluation remains costlier than fixed depth.

1 Introduction

The introduction motivates equilibrium networks for Monge–Ampère problems while identifying curvature growth and approximate evaluation as obstacles to reliable guarantees. It presents curvature bounds, residual-based evaluation estimates, and domain-wide certification, then states the paper’s related-work context and scope.

1 Introduction

  • The proposed representation uses a fixed-point iteration to set network depth at evaluation time, retaining the convexity principle of input convex neural networks.The equilibrium approach is situated alongside prior input-convex, equilibrium-network, and Monge–Ampère approximation work.Source: §1 · pp. 1–2, §1.1 · p. 3
  • Approximate equilibrium evaluation and depth-linked Hessian growth are distinct obstacles because the Monge–Ampère determinant amplifies second-derivative behavior.The paper distinguishes evaluation error, PDE residual, and solution error; a guarantee for the potential as an approximation to the PDE solution must bridge all three.Source: §1 · p. 2
  • A positivity cancellation improves the Hessian bound to order δ^-2, sharply, while third derivatives have a sharp δ^-3 bound.Residual-based estimates control state and input-derivative errors, and in two dimensions the Hessian determinant, at approximate equilibria; interval arithmetic over complete domain and boundary covers gives uniform residual bounds that a comparison argument converts to solution-error bounds.Source: §1 · p. 2
  • All six networks on two manufactured unit-disk problems are certified with residual below 0.25 and boundary error below 0.01.The study also measures adaptive-evaluation savings and certification cost.Source: §1 · p. 3

1.1 Related work

  • The paper’s a posteriori estimates extend general fixed-point derivative identities by accounting for coefficient perturbations when the state is approximate.The related-work discussion also distinguishes evaluation error from PDE residual and solution error, and identifies the positive softplus resolvent as the source of the curvature cancellation.Source: §1.1 · p. 3

1.2 Organization and scope

  • The guarantees concern evaluation of fixed trained models, not convergence of training or superiority over explicit ICNNs.The paper’s organization places curvature analysis, approximate-state certification, and full-domain estimates before the adaptive-evaluation experiments.Source: §1.2 · p. 4

2 Assumptions, notation and the equilibrium

The equilibrium representation is analyzed with fixed parameters and a contractive state map. Its convexity follows from the softplus activation and the stated assumptions, while the determinant certificate and experiments are restricted to two input dimensions.

2 Assumptions, notation and the equilibrium

  • The representation allows x in R^n and m hidden units, while determinant certification and experiments are stated for n = 2.The parameter matrices and vectors have the dimensions specified in the paper’s notation.Source: §2 · p. 4
  • Parameters remain fixed when differentiating with respect to x, and the contraction factor q lies strictly between zero and one.Vector and matrix inequalities are entrywise unless stated otherwise; the infinity norm is the maximum absolute row sum.Source: §2 · p. 4
  • Under the assumptions, softplus is smooth, convex, and 1-Lipschitz, so the state map has a unique equilibrium and the resulting potential is convex.The input matrix and affine readout are unrestricted in sign, and the paper states that its precise results and proofs are in Appendix A.Source: §2 · p. 4

3 Implicit derivatives and positive Hessian structure

Implicit differentiation expresses the equilibrium’s first and second input derivatives through resolvent operators. These identities expose the positive structure used to establish that the potential’s Hessian is positive semidefinite.

3 Implicit derivatives and positive Hessian structure

  • The first and second state derivatives satisfy BJi = ΨA:,i and BKij = γ ⊙ vi ⊙ vj, with vi = V:,i.The paper notes that the order of ΨW and WΨ in the definitions matters.Source: §3 · pp. 4–5
  • The resolvent S = B^-1 has S ≥ I and infinity norm at most δ^-1, and β is defined as (S^T w) ⊙ γ.The definitions use the equilibrium pre-activation s, its softplus derivatives, and B = I − ΨW.Source: §3 · pp. 4–5
  • The Hessian is H = V^T diag(β)V and is positive semidefinite; the directional second derivative of the state is S[γ ⊙ (Vh)^2].The paper identifies these formulas as the positive structure underlying the curvature argument.Source: §3 · p. 5

4 A curvature bound with an optimal gap exponent

A positivity cancellation in the resolvent yields a uniform Hessian bound with inverse-gap exponent two, and an explicit family proves that exponent is sharp. The section also establishes a sharp inverse-gap exponent of three for third derivatives and extends the curvature estimate to suitable weighted norms.

4 A curvature bound with an optimal gap exponent

  • The Hessian bound is uniform in the input and bias, with width entering only through parameter norms.The positivity cancellation avoids multiplying worst-case resolvent and activation-curvature bounds; contraction alone does not uniformly bound the Hessian over arbitrary readouts and input weights.Source: §4 · p. 5
  • The quadratic estimate is tighter when q > 3/4, while the direct cubic estimate may be tighter when q ≤ 3/4.The bound retains the readout and input-weight norms; contraction alone does not provide a uniform Hessian bound over arbitrary such parameters.Source: §4 · p. 5
  • The inverse-gap exponent two is optimal: the one-neuron family has rescaled curvature tending to 1/8 as the contraction gap shrinks.The theorem rules out a uniform bound with any exponent p < 2; it establishes optimality of the exponent, not of the constant in Theorem 4.2.Source: §4 · p. 6, Figure 1 · p. 6

4.1 Sharp third-derivative bound

  • The inverse-gap exponent three for third derivatives is optimal over the class under assumption (H).The trilinear-form norm takes a supremum over three independent Euclidean unit vectors.Source: §4.1 · p. 7
  • The positive cancellation extends to weighted norms when a positive vector η satisfies Wη ≤ qη, including when ρ(W) < q < 1.The numerical experiments use the unweighted row norm.Source: §4.1 · p. 7

5 A posteriori evaluation from an approximate state

Section 5 turns approximate equilibrium states and derivatives into a posteriori certificates. State-aware bounds propagate residuals to the potential, its derivatives, and—in two dimensions—the Hessian determinant and convexity.

5 A posteriori evaluation from an approximate state

  • The state-aware derivative certificate accounts for both numerical residuals and coefficient changes caused by using an approximate state.The approximate quantities may come from different algorithms and need not solve their defining equations exactly.Source: §5 · pp. 7–8
  • The potential value approximation satisfies |uθ(x) − ˆu| ≤ ∥w∥1 Ez.Source: §5 · p. 8
  • In two dimensions, the determinant error is bounded by Edet, supporting bounds on the PDE residual and a positive-definiteness certificate.A strictly positive right-hand side in the stated convexity bound certifies positive definiteness at that input.Source: §5 · p. 8
  • A machine certificate requires outward enclosures or explicit error bounds for finite-precision residuals, nonlinear functions, norms, linear algebra, and determinant or eigenvalue computations.Float64 diagnostic checks are distinct from validated bounds.Source: §5 · p. 8

6 Local bounds, complete coverage and solution error

Section 6 extends pointwise certificates to regional bounds over a complete domain cover, then uses comparison to bound solution error. The evaluated regional route uses interval arithmetic; an analytical construction is treated separately in Appendix D.

6 Local bounds, complete coverage and solution error

  • The evaluated interval-residual method is distinct from the alternative analytical construction based on local derivative envelopes in Appendix D.Source: §6 · p. 8

6.1 A regional residual route with controlled arithmetic

  • The interval-residual route encloses determinants and bounds with outward rounding, while treating saved binary64 parameters as exact dyadic rationals for the frozen model.This certification does not verify the rounding history of optimization.Source: §6.1 · p. 9
  • Local boxes contain every exact equilibrium state over a region and can be tightened by interval intersections and verified derivative solves.If a solve cannot be verified, the residual bound is retained.Source: §6.1 · p. 9
  • Exact dyadic boxes and partition audits preserve a complete disk cover; circle-touching boxes are retained, though a valid bound can remain uninformative.Boundary arcs use interval π, sine, and cosine.Source: §6.1 · p. 9

6.2 From a uniform residual to a solution bound

  • For smooth targets on the unit disk, comparison gives an explicit barrier coefficient with square-root dependence on residual; manufactured problems also admit direct solution-error bounds.The direct bounds use the known manufactured solution and do not require a stability constant; nonsmooth Aleksandrov targets require additional measure-comparison results.Source: §6.2 · p. 10

7 Evaluation procedure and stopping criteria

Section 7 describes how to compute approximate states, derivatives, and determinant-error bounds, then refine evaluation if needed. It gives absolute and relative stopping criteria with qualifications near zero residual.

7 Evaluation procedure and stopping criteria

  • At each input, one factorization of an m × m matrix can be reused for the two first-derivative and three distinct second-derivative right-hand sides.An explicit inverse is unnecessary; dense factorization has the usual cubic cost in m.Source: §7 · p. 10
  • Stop when Edet ≤ τabs or, for positive observed residual, when Edet ≤ ηr̂ with 0 < η < 1.If the evaluation tolerance is unmet, refine the state or derivative computations and recompute the bounds.Source: §7 · p. 10
  • The true-residual-relative error ratio is bounded by η/(1 − η), not η; near r̂ = 0 an absolute tolerance remains necessary.The bounds are not assumed to vary monotonically with iteration count.Source: §7 · p. 10

8 Numerical experiments

The experiments test derivative-bound consistency, evaluation accuracy and cost, and the extension of pointwise guarantees to the full domain. Six trained models are assessed on two manufactured Dirichlet problems on the unit disk.

8 Numerical experiments

  • The experiments compare a posteriori bounds with observed evaluation errors, test adaptive stopping costs, and assess full-domain certification.Float64 diagnostics and interval guarantees are reported separately.Source: §8 · p. 11

8.1 Consistency of the derivative bounds

  • The 120-network consistency experiment reports no violations of the derivative bounds.The largest computed reference-state residual bound is 1.99 × 10^-12.Source: §8.1 · p. 11, Table 1 · p. 11

8.2 Problems, training and evaluation protocol

  • The six models use width 12, nominal q = 0.7, and 32-step finite-unroll training; the study does not benchmark an equilibrium implicit backward pass.There are three seeds per problem, and interior target values are used only for evaluation.Source: §8.2 · p. 12

8.3 Accuracy of the six trained models

  • The six models have sampled relative errors of 0.2847% ± 0.0333% for P1 and 0.2941% ± 0.0178% for P2; these are not uniform guarantees.The summaries are across three seeds.Source: §8.3 · p. 12

8.4 Pointwise certificates and iteration depth

  • At depth 32, all six sampled cohorts satisfy the float64 relative criterion, though a fixed iteration threshold does not certify every point at lower depths.At depth 12, 62.99–89.11% of P1 points meet the criterion; at depth 16, 86.52–92.38% of P2 points do.Source: §8.4 · p. 13

8.5 Cost of adaptive evaluation

  • At tolerance 10^-4, adaptive float64 evaluation is 1.45–1.99 times faster, while adaptive Arb evaluation is 1.48–2.66 times slower.The float64 comparison uses 512 points per model; the Arb comparison uses eight, and the cohorts and baselines differ.Source: §8.5 · p. 15, Figure 3 · p. 15

8.6 Complete-domain and boundary certification

  • Every model has a uniform residual bound below 0.25 and boundary error below 0.01 after complete-domain certification.For P2, seed 2, post hoc refinement lowers the residual bound to 0.249951 while retaining a complete cover; direct solution-error bounds are tighter than barrier bounds in every case.Source: §8.6 · pp. 16–17, Table 4 · p. 16, Figure 4 · p. 16

9 Discussion and limitations

The guarantees certify evaluation of frozen networks, not convergence of training or its approximation, optimization, and loss-discretization errors. Empirical coverage is limited, numerical bounds can be loose, and rigorous adaptive stopping is slower than fixed-depth certification in the tested implementation.

Evaluation and training

  • The certificates apply to frozen networks and do not establish convergence of sampled strong-loss training or control training-sequence approximation, optimization, and loss-discretization errors.Empirical coverage is limited to two manufactured problems with three seeds each, with no comparison against explicit ICNNs or other PDE solvers.Source: §9 · p. 17

Scope and tightness

  • The inverse-gap exponents are sharp, but optimal prefactors remain open; weighted norms and analytical local envelopes are theoretical extensions rather than part of the implemented regional path.The implemented path uses unweighted interval residuals and verified solves; the disk barrier applies to smooth convex targets.Source: §9 · p. 17
  • The disk solution-error estimate has square-root dependence, and direct interval bounds substantially exceed observed sample errors.These bounds can therefore be loose relative to the sampled errors.Source: §9 · p. 17

Performance and numerical guarantees

  • Adaptive float64 evaluation reduces measured runtime, but full Arb stopping is slower in the tested implementation, and the complete software stack is not formally verified.Interval batching, reuse, and stopping-policy design are possible improvements, not demonstrated gains.Source: §9 · p. 17

10 Conclusion

The paper establishes curvature and derivative bounds for nonnegative contractive softplus equilibrium networks and develops certificates for approximate evaluation. Its six-model study meets prescribed residual and boundary targets, while measured rigorous adaptive stopping remains more expensive than fixed-depth certification.

10 Conclusion

  • 73,728 point/depth interval certificates and a bounded fixed-model refinement yield complete covers meeting residual and boundary targets for all six models.The stated targets are residual below 0.25 and boundary error below 0.01.Source: §10 · p. 18
  • Positivity gives a uniform Hessian bound with optimal inverse-gap exponent two, while third derivatives have sharp exponent three.The Hessian cancellation is expressed as Sγ ≤1.Source: §10 · p. 18
  • Residual-based perturbation controls derivatives at approximate states, and regional interval residuals with complete boundary covers provide uniform frozen-model bounds and a disk residual-to-solution estimate.The determinant is included in the approximate-state derivative control in two dimensions.Source: §10 · p. 18
  • Adaptive evaluation saves time with the float64 estimator, whereas rigorous adaptive stopping is more expensive than fixed-depth certification in the measured cohorts.The conclusion presents these findings as quantifying both the usefulness and cost of certified evaluation.Source: §10 · p. 18

Use of AI tools

The appendix records AI-tool use and supplies supporting results on softplus equilibria and their derivative structure. It establishes existence, smoothness, and convexity, then gives a Hessian factorization and qualifications on strict convexity and determinant evaluation.

Use of AI tools

  • The authors used ChatGPT and Claude for language editing, notation and statement consistency checks, and reference verification, while retaining responsibility for the mathematical results and conclusions.They report reviewing and editing all AI-assisted content.Source: Use of AI tools · p. 18

A.1 Softplus and the equilibrium

  • Softplus is increasing, convex, and 1-Lipschitz, and under assumption (H) the fixed point is unique while the state components and potential are convex.The equilibrium is smooth, and the potential has the stated Lipschitz bound in terms of the model parameters and contraction gap.Source: §A.1 · p. 21

A.2 Implicit derivatives and Hessian structure

  • The Hessian factorizes through nonnegative weights and the matrix V; with strictly positive readout weights, positive definiteness is equivalent to rank A = n.Strictly positive readout weights are sufficient, not necessary, for strict convexity; the two-dimensional determinant formula is an algebraic identity, not a guarantee about numerical roundoff.Source: §A.2 · p. 22

B Proof of the third-derivative bound

The appendix proves the sharp third-derivative bound and extends the curvature and certification analysis through weighted norms, local envelopes, residual perturbation, and finite-unroll derivative recurrences. The resulting regional and disk comparison arguments connect local certificates to uniform residual and solution-error bounds.

B Proof of the third-derivative bound

  • The third-derivative bound has optimal inverse-gap exponent three, established using positivity and cancellation in the resolvent and a one-neuron sharpness family.The family has q = 1 − ε at x = 0, and ε^3u′′′(0) tends to 1/32.Source: B Proof of the third-derivative bound · pp. 22–23

C Positive weighted norms

  • A positive vector η satisfying Wη ≤ qη yields a weighted curvature estimate, and such a vector exists for every ρ(W) < q < 1.The spectral-radius endpoint need not be attained by a strictly positive minimizing vector.Source: C Positive weighted norms · p. 23

D Analytical local envelopes

  • Analytical local envelopes bound preactivations and derivatives over a region, enabling a center-plus-variation construction for the two-dimensional determinant residual.The local determinant-residual Lipschitz bound uses Hessian and third-derivative bounds together with Lip(f).Source: D Analytical local envelopes · p. 24

E.1 Proof of Theorem 5.1; E.2 Proof of Corollary 5.2

  • Residual perturbation estimates propagate approximate-state errors through first and second derivatives to the Hessian and, in two dimensions, its determinant.The proof bounds determinant error using the two-dimensional determinant identity and matrix norm inequalities.Source: §E.1 · pp. 24–25

F.1 Proof of Proposition 6.1; F.2 Proof of Corollary 6.2

  • Uniform interval enclosures over certified covering elements turn pointwise residual bounds into uniform output bounds; floating-point residuals alone are not upper bounds.The circle parameterization transfers angular bounds to the complete boundary.Source: §F.1 · p. 25, §F.2 · p. 25

F.3 Proof of Proposition 6.3 √

  • A convex disk barrier and the Monge–Ampère comparison principle convert residual and boundary bounds into a solution-error estimate.The proof applies the comparison principle in both directions to bound the difference between the two solutions.Source: §F.3 · p. 26

G Finite-unroll derivative recurrences

  • Finite-unroll derivatives can be computed by recurrences, but certificate coefficients must be recomputed at the final unrolled state because derivative residuals are generally nonzero.The recurrences initialize state and first- and second-derivative iterates at zero.Source: G Finite-unroll derivative recurrences · p. 26
Loading 2610.31386v1…