PaperlayerView exact revision on arXiv

Source-linked AI summary

On the gradient conjecture at infinity: Finite spherical length for gradient trajectories of definable potentials

Thang Do, Arnulf Jentzen, Benno Kuckuck

arXiv:2610.29211v1math.DScs.LGmath.AGmath.CAmath.LO

TL;DR

Deep-neural-network training trajectories may be unbounded or diverge to infinity, motivating questions about their behavior at infinity. The paper constructs energy transformations that yield Lyapunov-type functions with greater dissipation, and proves finite spherical length and directional convergence for every gradient trajectory of a definable C^1 potential.

  • Problem

    For DNN training, simulations and analyses in specific settings suggest that most trajectories are unbounded or diverge to infinity rather than converging to critical points.

    Source: §1.4 · p. 6

  • Method

    The proof constructs energy transformations to obtain Lyapunov-type functions with larger dissipation.

    Source: §3 · p. 25

  • Results

    Every gradient trajectory of a definable C^1 potential on an open set has a spherical projection of finite length and, in particular, converges directionally.

    Source: §1 · p. 3

  • Takeaways & Limitations

    For DNN training, the spherical projection is stated to have a unique limiting direction.

    Source: §1.4 · p. 6

    The result requires definability: replacing that assumption with analyticity does not in general preserve the conclusion.

    Source: §1.1 · p. 3

Abstract

from arXiv · show

We prove that for every open $ U \subseteq \mathbb R^n $ the spherical projection of every gradient trajectory $\theta(t)$, $t \geq 0$, of a definable $C^1$-potential $F \colon U \to \mathbb R$ has finite length and, in particular, converges (directional convergence). In other words, every gradient trajectory of a definable $C^1$-potential has finite total angular motion. Here, definable means definable in an arbitrary o-minimal structure expanding the real field. Notably, the gradient trajectory in this result is an arbitrary solution of the gradient flow differential equation (whose solutions may not be unique) with no further assumptions: The gradient trajectory may be unbounded and diverge to infinity or it may be convergent. The established theorem therefore provides affirmative answers to both the gradient conjecture at infinity for definable potentials and the definable Thom gradient conjecture. We apply our theorem to the training of deep neural networks (DNNs) to conclude that, in the case of definable and continuously differentiable activation and loss functions, the spherical projection of the DNN training flow must always converge (directional convergence).

1 Introduction and main result

The section establishes that every gradient-flow trajectory of a definable C1 potential has a finite-length spherical projection and directional convergence, including arbitrary solutions that diverge to infinity, with an application to DNN training flows under stated assumptions. It outlines a proof using a bounded continuous Lyapunov-type function to control angular length, with truncation and approximation addressing boundaries where transformations may fail to be differentiable.

1.1 Finite spherical length of gradient flow (GF) trajectories

  • Every gradient-flow trajectory of a definable C1 potential has a spherical projection of finite length, hence directional convergence.The theorem applies on open U ⊆ R^n and to trajectories over any convex time interval with nonempty interior.Source: §1 · p. 3, §1.1 · p. 3
  • The theorem covers trajectories that diverge to infinity or remain bounded, and requires only C1 regularity, so solutions need not be unique.If the trajectory has accumulation points at both ends of its time interval, the finite-length conclusion also holds for locally definable potentials.Source: §1.1 · pp. 3–4
  • Definability is essential: the finite-length conclusion can fail for analytic potentials when definability is absent.A function is definable when its graph is definable in an o-minimal structure; analyticity alone does not ensure this property.Source: §1.1 · p. 3, §1.3 · p. 5

1.2 Related works and key novelty of this article

  • The theorem resolves the definable Thom gradient conjecture and the definable gradient conjecture at infinity, establishing finite spherical length in both regimes.The regimes include convergence to a critical point and divergence to infinity.Source: §1.1 · p. 3, §1.2 · p. 5
  • The strong gradient, or non-oscillation, conjecture remains open in most cases, even for analytic potentials.The paper distinguishes this open conjecture from the definable conjectures addressed by its main result.Source: §1.2 · p. 5

1.3 Definability, o-minimal structures, and rectifiable curves

  • The section defines definable functions through their graphs in an o-minimal structure and introduces local definability for the corollary.An o-minimal structure requires every definable subset of the real line to be a finite union of intervals.Source: §1.3 · p. 5

1.4 Connections to the training of deep neural networks (DNNs)

  • For DNNs with definable, continuously differentiable activation and loss functions, the training flow's spherical projection converges directionally.This conclusion applies even in the diverging scenario where convergence of the full gradient-flow trajectory cannot be concluded.Source: §1.4 · p. 6

1.5 Outline of the proof and structure of this article

  • The proof reduces the critical-point and infinity cases to radial-angular differential systems and establishes finite angular length using a Lyapunov-type stability estimate.The transformed system tracks a radial variable and the spherical projection; a bounded Lyapunov-type function controls angular motion near the radial limit.Source: §1.5 · p. 7
  • The original Lyapunov energy dissipates the squared gradient norm, which may be too small to control trajectory length when the gradient is small.The proof seeks a transformed energy whose dissipation controls length up to a radial error.Source: §1.5 · p. 8
  • Within each radial-energy region, transformed energies and back-transform derivative bounds control angular dissipation, while definable weights balance the additional radial error.The regionwise estimate alone is insufficient because trajectories can cross region boundaries and the transformation depends on the radial component.Source: §1.5 · pp. 8–9
  • The proof constructs a bounded, continuous Lyapunov-type function Q from weighted transformed energies across finitely many radial-energy regions.The weights balance radial error against angular dissipation.Source: §1.5 · pp. 8–9
  • Truncating transformed energies away from their endpoints yields approximations that permit absolute-continuity estimates and passage to the limit across nondifferentiable region boundaries.At times spent on a boundary, the proof uses that an absolutely continuous function's derivative vanishes almost everywhere on each level set.Source: §1.5 · p. 9
  • The proof normalizes the energy for Proposition 5.3, except when the energy already takes values in (-1, 1), where normalization is unnecessary.In the latter case, the transformation procedure applies directly.Source: §1.5 · p. 9

2 Regularity properties of radial upper bounds for definable C1-scalar fields

This section develops geometric and smoothness preliminaries for radial upper bounds of definable C1 scalar fields, establishing continuity and definability of radial and angular derivatives on definable submanifolds. Using finite-length properties of bounded definable curves, it proves that these upper bounds are integrable and, on a sufficiently small interval, continuously differentiable, with results for one-dimensional and general bounded definable manifolds.

2 Regularity properties of radial upper bounds for definable C1-scalar fields

  • The section’s main results establish integrability, smoothness, and definability properties of radial upper bounds for definable C1-scalar fields on definable Cr-submanifolds.These are Lemmas 2.11 and 2.13.Source: §2 · p. 9
  • Lemma 2.11 bounds the radial derivative of a definable C1-scalar field using a definable, locally integrable C1-function of the radial variable.The field is defined on a relatively open subset of the product of a radial interval and a bounded definable C1-submanifold.Source: §2 · p. 9
  • Lemma 2.13 bounds |∂rh − a′| by β(r)|∇Mh| when h(r, x) = a(r) for sufficiently small r, with β definable and locally integrable.The bound is used in Proposition 5.3 to control radial error at times when the energy equals a boundary value.Source: §2 · p. 10

2.1 Embedded submanifolds

  • An embedded C1-submanifold is locally represented by a C1-diffeomorphism that maps it to a coordinate plane.This local normal form supplies a geometric preliminary for the section’s subsequent arguments.Source: §2.1 · p. 10

2.2 Smooth functions on embedded submanifolds

  • On embedded submanifolds, the angular gradient is uniquely defined, and the radial partial derivative of a definable C1-function is continuous and definable.The angular gradient represents derivatives along curves in the submanifold.Source: §2.2 · pp. 13, 15
  • For definable C1 data on an open subset of a definable C1-submanifold, the radial derivative and angular gradient are continuous and definable.The radial derivative result is established for f, and Lemma 2.9 gives the corresponding result for ∇MZ.Source: §2.2 · p. 16, §2.3 · p. 21

2.3 Integrability, smoothness, and definability properties of radial upper bounds

  • Every bounded definable curve has finite length near zero and converges there.The lemma also gives eventual coordinate-wise monotonicity or constancy and smoothness, supporting these conclusions.Source: §2.3 · pp. 18–19
  • Under Lemma 2.11’s hypotheses, the radial upper bound η is continuously differentiable, integrable, and definable on a sufficiently small interval near zero.The lemma assumes, among other conditions, a bounded definable C1-submanifold and a definable C1 function bounded on Ω.Source: §2.3 · p. 19
  • The proof uses definable choice to select points along the radial slices, then applies bounded definable curve regularity to the selected curve and function values.This construction is used to control the radial upper bound near zero.Source: §2.3 · pp. 18, 20
  • Lemma 2.11’s radial upper bound η is definable and, on a sufficiently small interval, continuously differentiable, locally bounded, and integrable.The proof concludes that its stated items (i) and (ii) hold after restricting the interval.Source: §2.3 · p. 22
  • For the one-dimensional manifold (0, H), Lemma 2.12 specializes the preceding result to radial upper bounds with the same regularity properties.The passage states that the bound is continuously differentiable, integrable, and definable on a sufficiently small interval.Source: §2.3 · p. 22
  • For a bounded definable C1-manifold, the radial upper bound β is continuously differentiable, integrable, and definable on some interval (0, r1).The result applies to definable C1 scalar fields h and a under the stated hypotheses of Lemma 2.13.Source: §2.3 · pp. 22, 25

3 Construction of the radial-energy regions and the energy transformations

This section constructs definable energy back-transformations on energy bands to strengthen dissipation of the radial-angular system’s Lyapunov energy. It develops auxiliary bounds to establish radial-derivative estimates that support a later estimate of angular length.

3 Construction of the radial-energy regions and the energy transformations

  • Because the original Lyapunov energy h has insufficient dissipation to bound angular length, the section constructs transformations that yield Lyapunov-type functions with greater dissipation.The construction introduces F0, F1, …, FN−1 and associated back-transformations E1, E2, …, EN.Source: §3 · p. 25

3.1 Smooth cylindrical definable cell decompositions

  • The section recalls definable partitions, compatible partitions, refinements, and smooth cylindrical definable cell decompositions as tools for constructing the energy transformations.Source: §3 · p. 25, §3.1 · pp. 25–26

3.2 Upper and lower bounds for the energy-derivatives of the back-transformations

  • Lemma 3.5 constructs the transformations and back-transformations and bounds their energy derivatives; Lemmas 3.6 and 3.7 provide a growth bound and radial-derivative bounds.The overview identifies these bounds as applying to ∂zEi and ∂rEi, respectively.Source: §3 · p. 25, §3.2 · p. 26
  • There exists a definable function ζ:D\C→M such that, for every (r,e)∈D\C, the selected point (r,ζ(r,e)) lies in Ω, has energy h(r,ζ(r,e))=e, and satisfies G(r,ζ(r,e))<2µ(r,e).The cited passage also states that O is definable and open, with ζ|O ∈ C2(O,M).Source: §3.2 · p. 31
  • The energy range is partitioned by finitely many ordered definable C2 curves, yielding bands on which the constructed energy maps are smooth and strictly increasing.On each band, the construction gives an inverse E_i with positive energy derivative T_i; the proof then establishes the six properties of Lemma 3.5.Source: §3.2 · pp. 32–34

3.3 Upper and lower bounds for the radial-derivatives of the back-transformations

  • For nonnegative definable integrable functions, Lemma 3.6 proves its two stated properties, handling a finite family by choosing a common smaller radius interval.The proof of its first item uses eventual monotonicity and dominated convergence.Source: §3.3 · pp. 34–35
  • Under the stated assumptions on the energy-band maps and their inverses, Lemma 3.7 yields definable C1 functions m_i ≥ 1 on a smaller radius interval.The lemma states two bounds for each band, but their formulas are not included in the supplied passages.Source: §3.3 · p. 35
  • The back-transformations Zi are definable and C1 on Ωi, satisfy h(r, x) = Ei(r, Zi(r, x)), and have Ti(r, Zi(r, x)) > 0.These properties hold for each i on its region Ωi.Source: §3.3 · p. 36
  • For each i, an auxiliary function ηi is continuously differentiable, definable, and integrable on a sufficiently small radial interval.The interval is (0, ρi) for some ρi in (0, r1).Source: §3.3 · p. 37
  • The proof combines these constructions to establish both items (i) and (ii) of Lemma 3.7, giving the stated radial-derivative bounds.The proof concludes after establishing item (i) and then item (ii).Source: §3.3 · p. 38

4 Construction and regularity of the truncation of the transformed energy

This section addresses the transformed energy’s limited regularity by truncating it and establishing level-set and regularity properties, completing Lemma 4.3 through an analysis of the decomposition of its complement.

4.1 Vanishing of the derivatives of absolutely continuous functions on their level sets

  • For an absolutely continuous function, its derivative vanishes almost everywhere on each level set f⁻¹({c}).Lemma 4.1 establishes this elementary fact for a function on a compact interval.Source: §4.1 · pp. 38–39

4.2 Construction and uniform regularity of the truncation of the transformed energy

  • Truncating the transformed energy to [ε, H −ε] yields a locally Lipschitz truncation and composition, while the trajectory quantities ζ and Q are locally absolutely continuous.The truncation is max{ε, min{H −ε, Z}}; the regularity results hold under Lemma 4.3’s stated assumptions.Source: §4 · p. 38, §4.2 · p. 41
  • Almost everywhere when ε < v(t) < H −ε, ω(ϱ(t), ζ(t))ζ′(t) is at most both 2m(ϱ(t))|ϱ′(t)| and −α(t)γ(t).These bounds apply on the interior truncation range under the assumptions of Lemma 4.3.Source: §4.2 · pp. 41, 44
  • J\A is identified with the times when ζ(t) equals either truncation level, ε or H − ε.Source: §4.2 · p. 44
  • Continuity of ζ and the fundamental theorem for absolutely continuous functions are used to establish an almost-everywhere relation on J\A.Source: §4.2 · p. 44
  • The proof combines the preceding estimates to establish items (iii), (iv), and (v), completing Lemma 4.3.Source: §4.2 · p. 44

5 Lyapunov-type stability for the radial-angular differential equation (DE) system

This section develops Lyapunov-type estimates for radial-angular differential systems, including bounds on their variation and errors along locally absolutely continuous trajectories. Under the stated assumptions, it uses these estimates to control radial-component behavior and establish finite angular length near r = 0, alongside uniform bounds and regularity results for definable primitives.

5 Lyapunov-type stability for the radial-angular differential equation (DE) system

  • The angular component has finite length when the radius tends to zero at the endpoint of a half-open time interval.These radial-angular systems arise by transforming gradient flow into radial and spherical components; the finite-length result is used to prove finite length of the spherical projection.Source: §5 · p. 45

5.1 Regularity properties for one-parameter primitives of definable two-parameter functions

  • Lemma 5.1 gives the primitive P bounds and Lipschitz continuity in z, differentiability in z with derivative ω, and a definable integrable function σ.These properties are used in the proof of the Lyapunov-type estimate.Source: §5.1 · p. 46

5.2 Lyapunov-type stability for the radial-angular dynamics

  • Proposition 5.3 establishes a Lyapunov-type estimate for radial-angular trajectories using a bounded function Q and a definable, integrable function m.The estimate applies to locally absolutely continuous radial and angular variables satisfying the displayed differential equations.Source: §5.2 · pp. 48–49
  • The section constructs definable auxiliary functions and integrable bounds that support the Lyapunov estimate.The construction includes definable βj functions that are continuously differentiable and integrable, and a nonnegative measurable function m with finite integral.Source: §5.2 · pp. 51–52
  • The cited stability estimate uses trajectories satisfying the radial-angular system r′(t)=−α(t)r(t)^2(∂rh)(r(t),x(t)) and x′(t)=−α(t)A(t)(∇Mh)(r(t),x(t)), for a,b∈I with a≤b.Source: §5.2 · p. 52
  • For the quantities r(t), x(t), and Vε shown in the cited inequality, the time derivative d/dt Vε(r(t), x(t)) is bounded above by −ℓ(t)1_Uε(r(t), x(t)) + M(r(t))|r′(t)|.Source: §5.2 · p. 55
  • Proposition 5.3 establishes estimates (5.74) and (5.41) for the radial-angular dynamics under locally absolutely continuous paths and measurable coefficients.The displayed dynamics use nonnegative α and measurable A.Source: §5.2 · p. 57

5.3 Upper bounds for the error in the Lyapunov-type estimate

  • The shell-measure estimate gives finite ν_R with ν_R((0,ρ]) ≤ 2Cρ; definable, locally bounded, integrable m also admits a definable non-increasing integrable majorant near zero.The measure bound is stated for every R and ρ in the lemma’s specified ranges.Source: §5.3 · p. 58

5.4 Upper bounds for parts of the length of the radial component in the DE system

  • For definable C1 h on an open subset of (0,R)×M with compact definable C2 manifold M, Proposition 5.6 asserts a uniform C for admissible radial-angular trajectories.The proposition quantifies over locally absolutely continuous r and x and measurable α and A, with the trajectory remaining in the domain.Source: §5.4 · p. 61
  • For each i, 0 < z_i(ρ,ζ) − z_i(ρ,ξ) ≤ 2B, while q_i(ρ,e) lies in [0,d_i(ρ)] and takes every value in (0,d_i(ρ)) for e inside (e_i(ρ),e_{i+1}(ρ)).The difference between the endpoint limits of z_i is at most 2B.Source: §5.4 · pp. 64–65
  • At almost every time when the energy equals a partition level, the radial derivative vanishes, and the angular derivative is given by the stated A-dependent expression.The passages also state that α(t)w(t) and α(t)g(t) vanish at those times.Source: §5.4 · p. 66

5.5 Finite length of the angular component in the DE system

  • For the radial-angular system, Corollary 5.7 supplies a definable, non-increasing, integrable function ψ and asserts finite-length bounds for the angular component under the stated trajectory conditions.The corollary covers trajectories confined to sufficiently small radii and trajectories whose radius tends to zero at the endpoint.Source: §5.5 · p. 67

5.6 Uniform upper bounds for the length of the angular component in the DE system

  • Proposition 5.8 provides uniform radius and length bounds for the angular component of the stated radial-angular DE system.The proposition assumes a compact, connected, definable C2 manifold and a definable C1 function h; its proof concludes by establishing the proposition’s bound.Source: §5.6 · pp. 70, 74

6 Spherical length at finite points and at infinity

The section establishes finite spherical length for gradient-flow trajectories of definable and locally definable potentials across finite-point and endpoint-approach settings. It also applies these results to DNN training flows, establishing global existence and directional convergence of their spherical projections to a unique vector.

6 Spherical length at finite points and at infinity

  • Every gradient-flow trajectory of a definable C1-potential has a spherical projection of finite length on any time interval with non-empty interior.The result applies to potentials defined on arbitrary open subsets of R^n.Source: §6 · p. 74

6.1 Uniform upper bounds for the spherical length of GF trajectories close to the origin

  • For definable C1 potentials on a punctured ball, the spherical projection has a uniform upper length bound for trajectory periods spent sufficiently close to the origin.The corollary specifies an epsilon-neighborhood and a constant bound, uniformly over the stated intervals and trajectories.Source: §6.1 · p. 75

6.2 Finite spherical length for GF trajectories converging to finite points

  • If a gradient-flow trajectory has bounded liminf norm, it converges to a finite point.This is Proposition 6.3(i) for the definable C1-potential setting stated in the proposition.Source: §6.2 · pp. 78–79
  • If a trajectory approaches the reference point and the stated open-set condition holds, its spherical projection has finite length on every sufficiently late tail.The condition requires an open O with ({ϑ} ∩ {p}) ⊆ O ⊆ (U ∪ {ϑ}).Source: §6.2 · p. 79
  • Gradient trajectories converging to finite points have finite spherical length, with the proof treating distinct cases for the limit point and dimension.The cases are when the limit differs from the reference point, when they coincide in dimension one, and when they coincide in dimension at least two.Source: §6.2 · pp. 81–82

6.3 Finite spherical length for general open domains

  • For definable C1 potentials on general open sets, spherical projections of gradient trajectories have finite length, including when trajectories pass through the reference point.The proof uses convexity of the set of times at which a trajectory equals the reference point.Source: §6.3 · pp. 84–85

6.4 Finite spherical length for general time intervals

  • For definable C1 potentials, spherical projections of gradient trajectories on any convex time interval with nonempty interior have finite length.The proof combines the forward- and reversed-time results with a length bound for curves decomposed around an interior point.Source: §6.4 · p. 86

6.5 Finite spherical length for locally definable potentials

  • For a locally definable C1 potential, if a gradient trajectory has a sequence approaching a domain point at the endpoint, then it converges to that point there.The result states that lim inf distance to the point is zero only if lim sup distance is also zero.Source: §6.5 · p. 87
  • For a locally definable C1 potential, a gradient-flow trajectory approaching points in U at both interval endpoints has a spherical projection of finite length.The corollary states this for the projection centered at an arbitrary point ϑ in U.Source: §6.5 · p. 89

6.6 Infinite spherical length for an analytic potential

  • An analytic potential has a gradient trajectory whose spherical projection fails to converge, alternating between opposite directions at times tending to infinity.The example is not definable in any o-minimal structure, although it is locally Ran-definable.Source: §6.6 · pp. 90–91

6.7 Finite spherical length and directional convergence for DNN training flows

  • For DNN training flows with definable C1 activation and loss functions, the corollary gives finite spherical length and directional convergence of the spherical projection.The argument invokes the fact that every finite-length curve converges.Source: §6.7 · pp. 92–93
  • From every initial point p ∈ R^n, a global gradient trajectory exists.The proof rules out finite-time termination by extending a putative trajectory past its endpoint, contradicting maximality.Source: §6.7 · pp. 94–95
  • Every considered gradient trajectory has a spherical projection of finite length and a unique directional limit.For arbitrary θ satisfying the stated gradient-flow conditions, the projection P∘θ converges to a unique v, with lim sup as t→∞ of |(P∘θ)(t)−v| equal to 0.Source: §6.7 · pp. 93, 95

Use of large language models

The authors acknowledge using OpenAI language models as research collaborators on mathematical arguments, proof strategies, literature review, and exposition. They state that the authors independently checked all mathematical claims and proofs and retain sole responsibility for the article’s contents.

Use of large language models

  • OpenAI language models assisted with developing mathematical arguments and ideas and exploring proof strategies.The authors also report using them to test arguments and identify gaps and possible counterexamples.Source: Use of large language models · p. 95
  • The models were also used to refine proofs and exposition and review the literature.Source: Use of large language models · p. 95
  • The authors independently checked all mathematical statements, claims, and proofs, and accept sole responsibility for the article’s sentences, statements, and arguments.Source: Use of large language models · p. 95
Loading 2610.29211v1…