PaperlayerView exact revision on arXiv

Source-linked AI summary

A General Topological Symmetry Index for Critical Orbits In Neural Network Loss Landscapes

Jingzhou Liu

arXiv:2610.28864v1cs.LGcs.AI

TL;DR

Symmetry-based analyses of neural-network loss landscapes still lack a unified way to characterize critical orbits across architectures without first constructing the critical points. The paper develops an equivariant-degree topological symmetry index that certifies such orbits and their isotropy types under stated conditions. It also provides computation methods and examples, while its degree theory requires compact Lie groups.

  • Problem

    Existing analyses often focus on particular architectures or presuppose known critical points, leaving a need for a unified criterion for symmetric critical orbits.

    Source: §1 · p. 2

  • Method

    The paper defines a topological symmetry index from equivariant degree, using nonzero degree coefficients to identify critical orbits with specified isotropy types.

    Source: §1 · pp. 2–3

  • Results

    Under the theorem’s hypotheses, the index is well defined and its nonzero coefficients certify critical orbits whose isotropy is at least the associated type.

    Source: §1 · p. 3

  • Takeaways & Limitations

    The paper gives degree-based computation methods and illustrates them on softplus MLP, O(3)-equivariant network, and transformer examples.

    Source: §1 · p. 3

    The degree theories require compact Lie groups and therefore do not cover noncompact symmetries such as positive scaling for ReLU.

    Source: §2 · p. 3

Abstract

from arXiv · show

A systematic understanding of the global structure of neural-network loss landscapes remains challenging due to their high dimensionality. Symmetries arising from network architectures and learning objectives provide a way to investigate aspects of this structure. Although a growing number of studies have drawn attention to these symmetries, a systematic characterization of them, as well as of the critical orbits they give rise to, remains incomplete. To address this gap, in this work we establish a topological symmetry index, based on the methodologies from the family of equivariant degree theories, to decode these symmetries. The frameworks apply to arbitrary network architectures and include methods applicable to degenerate critical orbits. The index detects the isotropy of the critical points and quantifies the corresponding net topological contributions. Several examples, including softplus MLPs, Transformers, and equivariant neural networks, are presented to illustrate the computations.

1. Introduction

Prior analyses often focus on particular architectures or assume critical points are already known, leaving a need for a unified criterion to detect symmetric critical orbits. The paper introduces a topological symmetry index whose equivariant-degree coefficients certify orbit existence and bound isotropy types.

1. Introduction

  • Most prior loss-landscape analyses focus on a specific architecture or presuppose knowledge of the critical points, motivating a unified criterion that avoids constructing them explicitly.The cited work spans spurious minima and bifurcations, symmetry-related training dynamics, and symmetry-aware optimizer or network design.Source: §1 · pp. 1–2
  • The index uses equivariant degree to give a computable criterion for critical orbits with specified symmetries in a prescribed region.Its coefficients are associated with subgroup conjugacy classes; a nonzero coefficient certifies an orbit whose isotropy is at least that type.Source: §1 · pp. 2–3
  • The index is built from a G-invariant loss on a parameter space with an isometric group action, making its gradient G-equivariant.The target zeros of the gradient are precisely the loss’s critical points, with integer-valued topological weights associated with orbit types.Source: §1 · p. 2
  • The index is well defined for a finite-dimensional orthogonal representation of a compact Lie group when the loss is twice continuously differentiable and its gradient does not vanish on the boundary of the invariant region.The theorem’s conclusion identifies critical orbits with symmetry at least the corresponding orbit type for the selected maximal nonzero coefficients.Source: §1 · p. 3
  • The paper develops degree methods for linear representations and G-manifolds, then illustrates them on softplus MLP, O(3)-equivariant network, and transformer examples.The examples use the linear isomorphism method, slice principle, and recursive formula, respectively.Source: §1 · p. 3

2. Some Equivariant Terminology

The index is constructed from an invariant loss, its symmetry action, and a bounded invariant region where the gradient stays nonzero on the boundary. Its equivariant gradient degree takes values in the Euler ring, whose coefficients encode orbit types.

2. Some Equivariant Terminology

  • The framework requires a compact Lie symmetry group, so it does not cover noncompact symmetries such as positive scaling for ReLU.The region must also be open, bounded, invariant, and chosen so the gradient does not vanish on its boundary.Source: §2 · pp. 3–4
  • Constructing the index requires a G-invariant loss with G-equivariant gradient, an identified parameter-space action, and a suitable invariant domain.The domain condition is the admissibility requirement on which the equivariant degree construction rests.Source: §2 · pp. 3–4
  • The resulting gradient degree is an Euler-ring element with integer coefficients attached to conjugacy classes of subgroups, and nonzero coefficients certify critical orbits with at least those isotropy types.The index is read from the maximal orbit types among the nonzero coefficients.Source: §2 · p. 4
  • Orbit types classify orbits by conjugacy classes of isotropy subgroups, while the section introduces Euler and Burnside rings as the algebraic structures for equivariant degrees.The Euler-to-Burnside ring homomorphism provides a reduction used in computing the degrees.Source: §2 · pp. 4–6

3. Equivariant Gradient Degree on Orthogonal G-representations

For orthogonal G-representations, the paper constructs an Euler-ring-valued gradient degree and gives several ways to compute it. The methods cover recursive orbit-type calculations, isolated-orbit slices, and linearization through symmetric isomorphisms.

3. Equivariant Gradient Degree on Orthogonal G-representations

  • The section establishes equivariant gradient degree on orthogonal representations for admissible gradient pairs and describes special Ω-Morse representatives up to admissible homotopy.The construction uses orbit-type strata and their normal bundles.Source: §3 · p. 6, §3.1 · p. 6

3.1. Methodology based on recursive construction.

  • The recursive construction defines an Euler-ring-valued degree with additivity, homotopy invariance, and an existence property: a nonzero coefficient implies a zero with at least the associated isotropy type.Its computation can be reduced through the Euler-to-Burnside ring truncation to equivariant Brouwer degree calculations.Source: §3.1 · pp. 7–8

3.2. Degree on the Slice.

  • For a neighborhood of an isolated orbit, the degree computation reduces to the normal slice at the orbit point, using its isotropy group.The slice principle relates the degree on that slice to the degree near the full orbit.Source: §3.2 · p. 8

3.3. Methodology Through Symmetric Linear Isomorphisms and Basic Degree.

  • For a G-equivariant self-adjoint linear isomorphism, linearization decomposes the space into isotypic components and computes the degree from basic degrees associated with negative eigenspaces.The basic degrees are invertible elements of the Euler ring.Source: §3.3 · pp. 8–9

4. Equivariant Gradient Degree on G-Manifolds

The manifold framework addresses settings where the linear isotypic decomposition is unavailable. It computes the equivariant gradient degree over orbit-type strata, beginning with maximal types.

4.1. G-Equivariant Gradient Degree on Manifolds.

  • A nonzero orbit-type coefficient certifies a critical point with at least that isotropy, and the manifold degree is additive and homotopy invariant.Source: §4.1 · pp. 9–10

4. Equivariant Gradient Degree on G-Manifolds

  • General G-manifolds lack the isotypic decomposition used in the linear setting, so their degree is computed recursively over orbit-type strata.Source: §4 · p. 9, §4.1 · p. 10

4.1.1. Computations of the Gradient G-Equivariant Degree on G-Manifolds.

  • For each maximal orbit type, a normal replacement is confined to a small invariant tubular neighborhood and agrees with the original field on that stratum.This separates that orbit type's degree contribution from the residual degree.Source: §4.1.1 · p. 10
  • The recursive construction repeats across remaining orbit types, preserving the restricted gradient on each stratum as their contributions accumulate.Source: §4.1.1 · p. 11

5. Examples

The examples apply equivariant gradient-degree methods to compute symmetry indices for a softplus teacher–student MLP, a steerable equivariant network, and a Transformer. They illustrate linearization and slice-based computations, as well as a recursive computation for a degenerate critical point.

5.1. A Softplus Teacher–Student MLP with Quartic Regularization.

  • For the softplus MLP, the invariant critical point has Hessian eigenvalues 1.1472 (multiplicity 1), 0.9486 (3), −0.0452 (3), and −0.1128 (9).The example uses k = 4, β = 1, and λ = 0.01, while the text states the framework applies to arbitrary width and hyperparameters.Source: §5.1 · pp. 12–13
  • On Ω := B_R(0) \\ B_ε(θ_o), the equivariant degree of ∇θL equals its degree on B_R(0) minus its degree on B_ε(θ_o); θ_o is the only zero there, and ∇L is admissibly G-homotopic to its linearization.Source: §5.1 · p. 13

5.2. A Steerable ENN with O(3)×S4 Parameter Symmetry.

  • For the steerable ENN, the slice principle reduces the gradient-degree computation near the critical orbit to an S4-equivariant map on the slice.The Hessian eigenvalues, multiplicities, and associated eigenspaces are listed in Table 3.Source: §5.2 · pp. 15–17, Table 3 · p. 16

5.3. Transformer via the Recursive Formula.

  • For the Transformer, the recursive formula gives gradient degree (S3 × S3) − (D1 × S3) − (S3 × D1) at the degenerate critical point θ = 0.The computation combines subgroup-lattice, Weyl-group, and fixed-space Brouwer-degree information.Source: §5.3 · p. 18

Appendix A. Euler and Burnside Rings

The appendix defines the Euler ring through conjugacy classes and integer combinations, then describes how its multiplication and nilpotence properties support computations.

Appendix A. Euler and Burnside Rings

  • Euler-ring elements have unique finite integer expansions over conjugacy classes, and the class of G is the multiplicative identity.The ring product retains the components associated with zero-dimensional Weyl groups when its generators belong to Φ0(G).Source: Appendix A · pp. 20–21
  • Multiplication by an invertible Euler-ring element preserves a nonzero coefficient at the same conjugacy class.The proof uses the ordering of classes under multiplication and uniqueness of the basis expansion.Source: Appendix A · pp. 20–21
  • The Burnside ring multiplication coefficients are evaluated recursively, starting with larger conjugacy classes and using transporter-based quantities.The transporter is defined as NG(S,K) = {g ∈ G : gSg−1 ⊆ K}.Source: Appendix A · p. 21
  • A class with positive-dimensional Weyl group is nilpotent, and any finite integer combination of such classes is also nilpotent.Classes with zero-dimensional Weyl group instead have every positive power nonzero.Source: Appendix A · pp. 21–22

Appendix B. G-Equivariant Brouwer Degree

This appendix introduces equivariant Brouwer degree as a computational foundation for the paper’s gradient-related degree theories, including approaches based on regular normal maps and linear isomorphisms.

Appendix B. G-Equivariant Brouwer Degree

  • Equivariant Brouwer degree assigns a Burnside-ring element to an admissible equivariant pair.The appendix presents its construction and properties as the basis for later gradient-related degree methods.Source: Appendix B · p. 22
  • The computational exposition covers reduction to a subgroup or normal slice, regular normal maps, and linear G-isomorphisms with basic degrees.Source: Appendix B · p. 22
  • For an orbit-type stratum, the ambient representation splits into tangent and normal directions.The appendix defines normal maps by requiring identity behavior locally in directions normal to the stratum at each zero.Source: Appendix B · p. 23
  • A regular normal map is C1 and has regular fixed-point restrictions at zeros of each occurring isotropy type.The restriction to the fixed-point space must have zero as a regular value at each such zero.Source: Appendix B · p. 23

B.1. Recursive-Construction Methodology (Based on Definition and Fundamental Properties).

This subsection lays out the defining properties and computational consequences of equivariant Brouwer degree, then describes localizing degree calculations to a critical orbit’s stabilizer and slice.

B.1. Recursive-Construction Methodology (Based on Definition and Fundamental Properties).

  • Homotopy invariance, normalization, excision, product, and suspension properties provide the main tools for computing equivariant Brouwer degree.The degree is uniquely determined by three axioms, with further consequences used in computation.Source: §B.1 · pp. 23–24
  • A nonzero degree coefficient for a conjugacy class certifies a zero whose isotropy contains a subgroup of that class.The existence property is stated for the zero-dimensional Weyl-group classes in the degree expansion.Source: §B.1 · p. 23
  • Near an orbit G(θo), the degree calculation can be reduced to a normal-slice problem with symmetry restricted to the stabilizer Gθo.The slice is the orthogonal complement to the orbit tangent space, and the construction uses a tubular neighborhood.Source: §B.1 · p. 24
  • For sufficiently small ε, the slice boundary condition holds when the slice map has an isolated zero; if θo is fixed by G, the slice is all of V.Source: §B.1 · p. 24

B.2. Methodology Through Regular Normal Map.

This subsection replaces an admissible equivariant map with a nearby regular normal map, whose degree is equal by homotopy invariance and can be computed from isolated zeros.

B.2. Methodology Through Regular Normal Map.

  • A regular normal map can approximate an admissible map uniformly within any prescribed ε while remaining equivariant.The linear homotopy between the maps is admissible, so their equivariant degrees agree.Source: §B.2 · p. 25
  • Normality, fundamental domains, and regularity make zeros amenable to local degree calculations and summation.Normality gives canonical local structure, fundamental domains remove orbit redundancy, and regularity with the Inverse Function Theorem yields isolated zeros.Source: §B.2 · p. 25
  • For each zero-dimensional Weyl-group class, the restricted map’s zeros in the domain are isolated and represented by finitely many Weyl-group orbits.Source: §B.2 · p. 25

B.3. Methodology Through Linear Isomorphism and Reduction to Basic Degree.

Because recursive and regular-normal approaches can require difficult explicit constructions, this section uses linearization to reduce degree computation to that of a linear G-isomorphism. It develops basic degree using the G-isotypic structure of the representation.

B.3. Methodology Through Linear Isomorphism and Reduction to Basic Degree.

  • Linearization replaces the nonlinear map locally or globally by a linear approximation, reducing degree computation to that of a linear G-isomorphism.The alternatives discussed require computing deg(f_H, Ω_H) or constructing an explicit regular normal approximation and suitable fundamental domains.Source: §B.3 · p. 25
  • G-equivariance preserves each isotypic component and makes the generalized eigenspaces G-invariant, providing the decomposition used to define basic degree.The construction intersects each generalized eigenspace E(β) with V_i to obtain E_i(β).Source: §B.3 · p. 25
  • A basic degree is expanded in orbit types, with the coefficient of a maximal orbit type called its leading coefficient.The section also states that every basic degree is involutive in the Burnside ring.Source: §B.3 · p. 26

Appendix C. Amalgamated Groups

Appendix C represents subgroups of product groups using subgroup data and quotient identifications. The resulting fiber-product subgroup is called an amalgamated subgroup.

Appendix C. Amalgamated Groups

  • A subgroup of G1 × G2 is represented by subgroups of each factor and surjective homomorphisms from them onto a common quotient.The passage introduces this representation using Goursat’s lemma [26].Source: Appendix C · p. 26
  • The fiber-product construction is called an amalgamated subgroup, and its abbreviated notation presupposes a fixed identification of the two quotient groups.The kernels are K0 = ker φ and H0 = ker ψ; the quotient groups K/K0 and H/H0 are identified with S.Source: Appendix C · pp. 26–27

Related papers

Loading 2610.28864v1…