Source-linked AI summary
Information-Induced Training Geometry: Exact Reduction, Canonical Completion, and Structured Expressivity
Zavier Li
TL;DR
Training data reveals only a visible cotangent space, leaving full optimizer geometry underdetermined. The paper studies full-column-rank SPD compression under affine-invariant geometry and derives canonical completions and exact reduction. It shows that metric submetry characterizes universal radial-visible reduction, while the canonical completion preserves the reference wherever visibility imposes no change.
Problem
A visible positive target constrains a full cometric but leaves an entire fiber of ambient geometries undetermined.
Method
The paper analyzes full-column-rank linear channels on the SPD cone with the affine-invariant Riemannian metric, using submetry and coherent-section theory.
Results
The canonical completion is the least affine-invariant deformation of a declared reference compatible with the visible target and supports exact lossless reduction.
Takeaways & Limitations
Exact information reduction requires metric submetry, while meaningful completion is reference-dependent and preserves unconstrained directions.
Takeaways & Limitations
The completion does not claim that a first-order oracle uniquely determines the visible target or that the reference is reference-free.
Abstract
from arXiv · showhide
Training data constrains optimizer geometry through the covectors visible to a declared information channel. We study how such partial information determines a full positive cometric relative to a reference and which degrees of freedom remain unidentified. Our central result resolves full-column-rank positive-definite compression under affine-invariant Riemannian geometry. The compression map is a split-Hadamard metric submetry and admits an explicit unique completion that is the affine-invariant nearest full geometry realizing a visible target and yields exact full-to-visible variational reduction. When the channel moves, the completions form a gauge-invariant rank stratification of the positive-definite cone. Its closed-form pullback pair metric separates visible-metric motion from subspace rotation through a reference-mismatch weight, yields an explicit positive-semidefinite multi-direction Gram matrix, and exposes the precise singularity of reference-valued modes. The mechanism is explained by a metric theorem equating ball submetry, attained fiber distance, and lossless reduction of every monotone radial visible decision problem. A smooth split-Hadamard theorem supplies coherent information sheets, proximal commutation, and solution-wise gradient-flow lifting. The positive-definite realization also gives closed-form prior-data shrinkage. Diagonal and block optimizer families reduce to relative-interior conic image tests with valid facial certificates, while deterministic and finite-sample bounds quantify recovery of the visible geometry and its subspace. Together these results characterize exact reduction, reference-dependent completion, and structured expressivity for the stated finite-dimensional affine-invariant model.
1 Introduction
The paper studies how partially observed loss covectors determine full training cometrics, establishing canonical affine-invariant completion and exact visible reduction. It extends this framework to moving channels, abstract Hadamard mechanisms, structured expressivity, and recovery guarantees.
- Motivation: A declared visible target constrains only the cometric on the span of example loss differentials, leaving an entire full-cometric fiber undetermined.The annihilator of the visible cotangent space is invisible to every example loss at the parameter point.
- General mechanism: Metric submetries are characterized by attained fiber distance and universal radial-visible reduction, while complete horizontal leaves provide coherent isometric sections.These sections commute exactly with proximal maps and lift gradient-flow solutions on Hadamard manifolds.
- Exact SPD reduction and completion: Every full-column-rank SPD compression is a split-Hadamard submetry with a unique AIRM-nearest completion and exact full-to-visible decision reduction.The completion also supports closed-form prior–data shrinkage and a radius-conditioned minimax identity for invisible ambiguity.
- Moving-channel geometry: Moving-channel completions stratify the SPD cone by rank(P−P0), while their pair metric separates visible-metric change from subspace rotation through reference mismatch.Reference-valued modes have zero rotation cost and are precisely singular directions; polarization yields a positive-semidefinite Gram matrix with rank and kernel criteria.
- Scope: The theory begins after a positive visible target is declared and does not identify Fisher, Gauss–Newton, covariance, damping, or audit constructions with one another.The moving-reference formula is kinematic rather than an optimizer evolution law, and exact reduction does not prescribe the visible decision selected by an optimizer.
2 Related Work
The paper builds on Riemannian submetry, information geometry, SPD geometry, and structured optimization, but studies a distinct reference-dependent completion of compressed positive cometrics. It extends these foundations with metric-submetry reduction, moving-channel geometry, and implementability certificates.
- Riemannian submersions and submetries: Classical Riemannian submetries motivate the paper’s ball-surjectivity framework, which is extended with a variational characterization.The related geometric literature connects metric submetries to smooth Riemannian submersions.
- Information geometry and natural gradient: Unlike Fisher, natural-gradient, and sufficiency-based work, the paper studies finite-dimensional positive cometrics, the compression A^T P A, and reference-dependent AIRM completion.Prior work supplies information-geometric motivation, whereas this paper targets completion of the full invisible fiber.
- SPD geometry and matrix completion: Unlike sparsity-based matrix completion, the paper completes the entire invisible fiber by selecting the AIRM-nearest reference geometry and characterizing it as a horizontal metric-submetry section.Totally geodesic SPD submanifolds provide related geometric context, but the paper additionally resolves transverse affine-compression fibers and radial-visible reduction.
- Quotient geometries: The moving-channel construction differs from fixed-rank PSD quotient geometries by stratifying the displacement P − P0, allowing indefinite displacement and producing a mismatch-weighted line element with a reference-valued kernel.Its weight is R + R^-1 − 2I, arising from the pullback of ambient AIRM geometry.
- Structured preconditioning and recovery: Structured optimizer families use convex and semidefinite certificates to test whether diagonal, block, matrix, or tensor restrictions can realize a declared visible target.Subspace perturbation and covariance concentration tools further support recovery bounds separating visible SPD error from subspace rotation.
3 Metric Foundations of Exact Information Reduction
Metric submetries are exactly the observation maps that preserve every reference-radial visible decision problem without loss, while coherent sections provide canonical lifts and exact optimization dynamics. Surjectivity, exact distance preservation, and sheet coherence are distinct requirements.
- Maximal exact information reduction: Metric submetry is equivalent to attained exact fiber distances and lossless reduction of every nondecreasing radial visible-functional decision problem.It is stronger than a nonexpansive surjection because every visible displacement has an ambient lift with no excess distance.
- Maximal exact information reduction: The universal exact-reduction theorem is sharp: hard ball costs and singleton visible losses recover full ball-surjectivity, so the characterization cannot be enlarged.
- Coherent exact reduction: Under coherent section identities, every section is an isometric embedding exactly when reduced minimizers lift to full minimizers and the reduction equality holds.Whenever these equivalent conditions hold, the observation map is a metric submetry; unique nearest points additionally select the lift canonically.
- Coherent exact reduction: Coherent proximal iterations are exact lifts of visible iterations, with full proximal dynamics inheriting this exactness under the stated rigidity and existence assumptions.
- Three distinct requirements: Surjectivity supplies visible representatives, submetry supplies exact distances, and coherence supplies a reusable canonical sheet; none implies the others without additional assumptions.
4 Split-Hadamard Information Maps
Split-Hadamard maps provide a global, holonomy-free structure in which horizontal leaves are complete and totally geodesic, yielding rigid coherent information sheets. They are metric submetries with unique shortest completions, exact radial and proximal reduction, and solution-wise gradient-flow lifting.
- Definition and geometry: A split-Hadamard map requires integrable horizontal distributions whose maximal leaves are complete and totally geodesic, without requiring linear or compact fibers.This condition rules out holonomy between visible and invisible directions.
- Global sections: Each horizontal leaf maps globally and isometrically onto the visible manifold, with a unique horizontal, isometric, totally geodesic section through every point.Completeness and Hadamard uniqueness provide global extension and injectivity of horizontal lifts.
- Metric completion: The map is a metric submetry, and its section gives the unique shortest completion of each visible point.Equality in Riemannian-submersion length contraction forces the shortest ambient geodesic to be horizontal.
- Variational reduction: Every reduced minimizer lifts through the section, and under strictly increasing radial losses these lifts are exactly all finite-valued full minimizers.The result applies to the universal radial reduction, with strict monotonicity on the loss’s finite domain.
- Optimization dynamics: Moreau envelopes and proximal maps commute exactly, while visible gradient-flow solutions lift on the same existence interval and are unique when the visible gradient is locally Lipschitz.For proper lower-semicontinuous geodesically convex bounded-below ϕ, z = s_x ◦ y solves the lifted flow.
5 SPD Information Compression and Canonical Completion
For full-column-rank channels, SPD compression admits a unique reference-dependent canonical completion that is AIRM-nearest, geometrically exact, and supports lossless visible decision reduction. The theory also characterizes invisible update freedom, Gaussian-KL equivalence, and prior–data geodesic regularization while showing that reference-free completion is impossible.
- Reference dependence and ambiguity: No completion rule depending only on (W,R) can be invariant under all automorphisms fixing W, so canonical ambient completion necessarily requires additional structure such as P0.The associated local ambiguity model is centered at the reference-selected completion and is not a reference-free uncertainty set or minimax center.
- Complete SPD information-compression theorem: Theorem 5.2 establishes a unique canonical completion Ψ_P0,A(R), which is the AIRM-nearest full SPD realizing the visible target R.The completion is selected relative to P0; the visible target R is an input rather than something uniquely determined by a first-order oracle.
- Complete SPD information-compression theorem: Compression is split-Hadamard: Ψ_P0,A is an isometric totally geodesic embedding, and visible AIRM geodesics have unique horizontal ambient lifts.The construction follows from equality in the contraction of the whitened Frobenius norm under orthogonal compression.
- Complete SPD information-compression theorem: Every reduced minimizer lifts canonically, and with strictly increasing radial penalty every finite-valued full minimizer is canonical, yielding exact radial decision reduction.This applies to admissible channel/reference families, nondecreasing radial functions, and extended-real visible functionals.
- Visible update quotient and invisible drift: The visible update is fixed modulo ker A^⊤, while every invisible component in ker A^⊤ is feasible; the canonical representative has no invisible component.Thus the information channel identifies an update quotient rather than a unique full update.
- Gaussian KL and prior–data completion: The same completion uniquely minimizes Gaussian KL and supports prior–data geodesic regularization, with minimum value τ(1−τ)D² and perturbation bound τd_AI(R1,R2).The Gaussian interpretation retains the reference conditional law on invisible coordinates.
6 Gauge, Rank Strata, and Moving Information
Allowing the information channel to move yields a gauge-invariant completion bundle with complete rank stratification and an explicit quotient metric. The geometry separates visible-target motion from subspace rotation, while identifying singular reference-valued modes and covariant moving-reference behavior.
- Completion bundle and rank stratification: The completion parameterization is a diffeomorphism on the regular subset, with an intrinsic inverse up to GL(m), and its strata exhaust and close.Regularity fails when visible modes equal their reference values, making the total descriptor redundant.
- Completion-pair metric: The pullback affine-invariant metric separates visible-target changes from visible-subspace rotations, making these motion blocks orthogonal on the regular quotient.The metric becomes a genuine quotient metric on the regular subset after normalization and horizontal gauge fixing.
- Completion-pair metric: For any finite set of directions, the completion-feature Gram matrix is positive semidefinite, with rank equal to the span dimension of their completion features.This gives an explicit multi-directional description of the pair metric through a direct-sum Frobenius representation.
- Singularities and control: At singular descriptors, rotating a reference-valued visible mode is unidentifiable from the completed matrix, although the ambient SPD manifold and AIRM remain smooth.Quantitative control of subspace motion is available when spec(R) stays a positive distance from 1 and R has bounded eigenvalues.
- Reduced value versus completion motion: The reduced completion value contains no independent information-subspace coordinate, whereas the pair metric measures motion of the explicit canonical minimizer.The completion kernel remains analytic for every positive-definite R, including repeated eigenvalues, and the same identity has a gauge-covariant moving-reference form.
Under G′
Under G′, the fixed-reference, fixed-rank geometry has a frame-independent kinematic energy decomposition. Continuous rank loss is constrained to visible modes approaching the reference geometry, while dynamical learning or control requires an additional evolution rule.
- Under G′: The fixed-reference, fixed-rank curve admits an exact energy decomposition, and the resulting expression is independent of the moving frame.The line element is a kinematic identity on the completion bundle.
- Under G′: Continuous rank loss can occur only through visible modes approaching the reference geometry.If the active rank drops by at least r, then at least r singular values of R_t−I converge to zero.
- Under G′: The geometric identity alone does not define learning or control dynamics for (P_0, A, R).An additional evolution rule is required for these variables.
7 Structured Expressivity Certificates
Structured optimizer families are expressive exactly when the visible target lies in the appropriate relative-interior cone image. Diagonal and block families admit attained minima and facial certificates that distinguish strict infeasibility from boundary targets requiring semidefinite structure.
- Diagonal expressivity: Diagonal structured complexity is finite exactly when the visible target has a strictly positive representation R = Σ_j p_jG_j; if the G_j are independent, that representation is unique.When finite, the infimum is attained.
- Diagonal expressivity: Diagonal targets have two failure certificates: strict separation proves infeasibility, while facial exposure identifies boundary targets whose representation requires some diagonal weights to vanish.The second certificate is nontrivial on the span of the image cone and exposes the proper face containing the target.
- Diagonal example: An end-to-end example shows a visible request with a unique nearest full-SPD realization, but only a semidefinite diagonal realization, certified by a separating functional.All feasible full geometries share the same visible update, while invisible components vary within ker A⊤, demonstrating exact reduction and invisible drift.
- Block expressivity: For block cometrics, finiteness is equivalent to R lying in the relative interior of the closed block cone, equivalently to realization by positive-definite blocks; the infimum is attained.Boundary targets in the cone but outside its relative interior admit semidefinite realizations yet are unreachable by strictly positive optimizer blocks.
- Certificates and scope: The diagonal and block certificates reduce to finite-dimensional linear or semidefinite feasibility systems, while Kronecker, low-rank, and tensor families require separate image geometry.Algorithmic complexity and numerical tolerances are outside the characterization.
8 Recovery of Visible and Completed Geometry
Recovery separates visible-geometry error from mismatch-weighted subspace rotation, with the latter vanishing for reference-valued modes. Known-subspace recovery preserves intrinsic dimension, while unknown-subspace recovery targets a spectral eigenspace and requires identifiability conditions.
- Joint perturbation: The joint perturbation decomposes completed-geometry error into visible error plus mismatch-weighted subspace error, which vanishes for reference-valued modes.The fixed-channel section transfers visible SPD error isometrically, while estimating the information subspace adds a Grassmann term.
- Known visible subspace: Known-subspace finite-sample recovery uses a self-contained net–Hoeffding bound whose dimension dependence is intrinsic to the visible space.The bound is deliberately conservative; matrix Chernoff or Bernstein methods can improve dimension or effective-rank dependence under stronger assumptions, without claiming an optimal rate.
- Unknown visible subspace: Unknown-subspace recovery bounds completed-geometry error by aligned visible second-moment error plus mismatch-weighted eigenspace rotation.The target is the leading m-dimensional eigenspace of the population second moment, matching population covector support under a rank-m model but not necessarily a finite observed span.
- Unknown visible subspace: The self-contained unknown-subspace sample bound pays an ambient-dimension cost and requires an eigengap or another identifiable subspace condition.Matrix concentration or effective-rank assumptions may improve the ambient factor; the second moment is uncentered and equals covariance only for zero mean.
9 Discussion
The discussion establishes a hierarchy linking submetry, coherent sections, split-Hadamard geometry, and closed-form SPD compression, while showing that completion and subspace motion depend on a declared reference. It also identifies structured and statistical certificates, limits of applicability, and downstream uses beyond the paper’s assumptions.
- What the theory establishes: Submetry is necessary and sufficient for universal radial-visible reduction, while coherent sections and split-Hadamard geometry provide reusable information sheets and global horizontal submanifolds.SPD compression realizes this hierarchy in closed form and adds a rank quotient, completion-pair metric, and structured certificates.
- Visibility and reference: A visible target leaves an infinite-diameter invisible fiber, so the canonical completion is meaningful only relative to P0 and preserves unconstrained directions with minimum AIRM deformation.Even the radius-bounded minimax problem has risk equal to the radius.
- Moving information: The mismatch weight Ξ(R) makes subspace rotation costly only as visible geometry separates from the reference, so continuous rank changes pass through reference-valued modes.These are kinematic statements; selecting trajectories requires an additional modeling, optimization, or control principle.
- Structured optimizers: Diagonal and block restrictions distinguish strict representability, degenerate boundary representability, and infeasibility after closure through relative-interior tests and conic certificates.A single residual cannot distinguish these three outcomes.
- Statistical interpretation: Known-subspace recovery is m-dimensional, whereas subspace estimation adds an ambient second-moment problem and eigengap; principal-angle error then converts into weighted completed AIRM error.The resulting bound separates uncertainty in R from uncertainty in the information channel and uses bounded i.i.d. observations with net–Hoeffding arguments.
- Boundaries and downstream use: Exactness does not make arbitrary observation maps submetries, and the SPD pair metric is specific to full-column-rank linear compression; other models and structured families require separate verification.The paper provides a foundation for later metric-evolution, constrained-path, and intervention-based optimizer audits, while leaving dynamical or causal assumptions outside scope.
10 Reproducibility and Ethics Statements · A Proofs for Metric and Split-Hadamard Reduction · B Proof of the Split-Hadamard Theorem
The appendices establish the metric and split-Hadamard reductions through complete proofs, while the reproducibility and ethics statements delimit the theory’s scope and interpretive risks. They also prove global isometry, unique shortest lifts, proximal commutation, and solution-wise gradient-flow lifting under the stated completeness and Hadamard assumptions.
- 10 Reproducibility and Ethics Statements: The paper is fully reproducible as a theory paper: claims are formalized and proved, finite-sample assumptions are explicit, and no empirical optimizer-performance claim is made.The finite-sample results specify boundedness, eigengap, confidence, and dimension dependence for each conclusion.
- 10 Reproducibility and Ethics Statements: The paper introduces no datasets, human-subject experiments, or deployed systems, but warns that visible geometric certificates do not explain motion from memory, noise, higher-order oracles, or discretization.The theory constrains only what a declared information channel determines.
- A Proofs for Metric and Split-Hadamard Reduction: The metric theorem proves that ball submetry, attained fiber distance, and universal monotone radial reduction are equivalent, with attainment supplying exact equality and canonical minimizer lifting.The proof derives Lipschitz contraction, reverse ball inclusion, and equality through fiber minimizers; the converse uses singleton indicators and ρ(t)=t.
- A Proofs for Metric and Split-Hadamard Reduction: Under coherent isometric sections, the reduction theorem yields a submetry, exact proximal commutation, and canonical minimizer lifting by induction in complete CAT(0) spaces.Applying radial reduction to squared distance proves the proximal identity, while coherence preserves the section across iterations.
- B Proof of the Split-Hadamard Theorem: A complete local Riemannian isometry onto a Hadamard manifold is global: geodesic lifting extends by completeness, and simple connectivity reduces the covering to one sheet.The resulting bijective local isometry is a global Riemannian isometry.
- B Proof of the Split-Hadamard Theorem: Applying that lemma to complete horizontal leaves proves that each leaf maps globally and isometrically to the Hadamard base, with intrinsic and ambient distances agreeing.Total geodesy and Hadamard uniqueness make the leaf geodesically convex, and horizontal sections are inverses of the global isometry.
- B Proof of the Split-Hadamard Theorem: The split-Hadamard proof establishes unique shortest lifts, the ball identity, radial and proximal reduction, and rigidity because equality forces minimizing geodesics to remain horizontal.The horizontal inverse section also lifts visible gradient-flow solutions uniquely through the chain rule and standard ODE uniqueness.
C Proofs for SPD Compression and Completion … F Proofs for Deterministic and Statistical Recovery
The proofs establish canonical, uniquely nearest SPD completions and exact affine-invariant reduction, then characterize rank-stratified geometry, structured expressivity, and deterministic/statistical recovery guarantees. Together, they show when visible information determines geometry, how channel motion and structure affect it, and how estimation errors propagate.
- C Proofs for SPD Compression and Completion: No invariant completion selector exists without a reference or equivalent off-visible structure, and canonical completion uniquely minimizes the forward Gaussian KL objective.The scaling argument rules out reference-free invariant selection; blockwise KL decomposition minimizes the cross block at zero and the invisible block at identity.
- C Proofs for SPD Compression and Completion: The compression map is split-Hadamard: canonical completions form complete, totally geodesic horizontal sections, yielding Riemannian submersion and horizontal-lift properties.The proof combines global contraction with equal-distance normalized lifts and identifies the full horizontal space through surjectivity and dimension counting.
- C Proofs for SPD Compression and Completion: For every visible target, the canonical completion is the unique affine-invariant nearest feasible full geometry and supports exact reduction of monotone radial decision problems.Strict monotonicity makes replacing any noncanonical feasible point by its canonical completion strictly improve the objective, while every reduced minimizer lifts canonically.
- D Proofs for Rank-Stratified Geometry: Rank-m SPD strata are gauge-invariant smooth manifolds parameterized bijectively by an m-dimensional subspace and a compressed congruence class, with lower-rank strata forming their closures.The intrinsic subspace is range(P−P0), frame changes act by GL(m) congruence, and spectral separation gives a smooth local inverse.
- E Proofs for Structured Expressivity: Diagonal and block optimizer families are feasible exactly through relative-interior conic image tests, with attainment, uniqueness where generators are independent, and separating-matrix facial certificates.The block image cone is closed, consists of PSD matrices supported on prescribed ranges, and admits dual certificates via blockwise PSD inequalities.
- F Proofs for Deterministic and Statistical Recovery: Visible-metric and channel errors combine through the explicit pair-distance bound, with the smaller reference-mismatch weight obtained by choosing the better order of rotation and metric change.One path first rotates the channel while holding the visible geometry fixed; the reversed path rotates after changing the visible geometry, yielding weights χ(S) and χ(R).
- F Proofs for Deterministic and Statistical Recovery: With high probability, covariance concentration gives dAI(R, R̂) ≤ √m ε, while spectral-gap perturbation controls subspace error and transfers both errors to the completed estimator.The proof uses net-based Hoeffding bounds, Loewner sandwiches, Weyl eigenvalue control, and the deterministic recovery theorem.