Source-linked AI summary
Riemannian Metric and Geometric Mean for Positive Semidefinite Matrices of Fixed Rank
Silvere Bonnabel, Rodolphe Sepulchre
TL;DR
The paper addresses the lack of a natural metric and geometric mean for fixed-rank positive semidefinite matrices. It constructs a quotient Riemannian geometry with computable closeness and a rank-preserving mean, while noting that the proposed closeness curve and mean are not always exact Riemannian geodesic counterparts.
Problem
The natural metric for positive definite matrices applies only to full-rank matrices, leaving fixed-rank positive semidefinite matrices without the same geometric framework.
Method
The paper uses the quotient geometry S+(p, n) = (Vn,p × Pp)/O(p), decomposing geometry into Grassman and lower-dimensional cone components and computing closeness through SVD-based approximating curves.
Results
The resulting metric is invariant under orthogonal transformations, scalings, and pseudoinversion, the space is geodesically complete, and the induced geometric mean preserves rank.
Takeaways & Limitations
The framework provides computable geometric tools for fixed-rank matrices and may support low-rank computations in applications such as MRI tensor processing, radar processing, kernel methods, and bioinformatics.
Takeaways & Limitations
The approximating curve is not necessarily a geodesic, its length does not satisfy the triangle inequality, and the associated mean need not equal the Riemannian mean.
Abstract
from arXiv · showhide
This paper introduces a new metric and mean on the set of positive semidefinite matrices of fixed-rank. The proposed metric is derived from a well-chosen Riemannian quotient geometry that generalizes the reductive geometry of the positive cone and the associated natural metric. The resulting Riemannian space has strong geometrical properties: it is geodesically complete, and the metric is invariant with respect to all transformations that preserve angles (orthogonal transformations, scalings, and pseudoinversion). A meaningful approximation of the associated Riemannian distance is proposed, that can be efficiently numerically computed via a simple algorithm based on SVD. The induced mean preserves the rank, possesses the most desirable characteristics of a geometric mean, and is easy to compute.
1 Introduction
The paper extends the natural geometry of positive definite matrices to fixed-rank positive semidefinite matrices, addressing the full-rank restriction while preserving key invariances and computational structure. The resulting geometry supports efficient closeness and mean computations for low-rank matrices.
- Computational motivation: Low-rank factorization reduces the cost of typical matrix operations, whose full-size positive definite implementations require O(n3) operations.When p is moderate, the resulting algorithmic complexity grows linearly with problem size.
- Motivation and contribution: The paper extends the natural metric from full-rank positive definite matrices to fixed-rank positive semidefinite matrices S+(p, n), p < n.The extension uses a quotient geometry that preserves many desirable properties of the positive cone.
- Motivation and contribution: The proposed metric is invariant under orthogonal transformations, scalings, and pseudoinversion, and makes S+(p, n) geodesically complete.These transformations preserve angles and are identified as the key invariance class for the new geometry.
- Applications: The framework is intended to generalize positive-definite algorithms to the semidefinite case, including matrix-nearness problems based on Bregman divergences.The paper specifically identifies projections onto important matrix sets as an open application area.
- Method: A meaningful approximation to the Riemannian distance is computed by an SVD-based algorithm with numerical complexity O(np2).The approximation uses special curves because explicit geodesics are out of reach.
- Method: The quotient geometry decomposes the computation into a Riemannian problem on the lower-dimensional cone Pp and a Grassman-manifold problem on p-dimensional subspaces of Rn.This decomposition separates matrix-shape variation within a subspace from variation of the subspace itself.
2 Riemannian distances and geometric means on the symmetric cone
The symmetric positive definite cone has a quotient-based natural metric that is invariant under general linear congruence and yields Riemannian geodesics, distances, and geometric means. Its geometry is preferable to the flat metric for several structural and computational reasons.
- Quotient geometry: The positive definite cone Pn is represented through a quotient geometry associated with the factorization A = ZZT and the action of O(n).This structure realizes Pn as a reductive homogeneous space Gl(n)/O(n).
- Natural metric: The natural metric is invariant under the congruence action of Gl(n), allowing points and tangent vectors to be transported from the identity.At the identity, the metric is defined by the usual scalar product on symmetric matrices.
- Geodesics and distance: The exponential map at the identity is the matrix exponential, and the geodesic length satisfies d(exp X, I) = ∥X∥F.Metric invariance extends the geodesic characterization from the identity to arbitrary points.
- Geodesics and distance: The Riemannian distance is expressed through generalized eigenvalues and is invariant under simultaneous matrix inversion.The invariance follows because log2(λk) is unchanged when λk is replaced by λk^-1.
- Geometric mean and completeness: The natural metric makes the positive definite cone geodesically complete, unlike the flat metric, and its associated mean has desirable geometric properties.The natural metric also coincides with the metric induced by the logarithmically homogeneous self-concordant barrier −log det A.
3 Extending the metric: A geometric insight in the plane
The rank-one planar case shows how a metric can combine subspace direction and scale while remaining invariant to rotations and scalings. It also demonstrates why full general-linear invariance is impossible for a continuous distance on rank-deficient matrices.
- Limits of invariance: No continuous distance can remain invariant under every general-linear change of basis on this rank-deficient space.As matrices approach a common singular limit, continuity would force distinct matrices to have zero distance.
- Planar geometry: For rank-one 2 × 2 matrices, the space is represented by a positive scale and a projective line, corresponding to R+* × RP1.The sign ambiguity of a generating vector identifies θ with θ + jπ.
- Planar geometry: The proposed planar metric combines angular separation and logarithmic scale separation: dS+(1,2)(A, B) = |θ2 − θ1|2 + k|log(r1/r2)|2, k > 0.The angular term handles orthogonal transformations, while the logarithmic term handles scaling.
- Symmetry: Every invariant distance under the subgroup of positive scalings and rotations depends on the scalar invariants r2/r1 and θ2 − θ1.Symmetry and distance requirements constrain it to a positive monotone function of absolute logarithmic scale and angular differences.
- Physical interpretation: The distance is physically meaningful for rank-deficient covariance matrices because it is independent of measurement units and coordinate-frame orientation.The example models variables whose realizations lie on lines and disperse in different directions.
- Noise example: In the noisy covariance example, the full-rank geometric mean collapses toward the null matrix as noise vanishes, whereas the rank-one midpoint remains a nontrivial covariance.The rank-one midpoint has eigenvalues (0, 2) and is hardly affected by noise.
4 A new Riemannian metric on the symmetric semidefinite cone
The paper represents fixed-rank positive semidefinite matrices as a quotient manifold and equips this space with a Riemannian metric combining Grassmannian and positive-cone geometry. The construction yields a Riemannian manifold with invariance under orthogonal transformations, scaling, and pseudoinversion, while retaining a non-orthogonal horizontal–vertical decomposition.
- Quotient geometry: Fixed-rank positive semidefinite matrices admit the quotient representation S+(p, n) ∼= (Vn,p × Pp)/O(p), with equivalent representatives related by orthogonal changes of basis.The representation A = UR2UT is unchanged under (U, R2) ≡ (UO, OTR2O).
- Metric construction: The proposed metric combines the natural metric on Pp with the standard metric on the Grassmann manifold.This separates variation of the rank-p subspace from variation within the positive-definite factor.
- Riemannian structure: The quotient construction makes S+(p, n) a Riemannian manifold with a horizontal space representing tangent directions modulo fiber motions.The horizontal space is complementary to the vertical space associated with the O(p) fibers.
- Invariance: The metric is invariant under orthogonal transformations, scalings, and pseudoinversion.Orthogonal transformations and scaling act separately on the Grassmannian and positive-cone components, while pseudoinversion follows from inversion invariance on Pp.
- Geometric qualification: The horizontal and vertical spaces are complementary but not orthogonal, although they tend toward orthogonality as k →0.This non-orthogonality is unusual and affects subsequent results.
- Metric family: The construction extends beyond the natural cone metric because the theorem also holds for any GL(n)-invariant metric on the positive cone.A family of such invariant metrics can likewise be extended to S+(p, n).
5 Horizontal geodesics in the structure space approximate geodesics in S+(p, n)
The paper constructs horizontal geodesics in the quotient structure space to define curves connecting fixed-rank positive semidefinite matrices, then uses their lengths as efficient approximations to Riemannian geodesic distances. These curves preserve key invariances and yield a geodesically complete manifold, but their lengths are not generally true distances.
- Curve construction: SVD-based representatives of two matrices define a curve in S+(p, n) connecting the matrices through a horizontal geodesic in the structure space.The construction uses principal vectors and angles to align the associated subspaces before connecting the positive-definite factors.
- Curve construction: The constructed curve remains in S+(p, n), has a horizontal-geodesic lift, and has a computable squared total length in the proposed Riemannian manifold.Its endpoints are A and B, and the lifted curve is geodesic in Vn,p × Pp.
- Invariance and computation: The measure is invariant under orthogonal transformations, scalings, and pseudoinversion, and reduces to natural-cone or Grassmann distance in the corresponding special cases.These transformations preserve angles, while the special-case reductions occur when supports coincide or matrices are rank-p projectors.
- Geometric interpretation: The curve length combines Grassmann distance between support subspaces with natural-cone distance between ellipsoids sharing a subspace.It therefore generalizes the positive-definite natural distance, while not necessarily being the Riemannian distance on S+(p, n).
- Geometric interpretation: The closeness measure is symmetric and vanishes only for identical matrices, but it can violate the triangle inequality because the constructed curve is not always geodesic.A three-segment alternative through intermediate matrices can be shorter than the direct constructed curve.
- Invariance and computation: The computation costs O(np^2) for principal-angle quantities plus O(p^3) for a generalized eigenvalue problem, making it efficient when p ≪ n.The constructed curves are always at least as long as geodesics and approximate them well when the Grassmann contribution is highly penalized; the manifold is geodesically complete.
6 Geometric mean in S+(p, n)
The paper defines a rank-preserving geometric mean for fixed-rank positive semidefinite matrices from a closeness measure, and shows that it retains key geometric-mean properties while remaining computationally tractable.
- The closeness measure provides a direct formula for the halfway matrix defining the proposed geometric mean.
- The mean is well-defined generically when the largest principal angle is not π/2, but midpoint uniqueness fails at orthogonal principal directions.
- The proposed mean differs from the Ando mean for fixed-rank matrices and is rank-preserving, whereas the Ando mean can have rank bounded by the intersection dimension of the ranges.
- The mean satisfies permutation invariance, monotonicity, congruence-related invariance, and self-duality under pseudoinversion.
- For rank-p projectors, the proposed mean agrees with the Riemannian mean of their ranges on the Grassmann manifold and with the fixed-rank Riemannian mean.
- When matrices share the same range, the geometric mean need not equal the Riemannian mean, although it approximates it well when the metric-weight parameter k is small.
7 Conclusion
The paper extends natural Riemannian geometry from the positive cone to fixed-rank positive semidefinite matrices through quotient geometry. It obtains a decoupled, computable closeness measure and a rank-preserving geometric mean, with potential applications to low-rank matrix computations.
- The quotient geometry S+(p, n) = (Vn,p × Pp)/O(p) yields a natural metric whose contributions decouple into Grassmann and positive-cone components.
- The resulting closeness measure provides a computable Riemannian distance and a natural geometric mean that preserves matrix rank.
- The computational tools may support low-rank approximations of large-scale positive definite matrices in areas including MRI, radar processing, kernel methods, machine learning, and bioinformatics.
Appendix
The appendix develops the quotient-geometric construction and establishes the metric, distance, lifting, invariance, uniqueness, and completeness properties used by the paper.
- The positive cone is represented as Gl(n)/O(n), with symmetric and skew-symmetric Lie-algebra components providing the reductive decomposition.
- Curves in the quotient are represented by trajectories in Gl(n) modulo right multiplication by orthogonal matrices.
- The chosen curves are horizontal, so their structure-space lengths induce the corresponding quotient-space distance.
- SVD principal vectors are non-unique under repeated singular values but remain related by joint block-orthogonal transformations.
- The quotient distance is related to horizontal lifts in the structure space, with Grassmann and positive-cone distance contributions appearing in the construction.
- The fixed-rank manifold is geodesically complete because the structure space is complete and horizontal lifts can be extended globally.