Shape Operator PCA: Curvature-Aware Projections for Geometric Machine Learning

Alexandre L. M. Levada

Source-linked AI summary

Shape Operator PCA: Curvature-Aware Projections for Geometric Machine Learning

Alexandre L. M. Levada

arXiv:2608.15313v1Aug 15, 2026cs.LGcs.AIcs.CVstat.ML

TL;DR

Variance-based PCA can miss intrinsic manifold geometry relevant to clustering. SHOPCA incorporates curvature through the mean shape operator and selects regularization unsupervised, achieving average gains of 212% in ARI, 73% in FM, and 140% in V-measure over PCA.

Problem
Variance captures data spread but can miss intrinsic manifold curvature, motivating geometry-aware metric learning and dimensionality reduction.
Method
SHOPCA regularizes PCA covariance with a mean shape operator and selects its geometry-variance trade-off using a label-free spectral eigengap criterion.
Results
Across 30 real-world datasets, SHOPCA achieved average relative gains of 212% in ARI, 73% in FM, and 140% in V-measure over standard PCA.
Takeaways & Limitations
SHOPCA offers a linear, closed-form representation-learning pipeline whose projection and regularization selection require no class labels.
Takeaways & Limitations
High-dimensional raw data incur shape-operator estimation cost, and initial linear compression can discard informative structure; the formal treatment assumes hypersurfaces.

Abstract

from arXiv · show

In this paper, we propose SHOPCA (Shape Operator-based Principal Component Analysis), a novel method for unsupervised metric learning and dimensionality reduction that incorporates differential geometric information into the covariance structure of classical PCA. SHOPCA regularizes the global covariance matrix using the mean shape operator, defined as the average of the absolute local shape operators estimated from the data manifold, steering principal components toward directions of both maximum variance and informative curvature. A single trace-normalized mixing coefficient $α$ controls the regularization, recovering standard PCA at $α= 0$ and a curvature-driven embedding as $α\to \infty$. We further introduce a fully unsupervised criterion for selecting $α$ based on the spectral eigengap of the regularized covariance matrix, maximizing the relative separation between the top-$d$ and remaining eigenvalues without using class labels. We evaluate SHOPCA on more than 50 real-world benchmark datasets, comparing it with PCA, ISOMAP, and UMAP using Adjusted Rand Index (ARI), Normalized Mutual Information (NMI), Fowlkes-Mallows index (FM), and V-measure. Results show that SHOPCA consistently improves clustering quality over PCA across a broad range of datasets and surpasses UMAP on small-sample settings, where iterative neighborhood-based manifold estimation can degrade. SHOPCA is computationally tractable, parameter-efficient, and applicable to domains requiring fully unsupervised, geometry-aware dimensionality reduction.

1 Introduction

The paper motivates geometry-aware dimensionality reduction by arguing that projection and metric should be co-designed from intrinsic geometry, then introduces SHOPCA as a linear, closed-form bridge between PCA and differential geometry. SHOPCA regularizes PCA with curvature information while avoiding iterative optimization and explicit manifold parametrization.

Motivation: Similarity measurement underlies nearest-neighbor classification, hierarchical clustering, and spectral methods, making the input-space distance metric a foundational design choice.
Motivation: Kernel-PCA interpretations of spectral dimensionality reduction support co-designing the projection and metric, motivating geometry-aware regularization.
Problem: The paper identifies computational heaviness and poor suitability for small-sample, high-dimensional regimes as practical limitations of typical GML-inspired approaches.
Contribution: SHOPCA enriches PCA’s global covariance with data-manifold curvature through the mean shape operator in a single linear, closed-form procedure for unsupervised metric learning and dimensionality reduction.
Contribution: At α = 0, SHOPCA exactly recovers standard PCA, while α →∞ yields a purely curvature-driven embedding through trace-normalized regularization.
Contribution: SHOPCA requires no iterative optimization, explicit manifold parametrization, or neighborhood-graph construction beyond local k-NN neighborhoods used for curvature estimation.

2 Differential Geometry Basics

This section establishes the geometric foundations of SHOPCA by contrasting covariance-based PCA with manifold curvature and defining the operators used to capture local geometry. It connects tangent spaces, fundamental forms, and the shape operator to local covariance-based estimation.

Classical PCA captures data spread through covariance but is blind to manifold bending, even though high-curvature directions may reveal class boundaries and clusters.
A smooth m-dimensional manifold is locally Euclidean and embedded in R^d, typically with intrinsic dimension m ≪ d.
The tangent space T_xM provides the best first-order linear approximation of M, estimated locally from leading eigenvectors of k-neighbor sample covariance matrices.
The first fundamental form describes intrinsic metric properties, while the second fundamental form measures how the manifold curves away from its tangent space in the ambient space.
The shape operator is a self-adjoint tangent-space map whose eigenvalues are principal curvatures and whose eigenvectors identify directions of maximal or minimal bending.It relates the first and second fundamental forms through II_x(v,w) = I_x(S_x(v),w).
SHOPCA estimates local geometry from k-nearest-neighbor covariance matrices, treating the inverse covariance as an approximation to the metric tensor.The local neighborhood supplies a sample of the data manifold, and g_i ≈ C_i^-1 approximates the pullback metric induced by the embedding.

3 Shape Operator Based PCA

SHOPCA augments PCA’s covariance structure with a curvature-derived mean shape operator, producing an unsupervised projection that reflects both variance and intrinsic geometry. Its mixing parameter is selected spectrally, with standard PCA and curvature-driven embeddings recovered at the limiting regimes.

Curvature-regularized PCA: SHOPCA augments the global covariance matrix with a curvature-derived term so principal components capture statistical spread and intrinsic geometric structure.The extension requires no class labels, explicit manifold parametrization, or nonlinear optimization.
Limiting regimes: λ = 0 exactly recovers standard PCA, whereas λ →∞ yields Shape Operator Eigenmaps, a purely curvature-driven spectral embedding.In the latter regime, the variance term becomes negligible and components are determined entirely by the mean shape operator.
Unsupervised parameter selection: The optimal λ is selected without labels by maximizing the relative eigengap between the m-th and (m+1)-th eigenvalues of the regularized covariance.This replaces label-dependent cross-validated grid search with a purely spectral, geometric criterion.
Unsupervised parameter selection: The candidate grid is centered at the scale equating curvature and covariance traces, with dimensionless multipliers such as c_j ∈ [0.1, 100].This makes c_j = 1 represent a comparable variance/curvature balance across datasets while leaving the model formulation unchanged.
Computational cost: O(Jd^3) is the added cost of evaluating the eigengap over J candidates, negligible relative to estimating the mean shape operator.A single candidate requires forming the regularized covariance and computing its eigenvalues.

4 Theoretical Justification for Regularization

This section places SHOPCA within covariance shrinkage theory, establishes local spectral guarantees and aggregation limitations, and formalizes its variance–curvature and metric-learning interpretations. It also bounds eigenvalue movement from PCA as regularization increases.

Covariance shrinkage: SHOPCA is a covariance shrinkage estimator whose curvature-derived target trades bias for variance while favoring directions informative about class structure.This contrasts with generic shrinkage targets such as isotropy and explains the method’s small-sample robustness.
Spectral guarantees: Each discrete local shape operator has real, non-positive eigenvalues because it is similar to a symmetric positive-semidefinite matrix with a sign reversal.Thus every local principal-curvature estimate is real, even though the aggregate operator does not inherit the same guarantee.
Spectral limitations: The aggregate shape operator and regularized covariance generally lack a formal real-spectrum guarantee because local operators use different point-specific similarity transforms.Their eigenvalues are observed to be numerically real, with approximate preservation closest when local metrics vary slowly across points.
Variational interpretation: The stationary points of the Rayleigh quotient for the regularized covariance are exactly the eigenvectors of its symmetric part, giving a variance–symmetrized-curvature interpretation.The algorithm instead diagonalizes the potentially non-symmetric operator; the interpretations coincide when ¯S_skew = 0 and are close when ||¯S_skew||2 ≪ ||¯S_sym||2.
Perturbation analysis: For every regularized eigenvalue, Bauer–Fike bounds its distance to some PCA eigenvalue by λ||¯S||2, while eigenvalue ordering may still change discontinuously with λ.The bound quantifies spectral displacement but does not preserve the original eigenvalue correspondence.
Projection–metric co-design: When invertible, Σ_curv(λ) jointly defines SHOPCA’s projection and Mahalanobis-type metric, recovering the classical PCA-induced metric exactly at λ = 0.The leading eigenvectors and the metric induced by the same matrix are therefore one shared, unsupervised object.

5 Computational Experiments and Results

Across 30 datasets, SHOPCA consistently outperforms PCA on clustering metrics, with especially large gains when PCA or ISOMAP fails. Against ISOMAP, SHOPCA achieves higher aggregate scores, frequent metric-wide wins, and strong performance in sparse-sample settings.

SHOPCA versus PCA: SHOPCA outperforms PCA on all 30 datasets across ARI, FM, and V-measure.Mean scores are 0.3259, 0.4313, and 0.5169 versus PCA’s 0.1044, 0.2496, and 0.2153, respectively.
SHOPCA versus PCA: 212%, 73%, and 140% are SHOPCA’s relative mean gains over PCA in ARI, FM, and V-measure, respectively.Median scores also favor SHOPCA: ARI 0.3084 vs. 0.0384, FM 0.4294 vs. 0.2026, and VM 0.5166 vs. 0.1244.
SHOPCA versus PCA: 18-fold and 29-fold improvements on SATIMAGE, plus a greater-than-60× ARI gain on INDIAN_PINES, show SHOPCA rescuing near-chance PCA representations.On SATIMAGE, SHOPCA raises ARI from 0.0057 to 0.1012 and V-measure from 0.0092 to 0.2627; INDIAN_PINES ARI rises from 0.0061 to 0.3821.
Regularization selection: SHOPCA’s optimal λ∗ ranges from 8.35 to 345.71, and no dataset selects λ∗= 0.The selected regularization strength is therefore data-dependent, while purely variance-driven PCA is never optimal within the proposed framework.
SHOPCA versus ISOMAP: 53%, 12%, and 31% are SHOPCA’s relative mean gains over ISOMAP in ARI, FM, and VM, while SHOPCA leads all three metrics on 16 of 25 datasets.Aggregate means are ARI 0.3536 vs. 0.2314, FM 0.5765 vs. 0.5132, and VM 0.4765 vs. 0.3635.
SHOPCA versus ISOMAP: SHOPCA rescues failures of ISOMAP, raising ARI from 0.0658 to 0.1648 on ENGINE1 and from 0.0027 to 0.5717 on BREAST.It also improves IRISH from 0.0004 to 0.1439 and PRNN_SYNTH from 0.1275 to 0.4178, while improving COIL-20 and UMIST_FACES_CROPPED ARI by 16% and 57%.

6 Conclusions

SHOPCA extends PCA with a curvature-based mean shape-operator regularizer and label-free selection, while empirical results support gains over standard PCA. The conclusions also identify computational and modeling limitations and propose unsupervised tuning, symmetrized formulations, and adaptive aggregation as future directions.

Contributions: SHOPCA is a linear, closed-form PCA generalization that regularizes covariance with a locally estimated mean shape operator.The mean shape operator is aggregated from k-nearest-neighbor neighborhoods into a single d × d matrix.
Empirical evidence: 212% average relative ARI gain, 73% FM gain, and 140% V-measure gain over PCA were reported across 30 real-world datasets.No dataset selected λ∗ = 0, supporting the contribution of curvature information beyond variance alone.
Limitations: The shape-operator estimator has O(nd^2(k + d^2)) complexity in ambient dimension d, remaining the dominant cost for high-dimensional raw data.A pre-PCA step at T = 25 mitigates but does not eliminate this cost, and may poorly preserve informative structure.
Future work: Future work includes jointly selecting neighborhood size k and embedding dimension m using label-free machinery and bootstrap or consensus stability.This direction aims toward a fully self-contained, hyperparameter-free pipeline.
Future work: Future studies should compare the symmetric curvature formulation with the asymmetric version and test adaptive weighting of local shape operators.The comparison would assess clustering performance, while adaptive weighting would emphasize regions of high geometric complexity.

Statements and declarations

The study reports support from CNPq and CAPES, provides reproducibility scripts online, and identifies OpenML as the source of publicly available experimental datasets.

Funding: Funding came from CNPq through grant 301432/2025-2 and from CAPES via Finance Code 001.The work was supported by Brazil’s National Council for Scientific and Technological Development and CAPES.
Code availability: Python scripts for reproducing the reported results are available at https://github.com/alexandrelevada/ShapeOperatorPCA.The repository is identified as the source of the reproducibility scripts.
Data availability: All experimental datasets are publicly available through www.openml.org.The datasets used in the experiments can be accessed from OpenML.