Source-linked AI summary
A Riemannian Network for SPD Matrix Learning
Zhiwu Huang, Luc Van Gool
TL;DR
SPD matrix learning must handle the non-Euclidean geometry of SPD manifolds while supporting nonlinear computation. The paper proposes SPDNet, which preserves SPD structure through specialized layers and trains with manifold-aware backpropagation; experiments report effectiveness across three visual classification tasks.
Problem
Effective SPD-matrix computation is difficult because the data have a non-Euclidean structure underlying a Riemannian manifold.
Method
SPDNet uses bilinear mapping, eigenvalue rectification, and eigenvalue logarithm layers, trained with backpropagation and stochastic gradient descent on Stiefel manifolds.
Results
Evaluations on three visual classification tasks studied the effectiveness of the proposed network for SPD matrix learning.
Takeaways & Limitations
SPDNet provides a deep, nonlinear learning mechanism that preserves SPD structure and performs Riemannian computing before regular output layers.
Takeaways & Limitations
The paper leaves alternative nonlinearities such as sigmoidal activation for future discussion and assumes orthogonal transformation weights for compact Stiefel-manifold optimization.
Abstract
from arXiv · showhide
Symmetric Positive Definite (SPD) matrix learning methods have become popular in many image and video processing tasks, thanks to their ability to learn appropriate statistical representations while respecting Riemannian geometry of underlying SPD manifolds. In this paper we build a Riemannian network architecture to open up a new direction of SPD matrix non-linear learning in a deep model. In particular, we devise bilinear mapping layers to transform input SPD matrices to more desirable SPD matrices, exploit eigenvalue rectification layers to apply a non-linear activation function to the new SPD matrices, and design an eigenvalue logarithm layer to perform Riemannian computing on the resulting SPD matrices for regular output layers. For training the proposed deep network, we exploit a new backpropagation with a variant of stochastic gradient descent on Stiefel manifolds to update the structured connection weights and the involved SPD matrix data. We show through experiments that the proposed SPD matrix network can be simply trained and outperform existing SPD matrix learning and state-of-the-art methods in three typical visual classification tasks.
Introduction
SPD matrices support powerful visual and medical representations, but their non-Euclidean Riemannian structure complicates computation. The paper introduces a deep architecture that preserves SPD structure while enabling nonlinear learning and manifold-aware training.
- SPD matrices are used as statistical representations in medical imaging and visual recognition, including diffusion tensors and covariance descriptors.
- Directly applying Euclidean geometry to SPD matrices can produce undesirable effects, motivating Riemannian metrics that properly encode SPD-manifold geometry.
- Existing SPD learning methods commonly flatten manifolds through tangent-space approximations or map them into reproducing kernel Hilbert spaces.
- The proposed Riemannian network receives SPD matrices and preserves their structure across layers for nonlinear learning on Riemannian manifolds.
- The architecture incorporates Riemannian structures into both data and weight spaces, while its training uses new backpropagation with stochastic gradient descent on Stiefel manifolds.
Related Work
Related work extends deep learning beyond Euclidean data using spectral methods, matrix backpropagation, and Riemannian optimization. These approaches motivate geometry-aware computation and training for structured matrix networks.
- Spectral convolutional networks address graph-structured data through a non-shift-invariant convolution based on the Laplace-Beltrami eigenbasis.
- Matrix backpropagation generalizes the chain rule to compute derivatives of composed matrix functions with respect to matrix inputs.
- Other optimization methods use Riemannian gradients to handle symmetries in neural-network activity and parameter spaces treated as Riemannian manifolds.
Riemannian SPD Matrix Network
SPDNet preserves SPD structure while learning nonlinear representations through bilinear mappings, eigenvalue rectification, and Log-Euclidean computation. Its outputs can then connect to conventional Euclidean network layers.
- BiMap Layer: BiMap layers transform input SPD matrices into lower-dimensional SPD matrices on corresponding SPD manifolds.The transformation uses row full-rank connection weights, with semi-orthogonal weights optimized on compact Stiefel manifolds.
- BiMap Layer: Semi-orthogonal connection weights place BiMap optimization on compact Stiefel manifolds, where optimal transformation matrices can be obtained.Direct optimization over the corresponding non-compact manifold is infeasible because its distance function has no upper bound.
- ReEig Layer: ReEig layers introduce nonlinearity by replacing small positive eigenvalues with a rectification threshold.The operation uses eigenvalue decomposition and max(ϵI, Σ) to keep outputs away from non-positive matrices.
- ReEig Layer: The ReEig threshold is used even when input eigenvalues are already far above zero, so the layer is not originally designed as regularization.The paper also notes that alternative nonlinearities, such as sigmoidal functions, remain feasible but are not further discussed.
- LogEig Layer: The LogEig layer maps resulting SPD matrices into a flat space using matrix logarithms for Riemannian computation.Log-Euclidean computation is simple, avoids the high expense of other SPD computations, and preserves favorable theoretical properties.
- Output Layers: After LogEig, vectorized outputs can feed Euclidean fully connected and softmax layers for visual recognition.Pooling and normalization can likewise operate on matrix logarithms before mapping back with the matrix exponential.
Riemannian Matrix Backpropagation
Training SPDNet requires geometry-aware updates for Stiefel-constrained BiMap weights and matrix backpropagation through eigenvalue-based ReEig and LogEig layers.
- SPDNet represents the model as successive layer compositions with parameters W_k and layer losses defined from the final objective.
- Two training issues arise: Euclidean weight gradients do not preserve Stiefel orthogonality, and traditional backpropagation does not adequately handle EIG-based layers.
- Stiefel-manifold SGD computes a Riemannian gradient for BiMap weights, follows a tangential direction, and retracts the update back onto the manifold.
- Matrix backpropagation uses a generalized chain rule in which variations dX_k = F(dX_{k−1}) are propagated through the adjoint operator F*.
- An eigenvalue-decomposition virtual layer supports derivatives through the EIG operation, enabling partial derivatives for ReEig and LogEig layers.
- For ReEig and LogEig, the derived variations respectively use rectified eigenvalues and logarithmic eigenvalues to compute layer derivatives.
- Combining these gradient calculations with Riemannian SGD realizes matrix backpropagation for training SPDNet.
Discussion
SPDNet extends geometry-aware SPD mappings into a deep-learning setting by using a more general Stiefel-manifold map and introducing nonlinearity. It operates directly on SPD matrices through multiple tailored layers and can also build on covariance-pooling networks for deeper learning.
- SPDNet uses a more general Stiefel-manifold setting than earlier geometry-aware SPD maps and places it within deep learning.
- The network introduces nonlinearity while learning SPD matrices, unlike the cited earlier geometry-aware mapping approaches.
- Unlike covariance pooling that begins with images, SPDNet operates directly on SPD matrices and uses multiple layers tailored for SPD matrix deep learning.
- SPDNet can be built on top of a covariance-pooling network to support deeper SPD matrix learning beginning from images.
Experiments
Experiments evaluate SPDNet across emotion recognition, action recognition, and face verification, including comparisons with shallow and state-of-the-art methods. Results show improvements across tasks, with deeper BiRe configurations and Riemannian computing layers contributing to performance.
- Evaluation setup: SPDNet was evaluated on AFEW emotion recognition, HDM05 action recognition, and PaSC face verification tasks against established SPD-learning and state-of-the-art methods.The evaluated methods include CDL, LEML, SPDML, STM-ExpLet, RSR-SPDML, HERML-DeLF, and VGGDeepFace.
- Emotion Recognition: On AFEW, SPDNet-3BiRe achieved several improvements over state-of-the-art methods despite the small training set.The model used transformation dimensions 400 × 200, 200 × 100, and 100 × 50, with approximately two minutes of training per epoch.
- Ablation studies: Removing LogEig produced accuracies of 21.49% on AFEW and 4.89% on HDM05, while deeper networks and additional BiRe blocks improved performance.SPDNet-0BiRe reached 26.32% on AFEW, and the authors report the same configuration conclusions on PaSC.
- Ablation studies: The convergence curve indicates that SPDNet converges well after hundreds of epochs at rectification threshold ϵ = 10−4.Figure 2 also evaluates accuracy under different rectification-threshold settings on AFEW.
- Action Recognition: On HDM05, SPDNet-3BiRe outperformed shallow SPD-learning methods by more than 13% under the 130-class action-recognition protocol.The authors report that this improvement occurs when the training data is large enough.
- Face Verification: On PaSC, SPDNet achieved the highest performances for control and handheld face-verification tasks, despite using a softmax output layer considered unfavorable for verification.The model used transformation dimensions 401 × 200, 200 × 100, and 100 × 50.
Conclusion
The paper proposes a deep Riemannian network for nonlinear SPD-matrix learning and trains it with backpropagation and SGD on Stiefel manifolds. It validates the architecture on three visual classification tasks and identifies broader architectural extensions as future work.
- Conclusion: The paper proposes a deep Riemannian network architecture for nonlinear learning of SPD matrices.The conclusion frames this architecture as opening a possibility for SPD-matrix nonlinear learning.
- Conclusion: Training uses a new backpropagation procedure with an SGD setting on Stiefel manifolds.This procedure is used to train the proposed SPD network.
- Conclusion: Evaluations on three visual classification tasks study the effectiveness of the proposed network for SPD-matrix learning.The conclusion reports the evaluation scope without specifying individual datasets or metrics.
- Conclusion: Future work includes parallel BiMap, pooling, normalization, convolutional-network integration, general Riemannian manifolds, and more compact architectures.These directions are presented as ways to extend or deepen the Riemannian network.