Source-linked AI summary
GODS: Generalized One-class Discriminative Subspaces for Anomaly Detection
Jue Wang, Anoop Cherian
TL;DR
The paper tackles one-class learning when abnormal examples are unavailable. It proposes discriminative subspaces that sandwich labeled data, optimizes their non-convex objective on the Stiefel manifold, and reports state-of-the-art performance across five datasets, including Dash-Cam-Pose.
Problem
One-class learning must model normal data when annotations for abnormal or faulty conditions are unavailable.
Method
The method sandwiches labeled data with orthonormal subspaces using BODS and its multiple-hyperplane GODS extension, optimized by Riemannian conjugate gradients on the Stiefel manifold.
Results
More than 10% improvement in performance over classical and recent approaches is reported across the evaluated datasets.
Takeaways & Limitations
Experiments on five datasets, including the proposed Dash-Cam-Pose dataset, demonstrate state-of-the-art performance for the approach.
Takeaways & Limitations
The optimization objective is non-convex because of the orthogonality constraints.
Abstract
from arXiv · showhide
One-class learning is the classic problem of fitting a model to data for which annotations are available only for a single class. In this paper, we propose a novel objective for one-class learning. Our key idea is to use a pair of orthonormal frames -- as subspaces -- to "sandwich" the labeled data via optimizing for two objectives jointly: i) minimize the distance between the origins of the two subspaces, and ii) to maximize the margin between the hyperplanes and the data, either subspace demanding the data to be in its positive and negative orthant respectively. Our proposed objective however leads to a non-convex optimization problem, to which we resort to Riemannian optimization schemes and derive an efficient conjugate gradient scheme on the Stiefel manifold. To study the effectiveness of our scheme, we propose a new dataset~\emph{Dash-Cam-Pose}, consisting of clips with skeleton poses of humans seated in a car, the task being to classify the clips as normal or abnormal; the latter is when any human pose is out-of-position with regard to say an airbag deployment. Our experiments on the proposed Dash-Cam-Pose dataset, as well as several other standard anomaly/novelty detection benchmarks demonstrate the benefits of our scheme, achieving state-of-the-art one-class accuracy.
1. Introduction
The paper addresses one-class settings where normal data are available but abnormal examples are difficult or impossible to collect. It introduces discriminative subspaces, extends them from two hyperplanes to multiple hyperplanes, and evaluates them across a new pose dataset and established benchmarks.
- One-class learning models normal behavior when abnormal or faulty examples are unavailable, treating abnormalities as outliers.
- Classical methods include OC-SVM, SVDD, and kernelized variants, but they respectively rely on a single hyperplane, spherical data assumptions, or potentially limited scalability.
- The non-convex orthogonality-constrained objective is optimized using Riemannian conjugate gradients on the Stiefel manifold.
- More than 10% improvement in performance over classical and recent approaches is reported across Dash-Cam-Pose, UCF-Crime, JHMDB, and two UCI anomaly datasets.
- The proposed BODS classifier uses a pair of independently parameterized hyperplanes to sandwich labeled data from different sides.
- GODS generalizes BODS from two hyperplanes to multiple hyperplanes for richer characterization of the labeled set.
2. Background and Related Works
The related-work discussion contrasts one-class classifiers, geometric descriptions, kernel and subspace approaches, and motivates discriminative subspaces that bound data piecewise linearly rather than approximate it.
- One-class classifiers learn a function that is positive on the normal class and negative on the anomaly class using only labeled normal instances.
- OC-SVM learns a max-margin hyperplane separating the origin from the normal data, while kernel extensions address nonlinear boundaries but can be difficult to scale.
- SVDD encloses labeled data in a minimum-radius hypersphere, but its isotropic distribution assumption can be restrictive.
- PCA-based subspace summaries can be computationally costly on large datasets, motivating an alternative subspace formulation.
- The paper uses independently oriented hyperplanes to bound one-class data piecewise linearly in a discriminative objective, forming BODS and its multiple-hyperplane GODS extension.
3. Proposed Method
The paper develops BODS and generalizes it to GODS, using paired hyperplanes or orthonormal subspaces to sandwich one-class data from opposite sides while encouraging compactness around the data.
- BODS: BODS assigns every labeled point to the positive half-space of one hyperplane and the negative half-space of another.The two hyperplanes are jointly constrained by a distance term and classification-margin requirements.
- BODS: Unit-norm hyperplane weights reduce the distance term and help regularize the BODS classifier against overfitting.The weights are constrained to unit norm, placing them on a unit hypersphere.
- GODS: GODS replaces each hyperplane with a d × K orthonormal frame whose columns represent multiple discriminative directions.The orthogonality constraints place the frames on the Stiefel manifold and promote diverse directions.
- GODS: For GODS, each projected data vector must have minimum value above η on one side and maximum value below −η on the other.The K-dimensional projections impose positive and negative margin constraints across multiple hyperplanes.
- GODS: Instead of the expensive Procrustes distance, GODS minimizes the summed projection lengths of each data point onto the respective subspaces.This cheaper surrogate pulls both subspaces toward the data while retaining the sandwiching objective.
- GODS: Multiple hyperplanes provide a richer characterization of the one-class distribution than single-hyperplane schemes.The paper presents GODS as a piecewise-linear discriminative alternative to approximation-based subspace methods.
4. Efficient Optimization
GODS is optimized on a product of Stiefel and Euclidean manifolds using Riemannian conjugate gradients, with manifold-aware gradients, transports, and retractions; initialization addresses multiple local solutions.
- Manifold optimization: Orthogonality makes the GODS objective non-convex, but places its frames on the Stiefel manifold for geometric optimization.The paper uses Riemannian conjugate gradients because of their reported stable and fast convergence.
- Riemannian conjugate gradient: Riemannian conjugate gradients adapt conjugate-gradient updates to curved spaces through tangent-space gradients, vector transports, and manifold-preserving retractions.QR-based retractions keep iterates on the manifold, while vector transports align directions from different tangent spaces.
- Optimization implementation: The joint optimization is solved on the Cartesian product of Stiefel and Euclidean manifolds, with Euclidean gradients derived for the frame variables.Bias variables remain Euclidean, while the frame gradients are the non-trivial component needed by RCG.
- Optimization implementation: The frame-gradient expression selects the active hyperplane index associated with the minimum or maximum projected response.The selected index and corresponding data column determine the sparse matrix contribution in the gradient.
- Initialization: Initialization sorts points by distance from the origin and uses near and far groups with SVD before optimization.This heuristic is motivated by the possibility of multiple local solutions in the non-convex objective.
5. One-class Classification
At test time, GODS scores each point using the learned model and separates normal from other points by two-cluster K-means on those scores.
- Test-time classification: The learned GODS parameters produce a score for each test point using the model’s scoring function.The procedure applies the learned parameters from problem P3 to the test data.
- Test-time classification: K-means with K = 2 clusters the scores, assigning the smaller-score cluster to the modeled one-class set.The remaining cluster is assigned to the other class.
6. Experiments
The experiments evaluate GODS across pose, action, anomalous-event, sonar, and pump one-class tasks, using dataset-specific representations and metrics. GODS generally outperforms prior methods, with performance gains reported across datasets while maintaining practical training costs.
- Datasets: The evaluation covers Dash-Cam-Pose, JHMDB, UCF-Crime, Sonar, and Delft pump one-class tasks.Dash-Cam-Pose detects out-of-position human poses; JHMDB evaluates one-versus-rest action recognition; UCF-Crime targets anomalous video events; Sonar and Delft address signal and pump abnormality detection.
- Dash-Cam-Pose: Dash-Cam-Pose fills a gap in existing pose datasets by providing in-vehicle dashboard-camera clips annotated for pose anomalies.The dataset contains approximately 4,875 selected clips, with weak binary labels focused on front-seat humans.
- Representations and metrics: The study uses pose BoW and TCN representations, pretrained visual features for JHMDB, I3D features for UCF-Crime, and raw features for Sonar and Delft pump.Evaluation reports F1 for most datasets, AUC and false alarm rate for UCF-Crime, and additional class-sensitive measures for imbalanced data.
- Ablative studies: GODS subspaces confine synthetic data within minimal volume, while increasing the number of hyperplanes eventually saturates performance.The parameter study also reports that TCN outperforms BoW by nearly 3% on Dash-Cam-Pose with the proposed methods.
- State-of-the-art comparisons: GODS is over 30% better than OC-SVM and about 20% better than kernelized variants on Dash-Cam-Pose.On JHMDB and the other two datasets, GODS is reported as 3–13% better than the next-best method, with classification-accuracy improvement above 10%.
- Efficiency: GODS and BODS are reported as not computationally expensive compared with other methods while being empirically superior.Training-time behavior is examined as the number of training samples increases.
- State-of-the-art comparisons: On UCF-Crime, the reported comparison uses AUC and false alarm rate, and GODS is about 5–20% better than other methods under the one-class setup.Results against a method trained with weakly labeled abnormal videos are treated as incomparable because that setup is not one-class.
7. Conclusions
The paper presents a discriminative one-class formulation that sandwiches data between oriented subspaces and solves its constrained optimization in a Riemannian setting.
- The method uses discriminative subspaces oriented to sandwich the labeled data.
- The resulting nonlinear constrained optimization problem is addressed through Riemannian optimization and efficient numerical solutions.
- Experiments across five datasets, including the new Dash-Cam-Pose dataset, demonstrate state-of-the-art performance.