Source-linked AI summary
6D Rotation Representation For Unconstrained Head Pose Estimation
Thorsten Hempel, Ahmed A. Abdelrahman, Ayoub Al-Hamadi
TL;DR
Head pose estimation faces ambiguous or discontinuous rotation representations and information loss from angle binning. The paper introduces a continuous 6D rotation representation with geodesic loss for direct full-range regression, outperforming state-of-the-art methods by up to 20% across multiple datasets.
Problem
Head pose estimation methods can lose angle information through binning, while Euler-angle and quaternion representations introduce discontinuities or ambiguities that hinder learning full orientations.
Method
The method regresses a compressed continuous 6D rotation representation converted into a rotation matrix and trains with geodesic distance on the SO(3) manifold.
Results
The method outperforms other state-of-the-art methods on multiple datasets by up to 20%.
Takeaways & Limitations
The approach supports full-range rotation regression without angle-restricting binning and achieves robust performance across yaw, pitch, and roll.
Takeaways & Limitations
For fair comparison, evaluation retains only samples with Euler angles between -99° and 99°.
Abstract
from arXiv · showhide
In this paper, we present a method for unconstrained end-to-end head pose estimation. We address the problem of ambiguous rotation labels by introducing the rotation matrix formalism for our ground truth data and propose a continuous 6D rotation matrix representation for efficient and robust direct regression. This way, our method can learn the full rotation appearance which is contrary to previous approaches that restrict the pose prediction to a narrow-angle for satisfactory results. In addition, we propose a geodesic distance-based loss to penalize our network with respect to the SO(3) manifold geometry. Experiments on the public AFLW2000 and BIWI datasets demonstrate that our proposed method significantly outperforms other state-of-the-art methods by up to 20\%. We open-source our training and testing code along with our pre-trained models: https://github.com/thohemp/6DRepNet.
1. INTRODUCTION
Single-image head pose estimation is useful across facial-analysis applications, but existing approaches face landmark failures, information loss from angle binning, and discontinuous rotation representations. The proposed landmark-free method uses a rotation matrix, compressed 6D regression, and geodesic loss to support full-pose learning.
- Single-image head pose estimation supports applications including driver assistance, augmented reality, and human-robot interaction.
- Landmark-based methods can be accurate but depend on correctly localized facial landmarks, which occlusion and extreme rotation can impair.
- Angle binning stabilizes predictions but loses information, while four-or-fewer-dimensional rotation representations are discontinuous for neural-network learning.
- The proposed landmark-free method regresses a compressed 6D rotation representation that is subsequently transformed into a 3 × 3 rotation matrix.
- Geodesic loss replaces mean squared error to penalize predictions according to SO(3) manifold geometry.
- Training code, testing code, and trained CNN models are publicly available for research experimentation and practical application development.
2. METHOD
The method uses a continuous rotation-matrix representation while avoiding direct nine-parameter regression through a 6D-to-3D transformation. It trains with geodesic distance, which measures the shortest path between predicted and ground-truth rotations on SO(3).
- Rotation matrices provide a continuous, uniquely parameterized representation, but direct regression must satisfy the 3 × 3 orthogonality constraint.
- The remaining matrix column is determined by a cross product, ensuring orthogonality in the resulting rotation matrix.
- The predicted 6D representation is mapped into a 3 × 3 rotation matrix, reducing the network output to six parameters.
- Geodesic distance measures the shortest path between estimated and ground-truth 3D rotations, unlike Frobenius distance, which breaks SO(3) manifold geometry.
- The geodesic distance is used as the neural-network loss to compute orientation distance between predicted and ground-truth rotations.
3. EXPERIMENTS
Experiments evaluate 6DRepNet across synthetic-to-real and BIWI train/test settings, compare geodesic with ℓ2 loss, and assess backbone choice. The method achieves strong accuracy across datasets and rotation angles, with geodesic loss and RepVGG providing additional gains.
- Experiment 1: Training on synthetic 300W-LP and testing on real AFLW2000 and BIWI evaluates cross-dataset head-pose generalization.The standard metric is mean absolute error after converting predicted rotation matrices into Euler angles.
- Experiment 1: Almost 20% improvement on AFLW2000 establishes the method as the best-performing approach there, with the lowest yaw, pitch, and roll errors.On BIWI, the method achieves state-of-the-art overall MAE and balanced errors across the three angles.
- Experiment 2: The BIWI 70:30 split shows that 6DRepNet outperforms all compared methods in overall MAE and matches its balanced yaw, pitch, and roll performance.This supports robustness across datasets rather than dependence on a particular training set.
- Experiment 3: Geodesic-loss networks perform slightly better than networks trained with ℓ2-norm loss.The comparison uses mean absolute error across the repeated experiments.
- Experiment 4: RepVGG performs about 7% better than ℓ2 loss across all test scenarios, while ResNet50 still produces state-of-the-art AFLW2000 results.The backbone comparison uses MAE and evaluates RepVGG against ResNet50.
4. CONCLUSION
The paper presents an unconstrained end-to-end head pose estimator using a continuous 6D rotation representation and geodesic loss for direct regression. Unlike prior approaches, it predicts full rotations without angle-restricting binning and outperforms state-of-the-art methods on multiple datasets by up to 20%.
- The method performs unconstrained end-to-end head pose estimation from single images.
- A continuous 6D rotation matrix representation enables efficient direct regression of full rotations.
- Geodesic loss replaces MSE to support robust training of the rotation regressor.
- The method avoids angle-restricting binning while regressing full rotations.
- Up to 20% improvement over other state-of-the-art methods is reported across multiple datasets.