Source-linked AI summary
ETH-XGaze: A Large Scale Dataset for Gaze Estimation under Extreme Head Pose and Gaze Variation
Xucong Zhang, Seonwook Park, Thabo Beeler, Derek Bradley, Siyu Tang, Otmar Hilliges
TL;DR
Gaze-estimation research is hindered by limited pose and gaze coverage in existing datasets and heterogeneous evaluation procedures. ETH-XGaze addresses these gaps with a large-scale, high-resolution dataset and standardized benchmarking, and baseline evaluation shows improved robustness to head-pose and gaze-direction variation.
Problem
Existing gaze datasets have limited head-pose and gaze-direction variation, while custom preprocessing and evaluation protocols make methods difficult to compare.
Method
The paper constructs ETH-XGaze with broad pose, gaze, subject, lighting, and resolution variation, and proposes a standardized evaluation protocol and framework.
Results
Training on ETH-XGaze significantly improves robustness to head-pose and gaze-direction variation compared to existing datasets.
Takeaways & Limitations
ETH-XGaze provides a resource and benchmark for developing and fairly comparing robust gaze-estimation algorithms.
Abstract
from arXiv · showhide
Gaze estimation is a fundamental task in many applications of computer vision, human computer interaction and robotics. Many state-of-the-art methods are trained and tested on custom datasets, making comparison across methods challenging. Furthermore, existing gaze estimation datasets have limited head pose and gaze variations, and the evaluations are conducted using different protocols and metrics. In this paper, we propose a new gaze estimation dataset called ETH-XGaze, consisting of over one million high-resolution images of varying gaze under extreme head poses. We collect this dataset from 110 participants with a custom hardware setup including 18 digital SLR cameras and adjustable illumination conditions, and a calibrated system to record ground truth gaze targets. We show that our dataset can significantly improve the robustness of gaze estimation methods across different head poses and gaze angles. Additionally, we define a standardized experimental protocol and evaluation metric on ETH-XGaze, to better unify gaze estimation research going forward. The dataset and benchmark website are available at https://ait.ethz.ch/projects/2020/ETH-XGaze
1 Introduction
ETH-XGaze addresses limited head-pose and gaze-direction coverage and inconsistent evaluation in gaze estimation by introducing a large-scale, high-resolution dataset with standardized benchmarking. Its design samples broad pose, gaze, subject, lighting, and resolution variations for robust evaluation and analysis.
- Existing gaze datasets mostly cover frontal settings with narrow head-pose and gaze-direction ranges.
- ETH-XGaze samples head poses up to ±70° and gaze directions up to ±50° in the head coordinate system.
- ETH-XGaze uses high-resolution imagery that allocates considerably more pixels to the periocular region than existing datasets.
- The paper proposes a standardized evaluation protocol, metrics including a robustness evaluation, and a public website for submitting and comparing methods.
- A ResNet-50 baseline estimates gaze from monocular face patches and supports ablations over head pose, gaze angle, subjects, illumination, and input resolution.
- The dataset contains over 1 Mio samples from 110 participants, with consistent label quality and high-resolution images.
2 Related Work
Prior gaze-estimation datasets and methods have expanded beyond static frontal assumptions, but dataset coverage and evaluation remain fragmented. ETH-XGaze is positioned against limited pose ranges, limited facial resolution, sparse samples, and the absence of a unified benchmark.
- Early gaze-estimation methods often assumed static head pose, while later work progressively allowed greater head-pose freedom.
- Recent methods reduce gaze-estimation errors using domain adaptation, Bayesian networks, adversarial approaches, coarse-to-fine methods, and multi-region CNNs.
- Existing multi-view datasets face limited head-pose ranges, limited effective face resolution, or sparse discrete gaze sampling.
- The Columbia dataset uses five high-resolution cameras but records only 5,880 samples with discrete gaze directions.
- Gaze estimation lacks a unified evaluation protocol and a general evaluation dataset, while prior work often uses custom preprocessing and evaluation procedures.
- Table 1 compares datasets by participants, maximum head-pose and gaze ranges, data amount, and image resolution.
3 ETH-XGaze Dataset
ETH-XGaze is designed to maximize variation in head pose, gaze direction, subject appearance, illumination, and image resolution. Its 18-camera, high-resolution setup and controlled lighting support broad dataset coverage, gaze-estimation evaluation, and downstream data synthesis.
- Dataset design: ETH-XGaze maximizes variation across head pose, gaze direction, subject appearance, illumination condition, and image resolution.The collection procedure is explicitly designed to maximize the range of each dimension.
- Acquisition setup: 18 Canon 250D digital SLR cameras capture 6000 × 4000-pixel images from different viewpoints for broad pose coverage and future 3D reconstruction.Five paired cameras support geometry capture, while eight support texture acquisition.
- Participants and sampling: 110 participants contributed data spanning multiple ages, genders, ethnicities, eyewear conditions, and gaze-point and lighting configurations.Each participant contributed 525 full-lighting gaze points and 90 points under varying lighting conditions.
- Data characteristics: ETH-XGaze provides the highest effective face-region resolution among compared datasets and records 16 controlled illumination conditions.Its high-resolution imagery is intended to support gaze estimation, generative methods, and analysis of illumination effects.
- Data pre-processing: Images are normalized by estimating 3D head pose and warping each face into a fixed-size 448×448-pixel virtual-camera view.The process uses facial landmarks, camera calibration, a 3D morphable face model, and a virtual camera with specified focal length and distance.
4 Evaluation Protocol
ETH-XGaze defines a benchmark spanning cross-dataset, within-dataset, person-specific, and head-pose/gaze-direction robustness evaluations. Its baseline and protocols establish systematic comparisons across datasets and calibration settings.
- Evaluation Protocol: Four evaluations cover cross-dataset, within-dataset, person-specific, and robustness across head poses and gaze directions.The fourth criterion is enabled by ETH-XGaze's broad pose and gaze coverage.
- Cross-dataset Evaluation: Cross-dataset evaluation trains on ETH-XGaze and tests on other datasets, and reverses this direction to assess generalization.Pair-wise evaluations report gaze estimation errors in degrees and average rankings.
- Cross-dataset Evaluation: ETH-XGaze produces large cross-dataset errors when used as a test set, reflecting a domain gap caused by its larger head-pose and gaze-direction variation.Estimators must extrapolate to unseen poses and gaze directions, which the authors describe as difficult.
- Cross-dataset Evaluation: Training on Gaze360 achieves the best cross-dataset performance on ETH-XGaze, while ETH-XGaze achieves the best results on EYEDIAP and Gaze360 without data augmentation.The authors attribute the Gaze360 result to similar head-pose and gaze-direction ranges, while noting its lower-quality images and labels.
- Within-dataset Evaluation: The ResNet-50 baseline achieves an average error of 4.7 degrees within ETH-XGaze and comparable accuracy to current state-of-the-art methods on other datasets.The baseline uses a 224 × 224 full-face patch and outputs horizontal and vertical gaze angles.
- Person-specific Evaluation: Personal calibration with up to 200 samples improves gaze estimates by a large margin in person-specific evaluation.The protocol trains on TR plus calibration samples and tests on the remaining samples for each of 15 test subjects.
5 Demonstration of ETH-XGaze
ETH-XGaze enables controlled analysis of how head pose, gaze direction, illumination, participant count, and input resolution affect gaze estimation. The experiments show that broader training variation and matched higher-resolution inputs improve performance, while cross-resolution transfer remains difficult.
- Head Pose and Gaze Direction: ETH-XGaze supports ablations of head pose, gaze direction, and illumination because prior datasets lack sufficient coverage.The dataset is explicitly presented as suitable for evaluating these training factors.
- Head Pose and Gaze Direction: Training subsets constrained to narrower head-pose and gaze-direction ranges were evaluated on TE with equalized sample counts.The tested ranges were ±80, ±60, ±40, and ±20 in both horizontal and vertical directions.
- Error Distributions: Figure 8 compares gaze-estimation error distributions across head poses and gaze directions for models trained on different training sets.Horizontal and vertical errors are shown across the two variation axes.
- Illumination Condition: 9% performance degradation, from 7.8° to 8.5°, occurred when training used only full-lighting conditions instead of all lighting conditions.The result indicates that lighting variation affects gaze estimation performance.
- Personal Appearance: Increasing the number of participants improved baseline gaze-estimation performance, following the trend reported in prior work.The comparison is shown in the right side of Fig. 9.
- Input Resolution: Higher-resolution training and testing improved performance, but models trained at one resolution performed much worse at other resolutions.The baseline used 224 × 224 inputs and also evaluated 112 × 112 and 448 × 448 resolutions.
6 Conclusion
ETH-XGaze is a large-scale, high-resolution gaze dataset covering broad pose, gaze, appearance, lighting, and ground-truth variations. Baseline evaluation reports improved robustness to pose and gaze variation, while the accompanying standardized framework supports fairer algorithm comparisons.
- Dataset: ETH-XGaze combines broad head-pose and gaze-direction variation with high-resolution images, varied subject appearance, systematic illumination, and accurate gaze vectors.The dataset is introduced as a resource for future gaze-estimation research.
- Results: Baseline evaluation shows significantly improved robustness to head-pose and gaze-direction variation compared with existing datasets.The conclusion frames this robustness improvement as the dataset’s principal demonstrated benefit.
- Benchmark: The proposed standardized experimental protocol and evaluation framework is intended to enable fair comparison of gaze-estimation algorithms.The framework is to be distributed through the benchmark website alongside the dataset.