PureGaze: Purifying Gaze Feature for Generalizable Gaze Estimation

Introduction

• Changed environments, including different illumination, can degrade pretrained gaze models, while adaptation methods often require target samples and time-consuming setup.Those requirements harm the flexibility of adaptation methods.Source: Introduction · p. 1
• PureGaze learns a generalized model in the source domain for unseen target domains without using target images or labels.The method targets domain-specific factors such as illumination and identity that blend into captured images and are difficult to eliminate during feature extraction.Source: Introduction · pp. 1–2
• A self-adversarial framework simultaneously preserves gaze information and removes general facial-image information to purify features without defining every gaze-irrelevant factor.The practical implementation uses gaze estimation and adversarial reconstruction as the two tasks.Source: Introduction · p. 2, Figure 2 · p. 2
• The plug-and-play SA-Module is presented as a way to enhance existing gaze-estimation methods, and the paper reports state-of-the-art performance on many benchmarks.The authors also claim the framework improves cross-dataset performance without knowing the target dataset or using new samples.Source: Introduction · p. 2

Typical Gaze Estimation

• Gaze-estimation methods address subject variance, eye asymmetry, attention weighting, dilated convolution, and gaze or head redirection [Cheng, Lu, and Zhang 2018; Cheng et al. 2020b; Park, Spurr, and Hilliges 2018; Zhang et al. 2017; Chen and Shi 2019; Zheng et al. 2020].Other approaches use two eye-image networks or refine facial-image estimates with eye images [Fischer, Jin Chang, and Demiris 2018; Cheng et al. 2020a].Source: Related Works · p. 2

Cross-domain Gaze Estimation

• Cross-domain approaches include fine-tuning a source-pretrained model on the target domain [Zhang et al. 2018] and adversarial feature alignment between source and target.An ensemble approach also learns collaboratively with outlier guidance.Source: Related Works · p. 2
• The reviewed cross-domain methods use target-domain data, which the authors describe as not always user-friendly.Source: Related Works · p. 2

Definition of the Purification

• The formulation represents an image's features as disjoint gaze-relevant and gaze-irrelevant subsets, and seeks an extracted feature without the latter.It also aims to eliminate features with weak relations to gaze.Source: Definition of the Purification · p. 3

Self-Adversarial Framework

• The framework minimizes mutual information between image features and extracted features to reduce image information in the representation.The paper defines H(X, Y) as mutual information and notes it is zero when X and Y are independent.Source: Self-Adversarial Framework · p. 3
• A second task maximizes mutual information between gaze-relevant features and extracted features, preserving gaze-related information.Source: Self-Adversarial Framework · p. 3

Learning to Purify in the Framework

• Simultaneously optimizing both tasks forms a self-adversarial framework that removes gaze-irrelevant features while preserving gaze-relevant ones.The paper characterizes the combined objectives as a minimax problem in which weakly gaze-related features are discarded first.Source: Learning to Purify in the Framework · p. 3

PureGaze

• Gaze estimation preserves gaze information while adversarial reconstruction aims to remove general image information from extracted features.The backbone and reconstruction module oppose one another: the module reconstructs images, while the backbone prevents reconstruction.Source: PureGaze · pp. 3–4

Architecture of PureGaze

• A shared backbone serves gaze estimation and image reconstruction, with an MLP regressing gaze and a SA-Module reconstructing images.The backbone loss combines adversarial and gaze losses, while the MLP and SA-Module use the gaze and reconstruction losses, respectively.Source: Architecture of PureGaze · p. 4

Local Purification Loss

• LP-Loss applies an eye-centered attention map only to Ladv, leaving gaze estimation based on the whole face unchanged.The attention map is generated from a mixed Gaussian centered at the two eye centers, with customizable variance; other facial regions remain available to gaze estimation.Source: Local Purification Loss · p. 4

Truncated Adversarial Loss

• TA-Loss truncates Ladv at threshold k, setting it to zero when pixel difference exceeds k to avoid redundant reconstruction prevention.The authors argue that very large pixel differences are unnecessary because Ladv is intended to prevent reconstruction, not produce an inverse image.Source: Truncated Adversarial Loss · p. 4

Data-preprocessing

• Training uses Gaze360 and ETH-XGaze; evaluation spans MPIIGaze and EyeDiap across four source-to-target tasks.Gaze360 images without face-detection results are removed, and datasets are prepared using established rectification protocols.Source: Data-preprocessing · pp. 4–5

Performance Comparison with SOTA Methods

• Across four cross-dataset tasks, PureGaze leads typical gaze estimators, beats ADL* in all tasks, and is competitive with adaptation while using one model versus four.The authors report that PureGaze learns one model for four tasks, whereas domain adaptation methods require four models.Source: Table 1 · p. 5, Performance Comparison with SOTA Methods · p. 6

Comparison Methods

• The Baseline removes the SA-Module; comparisons include four typical gaze estimators and domain-adaptation methods ADL and ADL*.ADL* uses a discriminator to distinguish personal features in source domains and does not require target samples.Source: Comparison Methods · p. 5

Plug Existing Gaze Estimation Methods

• Adding the self-adversarial framework improves Full-Face and CA-Net across tasks; CA-Net+SA moves from worst to best on G→M and improves nearly 70% on G→D.The framework adds no inference parameters or training images.Source: Plug Existing Gaze Estimation Methods · p. 6, Table 2 · p. 6

Ablation Study of Two Loss Function

• Both LP-Loss and TA-Loss improve performance, with PureGaze performing best at σ2 = 20 and k = 0.75.The tested values were σ2 = 10, 20, 30, and 40, and k = 0, 0.25, 0.5, and 0.75; k = 0 ablates TA-Loss.Source: Ablation Study of Two Loss Function · p. 6, Figure 4 · p. 6

Visualize Extracted Feature via Reconstruction

• Reconstructions indicate purification reduces identity, illumination, and head-rest information while recovering bright gaze regions from low-light images.The method does not specify in advance which gaze-irrelevant factors it eliminates.Source: Figure 5 · p. 7, Visualize Extracted Feature via… · p. 7

• Domain-generalization methods can be applied directly to new domains, whereas domain-adaptation setup is usually time-consuming.Source: Discussion · p. 7
• The flexibility of domain generalization comes with a typical accuracy trade-off because these methods lack target-domain information.Source: Discussion · p. 7

2) Self-adversarial framework.

• The self-adversarial framework learns purified gaze features without requiring target-domain samples, making it a zero-shot cross-domain method.It removes some gaze-irrelevant features, based on the observation that gaze patterns are similar across domains while such factors are often domain-specific.Source: 2) Self-adversarial framework · p. 7
• Source-domain performance can change by ±0.2° because an overfitting model can perform better there than PureGaze.The authors identify learning a more generalized model as future work.Source: 2) Self-adversarial framework · p. 7
• The framework’s purification targets gaze-irrelevant features that are usually domain-specific and can bring performance drops across domains.The authors contrast these factors with gaze patterns, which they observe to be similar across domains.Source: 2) Self-adversarial framework · p. 7

Conclusion

• The method achieves state-of-the-art performance among typical gaze-estimation methods and competitive results compared with domain-adaptation methods.The conclusion reports these comparisons without giving quantitative values.Source: Conclusion · p. 7
• The plug-and-play framework purifies gaze features to improve performance in unknown target domains without touching those domains.The conclusion describes it as a domain-generalization framework.Source: Conclusion · p. 7
• The conclusion presents the framework as a new direction for addressing the cross-dataset problem in gaze estimation.Source: Conclusion · p. 7

Appendix

• The supplementary document gives background, derives PureGaze from the self-adversarial framework, and describes implementation details.Source: Appendix · p. 8
• It compares gaze-estimation methods on an apple-to-apple basis to examine advantages and disadvantages of domain generalization.Source: Appendix · p. 8
• In very dark environments, visualizations show PureGaze eliminating illumination factors from extracted features, and quantitative results across illumination intensities match those visualizations.Source: Appendix · p. 8

Background

• Appearance-based gaze estimation uses a mapping from facial appearance to gaze, with CNNs used by many methods to model that mapping (Wang, Su, and Ji 2019; Wang et al. 2019a).Source: Background · p. 8
• Few-shot and gaze-redirection methods address cross-person performance through person-specific calibration or redirected eye images (Park et al. 2019; Yu, Liu, and Odobez 2019).The cited few-shot approach combines auto-encoder feature representation with meta-learning for a person-specific gaze mapping.Source: Background · p. 8
• Cross-dataset estimation is described as more challenging than cross-person estimation, involving personal, environmental, and data-distribution differences; this work addresses the first two.Source: Background · p. 8
• Angular error compares actual and estimated gaze directions, and a smaller error indicates a better model.Source: Background · p. 8

Deduction from the Framework to PureGaze

• The derivation pairs adversarial image reconstruction, where E prevents R from reconstructing images from features, with gaze regression that estimates gaze from feature z.The two tasks respectively discourage image information in the feature and preserve gaze information.Source: Deduction from the Framework to PureGaze · p. 8

Implementation Detail

• The implementation uses a ResNet-18 backbone, a two-layer gaze MLP, and a five-layer SA-Module, while allowing these components to vary if the module outputs input-sized images.Adam is used with a learning rate of 0.0001 for all three networks.Source: Implementation Detail · p. 8

Comparison between Different Types of Methods

• PureGaze outperforms the same-backbone Baseline, while ADL also beats Baseline but requires setup and target samples; PureGaze does not target large domain-specific gains.The comparison also notes that PureGaze uses one model across domains, unlike ADL's domain-specific models.Source: Table 3 · p. 9, Comparison between Different Types of… · p. 9

Visualization on Very Dark Environment.

• In very dark scenes where subjects are invisible in the originals, reconstructions remove illumination while retaining gaze-relevant detail and accurately capture eye movement.The figure shows original images, manually augmented images, and purified-feature reconstructions in successive rows.Source: Figure 6 · p. 9, Visualization on Very Dark Environment · p. 9

Quantitative Evaluation in Illumination

• PureGaze improves over Baseline especially under extreme illumination, consistent with removing illumination information from extracted features.The evaluation trains on ETH-XGaze and tests on MPIIGaze, grouping images by mean intensity.Source: Quantitative Evaluation in Illumination · p. 9, Figure 7 · p. 10