PaperlayerView exact revision on arXiv

Source-linked AI summary

FAHCD-Net: Frequency-Adaptive Heatmap-Conditional Diffusion Networks for Robust Facial Landmark Detection

Jun Wan, Jiwei Hu, Shengkai Hu, Qilu Zhu

arXiv:2609.16842v1cs.CV

TL;DR

Facial landmark heatmap generation can introduce excess high-frequency information, while landmark detection is also challenged by structural variation, missing information, and noise. FAHCD-Net combines frequency-adaptive diffusion with smoothness regularization, and reports performance surpassing state-of-the-art methods on challenging benchmarks, including a 300W Challenging Subset NMEio of 4.49.

  • Problem

    Generated heatmaps can contain increased high-frequency information relative to ground truth, while facial landmark detection also faces structural variation, information loss, and noise.

    Source: §I · p. 1, C. FAHCD model · p. 5

  • Method

    FAHCD-Net cascades a frequency-adaptive heatmap-conditional diffusion model with smoothness regularization to generate landmark heatmaps while constraining frequency components and heatmap continuity.

    Source: §I · p. 2

  • Results

    FAHCD-Net outperforms state-of-the-art methods on challenging datasets; its NMEio on the 300W Challenging Subset is 4.49.

    Source: §I · p. 2, C. Evaluation of Robustness against… · p. 7

  • Takeaways & Limitations

    The reported results indicate that FAHCD-Net performs strongly on challenging facial landmark detection datasets, including COFW, 300W, WFLW, and AFLW.

    Source: §I · p. 2, C. Evaluation of Robustness against… · p. 7

Abstract

from arXiv · show

Facial Landmark Detection(FLD) is a crucial task in various applications and has achieved significant advancements in recent years. However, current FLD methods still struggle under challenging conditions, where facial structural variations, information loss, and noise interference severely compromise the integrity and accuracy of learned facial features. To address these issues, we propose Frequency-Adaptive Heatmap-Conditional Diffusion Network (FAHCD-Net), which integrates a Frequency-Adaptive Heatmap-Conditional Diffusion (FAHCD) model with a Smoothness Regularization (SR) loss in a cascaded framework. Specifically, the FAHCD model incorporates a Hierarchical Frequency Adaptation (HFA) module designed to suppress redundant high-frequency noise through multi-layer frequency decomposition and adaptive reconstruction, thereby preserving essential facial structures. Additionally, the SR loss is proposed to further mitigate the interference of high-frequency noise and enhance the smoothness of the generated landmark heatmaps. By cascading the FAHCD model with the SR loss, FAHCD-Net effectively leverages both statistical and frequency-based distribution characteristics of the data to progressively generate more accurate landmark heatmaps from noisy inputs. Extensive experiments on popular benchmarks demonstrate the effectiveness and robustness of the proposed method, achieving state-of-the-art performance in FLD tasks under challenging scenarios. The source code is available at https://github.com/HJWKryptonite/FAHCD-Net.

I. INTRODUCTION

Facial landmark detection has advanced through coordinate and heatmap regression, yet structural variation, missing information, and noise remain challenges in complex scenes. FAHCD-Net combines diffusion-based heatmap generation with frequency adaptation and smoothness regularization to address these challenges.

I. INTRODUCTION

  • FLD localizes facial points such as eye corners, the nose tip, lips, and facial contour, supporting applications including recognition, synthesis, expression analysis, and 3D reconstruction.The introduction identifies facial landmark detection as an important computer-vision task.Source: §I · p. 1
  • Expression and pose changes shift landmarks, occlusion and profile views hide landmark information, and blur or illumination introduce prediction noise.These challenges are grouped as structural variation, information loss, and noise interference.Source: §I · p. 1
  • Diffusion models are presented as suited to facial variation, missing features, and noise through distribution learning, generative prediction, and iterative denoising.The paper introduces diffusion-based generation as a way to improve landmark heatmaps.Source: §I · p. 1
  • FAHCD-Net combines a frequency-adaptive diffusion model and smoothness regularization to suppress excess high-frequency noise and generate more accurate landmark heatmaps.The HFA module decomposes and reconstructs frequency components, while SR constrains heatmap smoothness and continuity.Source: §I · p. 2

A. Facial Landmark Detection

  • Coordinate-regression methods map facial features to coordinates but can struggle with spatial relationships and local detail under large pose changes or partial occlusion.Examples include cascaded refinements, topology-adaptive graphs, and attention-based methods.Source: A. Facial Landmark Detection · p. 2
  • Heatmap-regression methods predict a heatmap per landmark, but low-quality heatmaps can blur details and impair localization.Prior approaches include multi-scale feature fusion, joint heatmap-offset prediction, and losses targeting semantic ambiguity.Source: A. Facial Landmark Detection · p. 3

B. Diffusion Models

Diffusion models transform noise into samples from a target distribution through a sequence of denoising transitions. The paper introduces this forward and reverse process as background for its frequency-adaptive heatmap-conditional model.

B. Diffusion Models

  • Diffusion probabilistic models use a Markov chain to transform noise from a simple distribution into data from a complex target distribution.The cited work describes DDPM, while later approaches modify sampling or operate in latent space.Source: B. Diffusion Models · p. 3

A. Preliminary

  • The forward process adds Gaussian noise to the initial data distribution at each step according to a predefined variance schedule.The reparameterization trick permits direct computation of the noisy sample at a chosen timestep.Source: A. Preliminary · p. 3
  • The reverse process uses a neural network to model transitions that progressively reconstruct data from noise.The transition mean is parameterized by a denoising network, and the model is supervised with the L_simple loss.Source: A. Preliminary · p. 3

III. METHODOLOGY

  • FAHCD-Net combines the FAHCD model's adaptive frequency decomposition and reconstruction with SR loss constraints on heatmap smoothness and continuity.The overview describes these components as integrated in a cascaded framework.Source: Figure 3 · p. 4

B. FAHCD-Net Overview

FAHCD-Net conditions denoising on facial image features and a landmark heatmap, predicting intermediate heatmaps rather than noise. Its stages cascade generated heatmaps as subsequent conditions, while SR loss constrains smoothness and continuity.

B. FAHCD-Net Overview

  • The denoising process takes facial image features and a conditional landmark heatmap as inputs, with the noisy heatmap at timestep t and ground-truth heatmap h_0 defining the target context.The reverse transition is conditioned on the noisy heatmap, image features, and conditional landmark heatmap.Source: B. FAHCD-Net Overview · p. 4
  • FAHCD-Net predicts intermediate heatmaps instead of directly predicting noise, then derives the generated landmark heatmap.The model uses a deterministic generation process following DDIM.Source: B. FAHCD-Net Overview · p. 4
  • The proposed SR loss addresses local mutations and redundant high-frequency noise by constraining generated heatmaps to be smooth and continuous.A balancing parameter λ combines the two losses in the complete objective.Source: B. FAHCD-Net Overview · p. 4
  • Three FAHCD-Net stages reuse the preceding stage's generated heatmap as the next condition, leveraging richer facial priors for landmark detection.The initial condition comes from a current FLD model, such as SHN.Source: B. FAHCD-Net Overview · p. 4

C. FAHCD model

FAHCD addresses frequency changes that degrade diffusion-generated landmark heatmaps by decomposing and reconstructing multi-scale features. Its HFA module and decoder integration aim to suppress redundant high-frequency noise while preserving facial structure.

C. FAHCD model

  • FAHCD-Net suppresses high-frequency noise more effectively than the version without HFA and SR loss, as shown by heatmaps and PSD maps.The caption reports that the noise affects precise landmark localization.Source: Figure 4 · p. 5
  • The paper identifies frequency changes during stepwise generation from Gaussian noise to the target heatmap as degrading heatmap quality.Source: C. FAHCD model · p. 5
  • Diffusion generation increases high-frequency information relative to ground-truth heatmaps while low-frequency information remains stable.The authors associate this increase with peak blurring, noise overfitting, and noise in smooth regions.Source: C. FAHCD model · p. 5
  • HFA applies multi-scale Gaussian blur as low-pass filtering, then combines decomposed frequency components with trainable weights.Three Gaussian kernels filter high-, mid-, and lower-frequency components before fusion.Source: C. FAHCD model · p. 5
  • The reconstructed frequency features are concatenated with encoder features and passed to the decoder to reduce redundant noise and model facial structure.FAHCD uses a U-net encoder-decoder structure.Source: C. FAHCD model · pp. 4–5

D. Smoothness Regularization Loss

The SR loss complements FAHCD by encouraging smooth, continuous landmark heatmaps and limiting excessive high-frequency noise. It can use either total variation or heatmap-gradient regularization, and is added to the heatmap loss during training.

D. Smoothness Regularization Loss

IV. EXPERIMENTS

The experiments evaluate FAHCD-Net across multiple datasets and include qualitative examples spanning challenging facial-image conditions.

IV. EXPERIMENTS

  • Experiments across multiple datasets are used to validate the effectiveness of FAHCD-Net.Source: §IV · p. 6
  • Qualitative results cover facial expressions with large poses and heavy occlusion.Source: §IV · p. 6
  • The qualitative scenarios also include illumination variation and blurred images.Source: §IV · p. 6

A. Datasets and Experimental Settings

The evaluation uses four public facial-landmark datasets with varied landmark counts and challenging conditions. It reports normalized error and failure rate, with image augmentation and fixed training settings.

A. Datasets and Experimental Settings

B. Evaluations under Normal Circumstances

FAHCD-Net reports strong performance on favorable-condition benchmarks and competitive results on occlusion-heavy datasets. The occlusion evaluation attributes its performance to diffusion-based noise resistance and HFA’s extraction of low-frequency structural information.

B. Evaluations under Normal Circumstances

  • The normal-circumstances evaluation uses the 300W Common and Fullsets and AFLW-frontal, which mostly contain clear images with minimal pose variation, illumination, occlusion, or noise.These benchmarks primarily represent favorable capture conditions.Source: B. Evaluations under Normal… · p. 7
  • NMEio 2.51 on the 300W Common Subset and 2.99 on the 300W Fullset, while AFLW-frontal NMEdiag is 1.17.The paper reports that the Common Subset and AFLW-frontal results outperform other methods; the Fullset result is described as commendable.Source: B. Evaluations under Normal… · p. 7, Table I · p. 7, Table IV · p. 8

C. Evaluation of Robustness against Occlusion

  • The occlusion evaluation uses COFW, the 300W Challenging Subset, and WFLW, datasets selected for significant occlusions.The paper frames this evaluation against the difficulty many methods have maintaining effectiveness on heavily occluded faces.Source: C. Evaluation of Robustness against… · p. 7
  • NMEip 4.65 on COFW and NMEio 4.49 on the 300W Challenging Subset; FAHCD-Net also shows exceptional performance on WFLW’s Make-Up and Occlusion Subsets.The authors attribute occlusion performance to diffusion-based noise resistance and HFA’s extraction of structural information from low-frequency signals.Source: C. Evaluation of Robustness against… · p. 7, Table II · p. 7, Table I · p. 7

D. Evaluation of Robustness against Large Poses and Expressions

FAHCD-Net reports accurate landmark detection under pose, expression, illumination, and blur variation. The section also examines mean-shape conditioning and how frequency controls affect generated heatmaps.

D. Evaluation of Robustness against Large Poses and Expressions

  • AFLW-full results are NMEdiag 1.30 and NMEbox 2.08; on the 300W Challenging Subset, FAHCD-Net reports NMEio 4.49 for pose and expression variation.The authors also report exceptional results on WFLW Pose and Expression Subsets, without giving values in these passages.Source: D. Evaluation of Robustness against… · p. 8

E. Evaluation of Robustness against Illumination and Blur

  • NMEio 4.29 on the 300W Challenging Subset is reported for illumination and blur; WFLW Illumination and Blur Subsets also show outstanding results.The authors attribute the improvement to HFA and SR suppressing high-frequency noise and aligning generated heatmap frequency distributions with ground truth.Source: E. Evaluation of Robustness against… · p. 8

1) Mean shape as Conditional Heatmap:

  • Replacing SHN-generated conditional heatmaps with a mean-shape prior causes a noticeable but moderate performance drop, indicating the framework does not strictly require high-quality initialization.The mean shape provides a coarse, stable spatial prior; the alternative also removes the need for an additional heatmap generator and simplifies training.Source: 1) Mean shape as Conditional Heatmap · pp. 8–9

2) Frequency Component Analysis:

  • Frequency analysis shows that the model without HFA produces high-frequency noise, while HFA reduces unnecessary components and SR promotes smoother, more continuous heatmaps linked to better localization accuracy.Fig. 7 presents heatmaps, high-frequency maps, PSD maps, and corresponding energy and NMEio values; the analysis compares frequency energy with localization error.Source: 2) Frequency Component Analysis · pp. 9–10, Figure 7 · p. 10

3) Different Smooth Regularization Item:

On 300W, the study compares TV and HG regularization as alternatives for the SR loss.

3) Different Smooth Regularization Item:

  • TV regularization significantly outperforms HG regularization, which tends to over-smooth heatmaps and blur crucial details.The authors suggest TV better balances noise suppression with detail preservation, maintaining sharp edges and local peaks in key areas.Source: 3) Different Smooth Regularization Item · p. 10

G. Ablation Study

The ablation study evaluates component contributions on 300W using the Inter-Occular setting. The conclusion presents FAHCD-Net as combining adaptive frequency control and smoothness regularization for robust landmark detection.

G. Ablation Study

  • Introducing the HFA module and SR loss significantly enhances robustness and prediction accuracy across the ablated configurations.The ablation is conducted on 300W with Inter-Occular evaluation; the supplied passages do not state the individual configuration values.Source: G. Ablation Study · p. 10

V. CONCLUSION

  • The conclusion attributes reduced overrecovered detail and redundant high-frequency noise to HFA and SR, and identifies combining frequency information with scenario-specific labels as future work.It describes the framework as cascading FAHCD with SR to support accurate detection in complex scenarios.Source: §V · p. 10

Related papers

Loading 2609.16842v1…