PaperlayerView exact revision on arXiv

Source-linked AI summary

HYDRO: Towards Non-Reversible Face De-Identification Using a High-Fidelity Hybrid Diffusion and Target-Oriented Approach

Felix Rosberg, Vitomir Štruc, Cristofer Englund, Eren Erdal Aksoy, Fernando Alonso-Fernandez

arXiv:2609.27011v1cs.CV

TL;DR

Target-oriented face de-identification can preserve subtle identity cues that make its outputs vulnerable to reconstruction. HYDRO combines target-oriented generation with noise and diffusion-based recovery; across three datasets it reports strong performance and an 85.7% reduction in reconstruction-attack success versus the closest competitor.

  • Problem

    Target-oriented methods can retain subtle identity cues that adversarial models exploit to reverse de-identification.

    Source: §I · p. 1

  • Method

    HYDRO de-identifies faces, adds noise to conceal residual identity information, and uses backward diffusion to improve fidelity, with an Eye Similarity Discriminator for gaze retention.

    Source: §I · p. 1, C. Reconstruction Robustness through… · p. 3, E. Loss Functions and Training Procedure · p. 5

  • Results

    HYDRO reports SOTA performance across three datasets and reduces reconstruction-attack success by 85.7% versus the closest competitor, while being the only target-oriented method resilient to reconstruction.

    Source: §I · pp. 1–2

  • Takeaways & Limitations

    The approach targets gaze and eye-region fidelity alongside de-identification, characteristics the paper identifies as important to naturalness and visual-data utility.

    Source: §I · p. 1

Abstract

from arXiv · show

Target-oriented face de-identification models aim to anonymize the identity of a target individual across different images or video frames, such that the target can no longer be reliably recognized, while maintaining key characteristics of the visual data. Such models commonly leverage generative encoder-decoder architectures to manipulate facial appearances, enabling them to produce realistic high-fidelity de-identification results, while ensuring considerable attribute-retention capabilities. However, target-oriented models also carry the risk of inadvertently preserving subtle identity cues, making them (potentially) reversible and susceptible to reconstruction attacks. To address this problem, we introduce in this paper a novel (robust) face de-identification approach, called HYDRO, that combines target-oriented models with a dedicated diffusion process specifically designed to destroy any imperceptible information that may allow learning to reverse the de-identification procedure. HYDRO first de-identifies the given face image, injects noise into the de-identification result to impede reconstruction, and then applies a diffusion-based recovery step to improve fidelity and minimize the impact of the noising process on the data characteristics. To further improve image fidelity and better retain gaze directions, a novel Eye Similarity Discriminator (ESD) is also introduced and incorporated it into the training of HYDRO. Extensive quantitative and qualitative experiments on three diverse datasets demonstrate that HYDRO exhibits state-of-the-art (SOTA) fidelity and attribute-retention capabilities, while being the only target-oriented method resilient against reconstruction attacks. In comparison to multiple SOTA competitors, HYDRO reduces the success of reconstruction attacks by 85.7% on average.

I. INTRODUCTION

HYDRO addresses the tension between privacy protection and useful, realistic face imagery, while targeting a reconstruction weakness in target-oriented methods. It combines target-oriented de-identification with diffusion-based noise and recovery, and reports strong fidelity, attribute retention, and reconstruction-attack resilience.

I. INTRODUCTION

  • Face de-identification must obscure identity while preserving useful attributes, but target-oriented models can retain subtle identity cues that adversarial models exploit to reverse de-identification.Earlier low-level techniques often struggle with realism, fidelity, and attribute retention, reducing data utility.Source: §I · p. 1

III. METHOD

  • HYDRO conditions generation on identity-encoder embeddings to move the output identity away from the input, then uses noise to conceal residual identity traces and diffusion recovery to restore fidelity.The Eye Similarity Discriminator also targets gaze-direction retention and eye-region fidelity.Source: §I · p. 1

I. INTRODUCTION

  • Across three datasets, HYDRO is reported SOTA in de-identification, fidelity, attribute retention, and attack resilience; it is the only robust target-oriented method and cuts attack success 85.7% versus the closest competitor.The comparison is against multiple SOTA baselines.Source: §I · pp. 1–2

II. RELATED WORK

  • Prior work spans inpainting, GAN inversion, target-oriented, and diffusion approaches, with trade-offs in attribute retention, realism, computation, or vulnerability to reconstruction attacks.Inpainting may omit facial semantics; GAN inversion faces projection challenges; diffusion approaches can require multiple sampling steps.Source: §II · p. 2

A. Overview of HYDRO

  • HYDRO combines a target-oriented generator, a conditional identity encoder, and a diffusion model, trained with global and eye-similarity discriminators.The Eye Similarity Discriminator is intended to improve eye-region fidelity and gaze retention.Source: A. Overview of HYDRO · p. 2, §III · p. 2

B. Target-Oriented Face De-Identification

  • A U-Net generator produces a de-identified face and learned blending mask, which combines generated and input content to retain attributes, background, and quality under occlusion.The mask is learned jointly with de-identification in an end-to-end manner.Source: B. Target-Oriented Face… · p. 3

C. Reconstruction Robustness through Diffusion

HYDRO adds a low-strength noise step to an initial de-identified image to obscure residual identity traces, then learns backward diffusion to restore fidelity. Exact recovery is described as requiring access to the noisy image, applied noise, and diffusion model.

C. Reconstruction Robustness through Diffusion

  • A single low-strength forward-diffusion step was empirically sufficient to minimize information loss and achieve competitive performance while limiting sampling time.The noise is added to the generator's initial de-identified output to remove residual identity traces.Source: C. Reconstruction Robustness through… · p. 3
  • The learned backward diffusion step improves fidelity and recovers a realistic image after noise degrades fidelity and low-level characteristics.The recovered image is finally blended with the input face.Source: C. Reconstruction Robustness through… · pp. 3–4
  • The privacy-strength discussion assumes attackers may know the noise schedule, while noting that additional diffusion-process information is needed for exact recovery.The passage frames this as a worst-case assumption about schedule knowledge.Source: C. Reconstruction Robustness through… · p. 4
  • Precise recovery of the initial de-identified image is described as infeasible without the noisy sample, exact applied noise, and diffusion model.This account concerns exact recovery of the initial image, not a quantified attack-success rate.Source: C. Reconstruction Robustness through… · p. 4

D. Eye Similarity Discriminator (ESD)

The Eye Similarity Discriminator (ESD) contrasts paired left- and right-eye features, and its similarity signal helps train the generator to improve eye quality and gaze retention. The generator combines identity, attribute, adversarial, and eye-similarity objectives, applying losses to both initial and blended outputs.

D. Eye Similarity Discriminator (ESD)

E. Loss Functions and Training Procedure

IV. RECONSTRUCTION-ATTACK MODEL

The reconstruction-attack model probes whether de-identified images retain information that can recover the target identity. It is trained in a black-box setting on paired original and de-identified images, including HYDRO outputs after diffusion.

Reconstruction Attack Model Architecture

  • The attack model is an unconditional U-Net with one output head that maps a de-identified image toward the original target face.Unlike the generator and diffusion model, it is not conditioned on identity or time-step information and does not return a mask.Source: §IV · p. 5

Reconstruction-Attack Model Training

  • The attack model simulates a black-box adversary with access to many input-output pairs, and HYDRO's attack model is trained on outputs after diffusion.A separate reconstruction model is trained for each compared de-identification method.Source: §IV · p. 5
  • Training combines pixel-level L1 reconstruction, perceptual similarity, and identity losses to recover target appearance and identity.The perceptual loss compares VGG16 feature maps, while the L1 term encourages low-level pixel similarity.Source: §IV · pp. 5–6

V. EXPERIMENTS AND RESULTS

Experiments assess visual fidelity, attribute retention, identity retrieval, and resilience to reconstruction attacks across FaceForensic++, CelebA, and LFW. Qualitative comparisons show HYDRO providing reliable de-identification after attack in the illustrated cases.

V. EXPERIMENTS AND RESULTS

  • Evaluation covers fidelity, pose, expression and gaze retention, identity retrieval, and identity retrieval after reconstruction attacks.Fidelity uses FID; attribute measures include mean L2 distances and image MSE, while identity retrieval is tested at three FARs with five recognition models.Source: §V · p. 6
  • The study evaluates FaceForensic++, CelebA, and LFW, using dataset-wide matching for the first two and genuine-pair protocol for LFW.FaceForensic++ samples 10 frames per video and separates videos with multiple people into identities.Source: §V · p. 6
  • In the qualitative comparison, HYDRO is the only method shown to provide reliable de-identification after reconstruction attacks.The figure places de-identification outputs before attack on the left and recovered outputs on the right.Source: Figure 4 · p. 7

A. Quantitative Results

HYDRO leads the reported comparisons in fidelity and attribute retention, and shows substantial resilience to reconstruction attacks. Qualitative comparisons show strong eye quality and preservation of non-face attributes, while challenging poses and large occlusions can still produce artifacts.

A. Quantitative Results

  • HYDRO lowers FID, shares the strongest average identity protection with RiDDLE and FIVA, improves pose, expression, and gaze retention, and uses more GFLOPS than the lightest methods.HYDRO is also the top performer on MSE, generally comparable to FIVA and G2Face while outperforming the rest.Source: A. Quantitative Results · p. 7
  • 85.7% average reduction in reconstruction-attack success versus RiDDLE, and 94.1% versus FIVA across three FAR scores.HYDRO’s attack model was trained on samples that included the diffusion step.Source: A. Quantitative Results · p. 7

B. Qualitative Results

C. Ablation Studies

The ablations show how HYDRO’s components balance reconstruction resistance, fidelity, gaze, and attribute retention. The conclusion also notes the computational cost of diffusion and the persistent trade-off between de-identification and attribute retention.

C. Ablation Studies

  • Adding the eye-similarity loss improves gaze retention, while diffusion recovery improves general attribute preservation and visual quality without harming de-identification.The reported gaze improvement and maintained FID indicate improved synthetic-eye quality; gaze loss alone also improves gaze retention, with further gains from ESD.Source: C. Ablation Studies · p. 10
  • Without noise or diffusion, reconstruction retrieves original identity 99% of the time; noise strength 0.0414 prevents attacks but degrades fidelity and attribute retention.The attack comparison is on FaceForensic++; the table caption identifies its noise and diffusion configurations.Source: C. Ablation Studies · p. 10, Table IV · p. 11
  • Applying diffusion mitigates reconstruction attacks and enhances visual quality and attribute retention, but adds GFLOPS and inference time.The reported computational overhead is evidenced in Tab. III.Source: C. Ablation Studies · p. 10
  • Disabling mask regularization or L1 reconstruction can improve FID but reduce attribute retention on CelebA and LFW, while worsening FID and gaze retention on FaceForensic++.The loss ablations cover FaceForensic++, LFW, and CelebA, and include full HYDRO for comparison.Source: C. Ablation Studies · pp. 10–11, Table V · p. 11

VI. CONCLUSION

  • HYDRO reports state-of-the-art de-identification, attribute retention, and fidelity alongside resilience to reconstruction attacks; diffusion raises GFLOPS and computation time.The conclusion says HYDRO reduces, but does not eliminate, the trade-off between attribute retention and de-identification.Source: §VI · p. 11

A . DISCRIMINATORS

The supplement describes HYDRO’s global and eye-similarity discriminators, then details its pixel-space diffusion model. The diffusion model uses a DDPM training schedule and one-step sampling in the final pipeline.

A . DISCRIMINATORS

  • The global discriminator and ESD use hinge adversarial training and gradient-penalty regularization; ESD additionally uses the eye-similarity loss.Fig. 11 depicts the two discriminator architectures.Source: A . DISCRIMINATORS · p. 12, Figure 11 · p. 12

B . DIFFUSION MODEL

  • HYDRO’s diffusion model is a U-Net with residual and attention blocks; unlike generator G, it processes the identity vector through an MLP and uses time-step embeddings.Fig. 12 shows the diffusion-model architecture.Source: B . DIFFUSION MODEL · p. 12, Figure 12 · p. 12
  • The diffusion model is trained in pixel space as an epsilon-prediction model using a DDPM schedule, and experiments use one-step DPM-Solver++ sampling with low-strength noise.The passage says this sampling setting is used in all experiments.Source: B . DIFFUSION MODEL · pp. 12–13
  • Forward diffusion adds Gaussian noise according to the linear beta schedule, and the reverse process is learned by modeling the noise in the sample.The passage defines alpha_t from beta_t and gives the forward noising equation.Source: B . DIFFUSION MODEL · p. 13

C . RECONSTRUCTION-ATTACK MODEL

The reconstruction-attack model probes whether de-identified outputs retain recoverable identity information. It is trained in a black-box setting on paired original and de-identified images, including HYDRO outputs after diffusion.

C . RECONSTRUCTION-ATTACK MODEL

  • The attack model R probes whether competing de-identification methods, including HYDRO, are vulnerable to reconstruction attempts.For HYDRO, the attack is trained on samples after diffusion has been applied.Source: C . RECONSTRUCTION-ATTACK MODEL · p. 13
  • R is an unconditional U-Net with one output head that predicts the pre-de-identification target image, which may retain traces of the original identity.The model does not use identity or time-step conditioning and does not return a mask.Source: C . RECONSTRUCTION-ATTACK MODEL · p. 13
  • R learns to map a de-identified image to an approximation of its original target image, expected to preserve the target’s identity information when reconstruction succeeds.The model is trained on paired target images and their de-identified counterparts in a black-box setting simulating access to many input-output examples.Source: C . RECONSTRUCTION-ATTACK MODEL · p. 13
  • Training combines L1 reconstruction, perceptual, and identity losses; the perceptual loss compares VGG16 feature maps of recovered and original images.The perceptual loss uses a VGG16 model pretrained on ImageNet and omits the linear layers at each feature-map scale.Source: C . RECONSTRUCTION-ATTACK MODEL · p. 13

D . GENERATOR LOSSES

The generator’s attribute-preservation losses seek to retain input-image characteristics in de-identified outputs. The supplement defines losses for pixel similarity, identity-encoder features, pose and expression, gaze, and mask regularization.

D . GENERATOR LOSSES

  • The attribute-preservation loss combines terms intended to retain information from input image x in de-identified output xd.The supplement provides formal definitions of these generator-loss components.Source: D . GENERATOR LOSSES · p. 14
  • The L1 reconstruction loss penalizes pixel-level differences between input image x and de-identified output xd.The passage identifies the L1 norm as the distance measure.Source: D . GENERATOR LOSSES · p. 14
  • The feature-similarity loss compares ArcFace identity-encoder features of x and xd to minimize higher-level perceptual differences.The supplement describes comparing intermediate representations across selected model layers.Source: D . GENERATOR LOSSES · p. 14
  • The L1 distance loss between expression and pose coefficients encourages retention of pose and expression in de-identified images.The coefficients are extracted with a 3DMM encoder.Source: D . GENERATOR LOSSES · p. 14
  • The loss set also includes L1 gaze loss between left- and right-eye heatmaps and L1 mask regularization to limit blending-mask complexity.The gaze heatmaps are extracted by a heatmap-based landmark encoder.Source: D . GENERATOR LOSSES · p. 14

E . REPRODUCIBILITY

HYDRO and its diffusion model use separate training configurations, with the diffusion model trained longer and using a specified DDPM noise schedule.

A. Implementation Details

  • HYDRO and its ablations use AdamW at 5e−5 for 50,000 iterations on 256×256 VGGFace2 images, with batch size 10 and mixed precision.Gradients are clipped at norm 1; the seed is 42.Source: A. Implementation Details · p. 14
  • The diffusion model uses AdamW at 1e−4 and is trained for 150,000 iterations on 256×256 VGGFace2 images.It uses batch size 10, mixed precision, and gradient clipping at norm 1.Source: A. Implementation Details · p. 14
  • Diffusion training uses a 1,000-step linear DDPM schedule, with βt increasing from 0.00085 to 0.012 and noise prediction as its prediction type.The diffusion model uses weight decay 1e−6 and epsilon 1e−8.Source: A. Implementation Details · p. 14

B. Identity Retrieval Details

These supplementary sections detail identity-retrieval and FID evaluation, qualitative video results, and inference costs. They also note that FID is bias-susceptible and that HYDRO’s diffusion step increases inference time.

B. Identity Retrieval Details

  • Identity-retrieval thresholds are derived from genuine and imposter pairs sampled from the VG-GFace2 training set, with Table VI documenting thresholds by FAR, backbone, and dataset.The evaluation lists CosFace, ArcFace, FaceNet, AdaFace, and ElasticFace.Source: B. Identity Retrieval Details · p. 14, Table VI · p. 15

C. Fr´echet Inception Distance Details

F . VIDEO RESULTS

  • HYDRO produces convincing video results on challenging poses and low-resolution faces, with less evident but still minute flickering than competitors.G2Face shows frame inconsistencies and struggles with side profiles; FIVA has difficulties with off-pose or small faces and visible chin and eye artifacts.Source: F . VIDEO RESULTS · p. 15

G . INFERENCE

  • HYDRO is almost 2.3 times slower than FIVA, the fastest target-oriented method at 256×256, while the authors describe this as a trade-off for privacy protection and reconstruction resilience.DeepPrivacy and FIT are the fastest methods; DeepPrivacy has poor attribute retention, while FIT’s lower resolution leads to colorization artifacts.Source: Table VII · p. 15, G . INFERENCE · pp. 15–16
Loading 2609.27011v1…