Source-linked AI summary
DeepPrivacy: A Generative Adversarial Network for Face Anonymization
Håkon Hukkelås, Rudolf Mester, Frank Lindseth
TL;DR
Face anonymization needs to remove identity information without degrading realistic image structure. DeepPrivacy uses a conditional GAN that generates faces from privacy-safe pose and background information, and its evaluation retains 99.3% of original average precision on anonymized WIDER-Face images. The paper also introduces the diverse FDF dataset for training and evaluation.
Problem
Face anonymization must remove all privacy-sensitive information while retaining the original data distribution and realistic image quality.
Method
DeepPrivacy is a conditional GAN whose generator uses background and sparse pose annotations while never observing the original face.
Results
99.3% of the original average precision is retained on anonymized WIDER-Face validation images.
Takeaways & Limitations
The model generates high-quality anonymized images across diverse backgrounds and poses while preserving face-detection performance.
Takeaways & Limitations
Non-traditional poses can cause corrupted faces when using sparse pose estimation.
Abstract
from arXiv · showhide
We propose a novel architecture which is able to automatically anonymize faces in images while retaining the original data distribution. We ensure total anonymization of all faces in an image by generating images exclusively on privacy-safe information. Our model is based on a conditional generative adversarial network, generating images considering the original pose and image background. The conditional information enables us to generate highly realistic faces with a seamless transition between the generated face and the existing background. Furthermore, we introduce a diverse dataset of human faces, including unconventional poses, occluded faces, and a vast variability in backgrounds. Finally, we present experimental results reflecting the capability of our model to anonymize images while preserving the data distribution, making the data suitable for further training of deep learning models. As far as we know, no other solution has been proposed that guarantees the anonymization of faces while generating realistic images.
1 Introduction
DeepPrivacy addresses face anonymization by replacing original faces with realistic, context-fitting faces without destroying the original data distribution. It combines privacy-safe conditional generation, a diverse face dataset, and evaluation showing near-preserved face-detection performance.
- Method: DeepPrivacy replaces faces using background and sparse pose information while never observing the original face.The conditional GAN uses a U-net generator and progressive training from 8 × 8 to 128 × 128 resolution.
- Dataset: FDF provides 1.47M annotated faces spanning diverse poses, occlusions, backgrounds, and identities.Each face has a tight bounding box and keypoint annotation.
- Evaluation: 99.3% of the original average precision is retained by DSFD on anonymized WIDER-Face images, versus 96.7% for pixelation, 90.5% for heavy blur, and 41.4% for black-out.The result averages across easy, medium, and hard difficulty levels.
- Goal: The model targets complete removal of privacy-sensitive information while maintaining realistic faces and seamless transitions across poses and contexts.The practical requirements include broad diversity in images, poses, backgrounds, and persons.
2 Related Work
Prior de-identification methods alter or complete image regions but may leave privacy information, artifacts, or insufficiently large masked areas. DeepPrivacy builds on GAN-based generation and progressive training while conditioning on context and pose rather than preserving the original face directly.
- Face de-identification: Pixelation and blurring are broadly applicable but inadequate, while existing methods do not assure removal of all privacy-sensitive information.The related work identifies a limited number of studies addressing face de-identification.
- Face de-identification: K-same can remove all privacy-sensitive information, but small alignment errors often produce ghosting artifacts.This illustrates a privacy-quality trade-off in prior anonymization.
- Face de-identification: Prior grayscale de-identification preserves facial attributes, whereas DeepPrivacy generates a context-coherent face from conditional information.DeepPrivacy does not directly train the generator to reproduce attributes from the original image.
- GANs: GANs model natural image distributions and generate images often indistinguishable from real data, motivating their use for anonymization.The paper applies GANs to remove privacy-sensitive information without destroying image quality.
- GAN-based anonymization: Unlike pixel-altering video anonymization, DeepPrivacy never observes the original face, supporting removal of all privacy-sensitive information.Its generator instead uses privacy-safe conditional inputs.
- GAN architecture: Progressive growing improves image quality and training time, while DeepPrivacy modifies it into a conditional GAN using context and pose.The resulting generator is also similar to the architecture proposed by Isola et al.
- Image inpainting: Existing face-completion GANs mask regions too small to remove all privacy-sensitive information, making larger masked areas more demanding.The passage identifies face completion as closely related to the anonymization task.
3 The Flickr Diverse Faces Dataset
Flickr Diverse Faces is a 1.47M-face dataset designed for varied face-anonymization conditions. It combines annotations with diversity in pose, occlusion, background, age, ethnicity, and identity, though FFHQ has higher resolution.
- Dataset characteristics: FDF contains 1.47M faces at a minimum 128 × 128 resolution, each with facial keypoints and a bounding box.The faces were crawled from YFCC-100M and extracted from 1.08M images.
- Dataset diversity: FDF spans age, ethnicity, facial pose, background, and occlusion, with scenes from traffic, sports events, and outdoor activities.Its examples are intended to represent varied real-world contexts.
- Dataset comparison: Compared with FFHQ and Celeb-A, FDF offers more facial-pose diversity and significantly more faces, while FFHQ provides higher resolution.The comparison concerns dataset scale, pose diversity, and image resolution.
- Annotations: FDF annotations are automatically produced with high-confidence state-of-the-art keypoint and bounding-box models, yielding few annotation errors.Keypoint and box predictions are matched for each face.
- Annotations: Each image includes a sparse seven-keypoint face annotation and a tight bounding box.The seven keypoints describe the face pose.
4 Model
DeepPrivacy is a conditional GAN that generates 128 × 128 anonymized faces from background and sparse pose information without observing the original face. Its U-net generator progressively grows resolution while injecting pose and background information throughout the architecture.
- Conditional generation: Seven keypoints—eyes, ears, shoulders, and nose—are encoded as a one-hot pose image for conditioning.The encoding uses K × M × M dimensions, where K is the number of keypoints and M is the target resolution.
- Progressive growing: Progressive growing doubles network resolution from 8 × 8 to 128 × 128 and supplies pose information at every resolution.New convolutional blocks and transition phases are added as resolution increases, improving training stability relative to training without progressive growing.
- Generator architecture: The U-net architecture preserves background information through skip connections between encoder and decoder layers.A 1 × 1 bottleneck convolution after each skip connection reduces decoder parameters.
- Generator architecture: After each upsampling layer, the generator concatenates the upsampled output with pose information and the corresponding skip connection.Convolutional layers use pixel normalization and LeakyReLU with α = 0.2.
- Discriminator architecture: The discriminator receives background and pose information conditionally, while alternative deep and wide designs address the generator’s larger parameter count.The deep discriminator uses additional layers and residual blocks; the wide version increases filters per convolution.
5 Experiments
Experiments evaluate whether DeepPrivacy preserves face-detection performance and image quality after anonymization. On WIDER-Face, detection retains nearly all original AP, while ablations examine pose conditioning, discriminator design, and model size; evaluation is limited by incomplete anonymization coverage.
- Qualitative evaluation: Every face is generated in the qualitative DeepPrivacy examples, which cover diverse poses, backgrounds, and persons without WIDER-Face training images.The model is trained on FDF and evaluated qualitatively on WIDER-Face.
- Face detection: AP decreases by 0.7%, 0.7%, and 0.6% on easy, medium, and hard difficulties, respectively.Table 1 reports face-detection AP on the WIDER-Face validation set using DSFD.
- Anonymization comparisons: DeepPrivacy generally achieves higher AP than black-out, blurring, and pixelation, except for 16 × 16 pixelation.The exception is partly explained because 16 × 16 pixelation leaves many faces unaffected, especially in the hard challenge.
- Evaluation boundary: Only 43% of WIDER-Face validation faces can be anonymized because automatically detected keypoints are unavailable for many occluded, low-resolution, or turned-away faces.Among non-anonymized faces, 22% are partially occluded, 30% heavily occluded, and 70% are smaller than 14 × 14.
- Ablation experiments: Pose conditioning significantly improves FID-based image quality on FDF, whereas a deeper discriminator struggles to converge and produces poorer results.The ablation table reports FID after discriminator training, with lower values indicating better quality.
- Ablation experiments: Increasing model size from 12M to 46M parameters generally improves image quality when other hyperparameters remain unchanged.The model-size ablation changes only the number of filters in each convolution.
6 Limitations
DeepPrivacy generates realistic anonymized faces across diverse backgrounds and poses, but image quality degrades in challenging cases such as heavy occlusion and irregular poses.
- DeepPrivacy can generate unrealistic images with high occlusion, difficult backgrounds, and irregular poses.These failure cases are illustrated in Figure 6.
- Faces occluded by high-fidelity objects, such as hands covering the face, are especially difficult to render realistically.The authors attribute this difficulty to the complex semantic reasoning required, which remains challenging for GANs.
- Non-traditional poses can cause DeepPrivacy to generate corrupted faces.The model uses sparse pose estimation; denser pose estimation would likely improve irregular-pose performance but restrict the practical use case.
7 Conclusion
DeepPrivacy is a conditional GAN for anonymizing faces without destroying the original data distribution. On WIDER-Face, it achieves 99.3% of the original average precision, while irregular poses, difficult occlusions, complex backgrounds, and video consistency remain open challenges.
- DeepPrivacy is a conditional generative adversarial network designed to anonymize faces without destroying the original data distribution.
- 99.3% of the original average precision is achieved on the anonymized WIDER-Face validation set.The result is reported for the current state-of-the-art face detection method.
- Ablation experiments suggest that larger model size and sparse pose information are necessary to generate high-quality images.These experiments were conducted on the FDF dataset.
- Irregular poses, difficult occlusions, complex backgrounds, and temporal consistency in videos remain subjects for further work.
Appendix A - Training Details
The model uses specified batch sizes, learning-rate, optimizer, and progressive-training phases, with exponential weight averaging to improve image quality.
- The model uses a learning rate of 0.00175 with the Adam optimizer.
- Each network expansion has a 1.2M-image transition phase and a 1.2M-image stabilization phase.
- An exponential running average of generator weights is used because it improves overall image quality.
- The final model was trained for 17 days on two NVIDIA V100-32GB GPUs.
Image Pre-Processing
The pre-processing pipeline crops and normalizes each detected face region, masking the face pixels before resizing the input to 128×128.
- For each detected face, the pipeline finds the smallest square bounding box surrounding the face bounding box.
- The expanded bounding box is resized to the target size of 128×128.
- Pixels inside the face bounding box are replaced with the constant value 128.
- Pixel values are shifted to the range [−1, 1].
Tensor Core Modifications
The implementation modifies the network and input pipeline to support efficient tensor-core training while preserving privacy-safe face generation. Mixed-precision training increases training speed substantially.
- Convolutional blocks use filter counts divisible by 8, and each GPU batch size is also divisible by 8 to meet tensor-core requirements.
- Detected faces are cropped to quadratic images, privacy-sensitive regions are replaced with a constant, and keypoints are encoded as a one-hot image before generation.
- 220% training-speed improvement is achieved with automatic mixed-precision training.