DeepPrivacy2: Towards Realistic Full-Body Anonymization

1. Introduction

• Face-focused anonymization leaves primary identifiers such as ears and gait, and secondary identifiers such as gender, unaddressed.The paper also notes that SG-GAN can have visual artifacts, leave original hair or accessories visible, and miss people farther from the camera.Source: §1 · p. 1
• FDH contains 1.5M human-figure images from YFCC100M, and the authors report that its larger scale greatly benefits generated-figure visual quality.The paper presents FDH as a large, diverse dataset for human-figure synthesis.Source: §1 · pp. 1–2
• The framework combines detections from multiple modalities and uses separate anonymizers for CSE-detected figures, other figures, and remaining faces.Each category uses an inpainting GAN; the paper reports that the GAN generates high-quality, diverse identities with few task-specific modeling choices.Source: §1 · p. 2
• The face anonymizer uses no pose guidance and supports text-guided attribute editing through semantically meaningful directions in the style-based generator's latent space.The face GAN is trained on an updated Flickr Diverse Faces dataset.Source: §1 · p. 2
• DeepPrivacy2 surpasses prior state-of-the-art realistic anonymization methods in image quality and anonymization guarantees.The paper reports extensive qualitative and quantitative evaluation.Source: §1 · p. 2

2. Related Work

• Naive masking, blurring, and pixelation degrade image quality, while K-same methods provide better privacy and usability than naive methods but still generate highly corrupted images.Learning-based anonymization can retain usability through realistic synthesis; inpainting methods provide stronger privacy guarantees than transformative methods because they never observe the original sensitive information.Source: §2 · p. 2
• Most deep-generative anonymization methods target faces, leaving body identifiers such as ears and gait, and secondary identifiers such as gender, untouched.Prior full-body methods are limited to low-resolution images or generate visual artifacts.Source: §2 · p. 2
• Prior full-body synthesis often transfers a source appearance across poses, garments, or motion, whereas this anonymization approach does not transfer a source appearance.Many such methods use low-variance datasets or do not handle large background-context variation.Source: §2 · p. 2

3. The Flickr Diverse Humans Dataset

• FDH contains 1.53M human-figure images from YFCC100M, with one centered figure per image and 288×160 resolution.The split contains 1,524,845 training images and 30K validation images.Source: §3 · pp. 2–3
• Each image includes CSE dense-pose estimates, 17 keypoint annotations, and a segmentation mask formed from CSE and Mask R-CNN detections.The dataset is automatically filtered using confidence thresholds, image-quality assessment, visible body-part count, and overlap between keypoint and CSE predictions.Source: §3 · p. 3
• Compared with previously adopted full-body synthesis datasets, FDH is much larger and contains diverse people from in-the-wild images.It is less curated than typical generative-modeling datasets, retaining unusual poses, perspectives, lighting, and contexts so anonymization can handle them.Source: §3 · p. 3

4.1. Detection

• The ensemble uses DSFD face detection, CSE dense-pose estimation, and Mask R-CNN instance segmentation, with separate categories for CSE-detected figures, figures without CSE detections, and remaining faces.For figures, the anonymized region is the union of Mask R-CNN and CSE segmentations; fallback detectors improve recall when CSE misses people, while tracking preserves identity across video frames.Source: §4.1 · p. 3, Figure 3 · p. 3

4.2. Synthesis Method

• Each detection category uses an independently trained generator that inpaints removed regions using the visible image, mask, and latent input.For the CSE-guided generator, the CSE embedding is also concatenated to the generator input.Source: §4.2 · p. 3

4.3. Recursive Stitching

• Ascending recursive stitching places foreground objects last, reducing visual artifacts at borders between overlapping individuals.The reverse order can let background objects overwrite foreground objects when detections overlap.Source: Figure 4 · p. 4, §4.3 · p. 4

4.2.1 A Style-Based U-Net Generator

• The generator pairs a U-Net context encoder with a StyleGAN2-style decoder, using instance normalization in skip connections and before style modulation.The model increases U-Net depth to five downsampling layers and removes V-SAM and discriminator surface supervision used in SG-GAN.Source: §4.2.1 · p. 4
• The face generator uses no keypoints, improving detection recall when keypoints are difficult to detect, and is trained on FDF at 256×256 rather than 128×128 resolution.The full-body generators are trained on FDH at 288×160 resolution, with separate versions with and without CSE embeddings.Source: §4.2.1 · p. 4

5. Experimental Evaluation

• The CSE-guided generator processes ∼11.6 FPS and the face generator ∼7.9 FPS on an NVIDIA 1080 8GB GPU; the full pipeline takes ∼2.8 seconds for 12 people.SG-GAN processes ∼7.3 FPS on the same GPU; the authors attribute their improved runtime to removing V-SAM and moving most parameters to lower-resolution layers.Source: §5 · p. 5

5.1. Synthesis Quality

• FID=5.6 and FIDCLIP=1.7 for CSE-guided generation, versus FID=6.1 and FIDCLIP=2.30 for unconditional generation.The section also reports that the model trained on COCO-Body starts to overfit early, with diverging discriminator logits and increasing FID.Source: Figure 5 · p. 5, §5.1 · p. 6, Figure 9 · p. 6
• At 128 × 128 on FDF, DeepPrivacy2 achieves FID=0.56 versus DeepPrivacy’s FID=0.68, without face keypoints; direct comparison on FDF256 is not straightforward.FDF256 uses 256 × 256 images and filters out lower-resolution images from the original dataset.Source: Figure 6 · p. 5, §5.1 · p. 6

5.2. Anonymization Evaluation

• DeepPrivacy2 provides anonymization guarantees similar to masking and significantly better than pixelation; SG-GAN performs worse, with detector failures and poor accessory/hair segmentation cited as causes.Pixelation still enables re-identification of several individuals.Source: Table 1 · p. 7, §5.2 · p. 7

5.2.1 Training and Evaluating on Anonymized Data

• Keypoint AP on COCO improves over traditional methods with DeepPrivacy2, while pixelation greatly affects training; CSE misses many individuals, hurting pose preservation with the unconditional generator.The result is reported in Table 2 on the COCO validation set.Source: Table 2 · p. 7, §5.2.1 · p. 7
• DeepPrivacy2 improves over pixelation and mask-out on Cityscapes instance segmentation, but the gap is less prevalent than for keypoint estimation.The authors suggest model weight initialization may explain the smaller gap.Source: §5.2.1 · p. 7, Table 3 · p. 8
• Removing CSE guidance severely hurts performance, especially when anonymized data is used for training.Source: §5.2.1 · pp. 7–8

6. Conclusion

• DeepPrivacy2 substantially improves image quality and privacy guarantees while supporting realistic anonymization of human figures and faces.The authors also report improved image quality and diversity for in-the-wild human figure synthesis and text-guided control of generated faces.Source: §6 · p. 8
• The framework is intended to support privacy-sensitive image collection while retaining image quality, but realistic human synthesis also has misuse potential, including DeepFakes.The authors point to detection challenges and model watermarking as mitigation efforts.Source: §6 · p. 8

6.1. Limitations

• The generator samples a limited, condition-shaped set of identities, with context such as a baseball field favoring a baseball-player identity.Figure 11 illustrates the generator sampling from a small subset of identities given the condition.Source: §6.1 · p. 8, Figure 11 · p. 8
• Anonymization cannot be guaranteed without human supervision because detectors can fail, and dense-pose descriptions may leave gait-based recognition possible.Detectors from different modalities are used to improve detection when one or more fail.Source: §6.1 · p. 8
• Incorrect pose descriptions severely degrade image quality, and full-body GAN edits do not transfer semantically consistently across poses and background contexts.The authors also report that the full-body GAN is harder to edit than the face GAN.Source: §6.1 · p. 8

A. Flickr Diverse Humans Dataset

• FDH retains detections only after they pass the DeepPrivacy2 detection pipeline's listed quality filters.The criteria include confidence, resolution, grayscale, automatic quality assessment, CSE vertices, segmentation overlap, and keypoint-to-CSE correspondence.Source: A. Flickr Diverse Humans Dataset · p. 9
• Detections are filtered when confidence is below 0.98, image area is below 144 × 80, images are grayscale, or NIMA score is below 3.The NIMA score comes from an open-source implementation.Source: A. Flickr Diverse Humans Dataset · p. 9
• CSE-based screening removes detections with fewer than 135 average unique vertices per body part or Mask R-CNN–CSE IoU below 0.5.The vertex count uses 26 body-part regions.Source: A. Flickr Diverse Humans Dataset · p. 9
• Keypoint-to-CSE filtering retains detections only when at least 8 keypoints match their corresponding semantic body parts.Keypoints are predicted by a pre-trained Keypoint R-CNN on COCO.Source: A. Flickr Diverse Humans Dataset · p. 9

B. The Updated Flickr Diverse Faces Dataset

• FDF256 derives from 1.08M YFCC100M images, with 242,031 training images and 6,533 validation images.Its dataset generation follows the original FDF procedure.Source: B. The Updated Flickr Diverse Faces… · p. 9
• Faces are resized to 256 × 256, and faces with original resolution below 64 × 64 are filtered out.Source: B. The Updated Flickr Diverse Faces… · p. 9
• Each face receives keypoints from a pre-trained Keypoint R-CNN R50-FPN and a bounding box from the official DSFD implementation.Source: B. The Updated Flickr Diverse Faces… · p. 9

C. Experimental Details

• All networks use Adam with batch size 32 and learning rate 0.002, and training uses only random horizontal flips for augmentation.The discriminator follows SG-GAN except that feature-pyramid-network layers are removed.Source: C. Experimental Details · p. 9
• Market1501 evaluation anonymizes 19,732 test images and uses the 3,368 original query images for re-identification.The reported mAP/R1 calculation includes test examples where no individual is detected.Source: C. Experimental Details · p. 9
• On Market1501, CSE accounts for 45% of all detections, Mask R-CNN for 52%, and DSFD for the remainder; no detector detects any individual in 10% of the 19,732 test images.Source: C. Experimental Details · p. 9

• Randomly selected Cityscapes images are shown in Figures 12, 13, and 14.Source: D. Random Anonymized Examples · p. 9
• Randomly selected COCO images are shown in Figures 15, 16, and 17.Source: D. Random Anonymized Examples · p. 9
• Randomly selected FDH images are shown in Figures 18, 19, and 20.Source: D. Random Anonymized Examples · p. 9
• Randomly selected FDF256 images are shown in Figures 21, 22, and 23.Source: D. Random Anonymized Examples · p. 9