Source-linked AI summary
CIAGAN: Conditional Identity Anonymization Generative Adversarial Networks
Maxim Maximov, Ismail Elezi, Laura Leal-Taixé
TL;DR
Computer vision increasingly processes personal visual data, creating a need for privacy-preserving processing that retains task-relevant information. CIAGAN uses conditional generative adversarial networks with identity guidance to anonymize faces and bodies in images and videos. The authors report state-of-the-art de-identification, diversity, and usability for computer vision tasks, while noting limitations when faces are not detected or are occluded.
Problem
Computer vision tasks often need to detect people without identifying them, but conventional anonymization can significantly alter images and reduce detection performance.
Method
CIAGAN uses conditional generative adversarial networks and a guiding identity signal from a siamese network to anonymize faces and bodies in images and videos.
Results
The authors report state-of-the-art de-identification with large diversity, and show that anonymized images can be used by existing detection and recognition systems.
Takeaways & Limitations
The framework supports privacy-preserving image and video processing while retaining information needed for detection, recognition, tracking, and related computer vision tasks.
Takeaways & Limitations
Anonymization depends on initial face detection, and landmark-based generation can remove occlusions such as glasses.
Abstract
from arXiv · showhide
The unprecedented increase in the usage of computer vision technology in society goes hand in hand with an increased concern in data privacy. In many real-world scenarios like people tracking or action recognition, it is important to be able to process the data while taking careful consideration in protecting people's identity. We propose and develop CIAGAN, a model for image and video anonymization based on conditional generative adversarial networks. Our model is able to remove the identifying characteristics of faces and bodies while producing high-quality images and videos that can be used for any computer vision task, such as detection or tracking. Unlike previous methods, we have full control over the de-identification (anonymization) procedure, ensuring both anonymization as well as diversity. We compare our method to several baselines and achieve state-of-the-art results.
1. Introduction
CIAGAN addresses the privacy and utility tension in computer vision by anonymizing people while preserving features needed for detection, recognition, tracking, and action recognition. Its design targets controllable, diverse, realistic, and temporally consistent outputs with new identities.
- Face blurring can cause large detection-performance drops because it significantly alters images.
- CIAGAN anonymizes people in images and video streams while preserving features needed by face and body detectors.
- The model uses a conditional generative adversarial network and a novel identity control discriminator to generate realistic anonymized images and videos.
- Anonymization generates a new fake identity from the person in the original image.
- A control vector governs the fake identity, providing control over the real-person-to-fake-identity mapping.
- The system targets realistic outputs usable by detection and recognition systems, with temporal consistency and pose preservation for video tasks.
- The authors report detector usability, state-of-the-art results across several datasets, diversity and control, and an ablation study of model components.
2. Related Work
Prior anonymization and image-translation methods can produce realistic outputs, but often leak identity, impair detection, or lack robust video consistency. CIAGAN is presented against these limitations with a controllable GAN-based pipeline.
- Face generation methods achieve realistic and diverse faces, but conditioning only on random noise provides no information about the source pose.
- Image-to-image translation methods have shown strong cross-domain results, but their suitability for subtle same-domain changes such as faces or bodies remains unclear.
- Heuristic anonymization through pixelization, blurring, masking, or segmentation can make faces undetectable and unusable in standard computer vision pipelines.
- Existing methods have limited video coverage, and reported temporal consistency is not well-preserved in the available video experiments.
3. CIAGAN
CIAGAN combines landmark and masked-background conditioning with conditional GAN generation and identity guidance to create realistic anonymized faces and bodies while preserving pose and supporting video consistency. A controllable identity representation enables consistent mappings within camera sequences and new mappings across cameras.
- 3. CIAGAN: The method is directly applicable to full bodies even though the methodology discussion specifically refers to face anonymization.
- 3. CIAGAN: CIAGAN uses landmark-based input to preserve pose and maintain temporal consistency for image and video anonymization.Landmarks provide a sparse representation with little identity information and are smoothed across neighboring video frames.
- 3. CIAGAN: Masked-background conditioning focuses generation on the face region and avoids background changes that could interfere with detection or tracking.The same pipeline can anonymize full bodies by replacing the face mask with a silhouette segmentation mask.
- 3.3. Conditional generative adversarial networks: Conditional GAN training combines source structure with conditioning information to generate realistic-looking outputs suitable for standard detection and tracking systems.The generator and discriminator are trained adversarially; LSGAN labels assign a fake label a and real label b.
- 3.4. Identity guidance: Identity guidance prevents reconstruction by randomly selecting a desired identity and injecting its one-hot representation into the generator bottleneck.The identity discriminator is a pretrained siamese network fine-tuned with contrastive loss to guide generated and real identity representations.
- The case for multiple object tracking: Within a camera sequence, a control vector preserves the same real-person-to-fake-identity mapping, while changing cameras assigns a new identity.
4. Experiments
CIAGAN is evaluated against classical, learning-based, and face-replacement anonymization methods across face and body datasets. Experiments show that it preserves detection, reduces identification, maintains visual quality and diversity, and supports temporal and full-body anonymization.
- Experimental setup: CIAGAN is compared with pixelization, blur, masking, image translation, and face-replacement methods across CelebA and MOTS datasets.The evaluation includes face detection, re-identification, visual quality, temporal consistency, and body anonymization.
- Detection and identification: Nearly 100% of faces remain detectable with HOG and SSH after CIAGAN anonymization, whereas blurring lowers detection and pixelization makes faces undetectable.The comparison evaluates anonymized CelebA images with classical and deep learning-based detectors.
- Detection and identification: Identification recall falls from more than 70% on original images to 1–1.5% on CIAGAN images, while pixelization reaches 0.3% but removes content needed for detection.A landmark-trained identifier also drops to 1.9% recall on landmarks extracted from CIAGAN faces.
- Comparison to SOTA: 0.034 identification rate is achieved by CIAGAN on LFW, improving on [7]’s score below 0.04 under the stated true-acceptance-rate protocol.FaceNet models pretrained on VGGFace2 and CASIA-Webface are used for evaluation.
- Visual quality and ablations: 2.08 FID is obtained by CIAGAN, while blurring and image-translation baselines have significantly higher scores.The ablation shows that replacing landmarks with full face images lowers detection by 1.6 percentage points and increases FID.
- Visual quality and ablations: Changing the identity control vector produces diverse realistic faces, while CIAGAN preserves pose and temporal consistency and also changes clothes and colors during body anonymization.Landmark-conditioned generation produces better-looking images than the full-face-input variant in the temporal-consistency comparison.
5. Conclusions and Future Work
CIAGAN is a conditional-GAN framework for face and body anonymization that outperforms prior de-identification methods while producing diverse generated images. However, it remains dependent on landmark detection, limiting coverage of extreme poses.
- CIAGAN anonymizes faces using a guiding identity signal provided by a siamese network.
- The method outperforms the state-of-the-art in de-identification while showing large diversity in generated images.
- CIAGAN depends on landmark detection, so future work targets full-image anonymization and handling extreme poses.The authors identify landmark dependence as a limitation and propose eliminating it.
Abstract
The supplemental document organizes evaluations of generation quality, diversity, facial occlusions, model limitations, and architecture details.
- The supplement compares generation quality with different baselines and qualitatively evaluates generation diversity.It also covers facial occlusions, model limitations, and model architectures.
1. FID of baseline methods
The paper evaluates generated-image quality with FID, where lower scores are better, and reports strong quality for CIAGAN relative to several baselines. Image-translation baselines produce limited variation and realistic images only with CIAGAN.
- FID compares generated-sample statistics with real-sample statistics, using feature embeddings and fitted multivariate Gaussians; lower scores indicate better quality.The Fréchet distance between the two Gaussians provides the quality measure.
- CIAGAN reaches a very low FID, indicating very high generated-image quality.
- Pixelization and blurring baselines obtain higher FID scores as their pixel-merging or blur levels increase.
- Landmark-to-image translation baselines achieve high FID and generate only slight variations of an average-looking face, whereas CIAGAN generates realistic images.The source domain is landmarks and the target domain is images.
- The table reports quality and diversity measurements for different methods.
2. Diversity of the generated images
The diversity evaluation presents multiple anonymized versions of each source image and finds that the generated identities remain sufficiently different from one another.
- The initial experiment showed source images alongside two different anonymized versions of each source.
- A follow-up experiment presents nine anonymized versions per source image, which retain sufficiently different generated identities.
3. Removing glasses and mustache
Using face landmarks rather than the full face, the generator removes accessories such as glasses and mustaches while preserving generated-image quality.
- Landmark-based input prevents glasses and mustaches in source images from appearing in generated faces.The generator has no information about these accessories because it receives facial landmarks instead of the entire face.
- Generated-image quality does not suffer when these accessories are removed.During blending, the generator inpaints the affected region.
4. Limitations
CIAGAN remains dependent on successful face detection and training data resembling deployment images, while landmark-based processing can remove occlusions such as glasses. Detected occlusions can instead be treated as background.
- Faces that are not detected cannot be anonymized because the model requires their landmarks as input.This prevents guaranteed anonymization when the initial face detector misses a face.
- Landmark-based generation removes occlusions in front of faces, including glasses.The paper suggests detecting these occlusions and including them in the background mask to resolve the issue.
- Generation quality decreases as deployment images diverge from the training images.The model performs best on datasets similar to CelebA and requires additional extreme-pose training for strong performance in such poses.
5. Network architecture
The architecture combines landmark and masked-background inputs with a desired-identity embedding in an encoder-decoder generator, alongside discriminator and embedding networks built from residual modules.
- Generator: The generator uses an encoder-decoder architecture that combines a six-dimensional landmark/background input with an identity embedding.The embedding is concatenated with encoded features at the bottleneck before decoding.
- Discriminator: The discriminator combines the generated face with the masked background image before processing the result through residual blocks and fully connected layers.Its architecture matches the siamese network used for identity guidance.
- Identity embedding: The embedding network maps a one-hot desired-identity label to a tensor fed into the generator bottleneck.This network is implemented as a transposed convolutional neural network.
- Residual modules: Residual block down, residual block up, and residual block modules are used across the networks.Their architectures are documented separately for the three module types.
- Architecture context: The comparison figure lists CIAGAN alongside blur, pixelization, Pix2Pix, and CycleGAN anonymization outputs.Other supplied figures show source images and anonymized versions generated with different control vectors.