Source-linked AI summary
Learning to Anonymize Faces for Privacy Preserving Action Detection
Zhongzheng Ren, Yong Jae Lee, Michael S. Ryoo
TL;DR
Privacy-preserving vision must retain useful action recognition without exposing identity information in recorded video. This paper learns a pixel-level face anonymizer through adversarial, multi-task training, and reports strong action detection near un-anonymized performance while reducing face verification accuracy on modified faces. The learned anonymizer’s deployment in embedded hardware is proposed as future work outside the paper’s scope.
Problem
Camera-based visual recognition requires privacy-sensitive high-resolution video, while existing anonymization heuristics lack privacy-optimality guarantees and can hurt recognition performance.
Method
A multi-task adversarial formulation trains a pixel-level face modifier against a face classifier while an action detector favors modifications that preserve action detection.
Results
The method achieves action detection significantly better than other anonymization baselines and close to un-anonymized detection, while face verification falls from 94.75% on original LFW faces to 66.95% on modified faces.
Takeaways & Limitations
The learned anonymizer confuses humans and machines in face identification while producing reliable action detection.
Takeaways & Limitations
Hardware-level deployment of the anonymizer for surveillance, smart-home cameras, and robots is outside the scope of this work.
Abstract
from arXiv · showhide
There is an increasing concern in computer vision devices invading users' privacy by recording unwanted videos. On the one hand, we want the camera systems to recognize important events and assist human daily lives by understanding its videos, but on the other hand we want to ensure that they do not intrude people's privacy. In this paper, we propose a new principled approach for learning a video \emph{face anonymizer}. We use an adversarial training setting in which two competing systems fight: (1) a video anonymizer that modifies the original video to remove privacy-sensitive information while still trying to maximize spatial action detection performance, and (2) a discriminator that tries to extract privacy-sensitive information from the anonymized videos. The end result is a video anonymizer that performs pixel-level modifications to anonymize each person's face, with minimal effect on action detection performance. We experimentally confirm the benefits of our approach compared to conventional hand-crafted anonymization methods including masking, blurring, and noise adding. Code, demo, and more results can be found on our project page https://jason718.github.io/project/privacy/main.html.
1 Introduction
The paper addresses the tension between visual recognition and privacy in camera systems by learning a face anonymizer that removes identity information while preserving action detection. Its adversarial formulation trains a modifier against a face discriminator while jointly favoring action-detection performance.
- Motivation: Ubiquitous cameras support visual understanding but raise privacy concerns through unwanted recording and exposure of privacy-sensitive video.High-resolution data may be loaded into local memory or sent to powerful servers, creating risks of unauthorized access.
- Motivation: Hand-crafted anonymization methods such as downsampling, masking, and segmentation-based processing lack guarantees of privacy optimality and can reduce recognition performance.Their information loss may hurt downstream visual recognition.
- Approach: The proposed video anonymizer modifies original videos to remove privacy-sensitive information while preserving scene-understanding performance through adversarial training.The anonymizer competes with a discriminator that attempts to extract private information from anonymized videos.
- Approach: Face identity is used as representative private information, while action detection represents the scene-understanding task.The approach targets identity protection without discarding the actions needed for recognition.
- Approach: A multi-task GAN formulation uses a face anonymizer to reduce identification accuracy, a face identifier to resist modification, and an activity detector to favor action-preserving changes.The three components jointly balance anonymization against activity detection.
- Scope: Applying the learned anonymizer in surveillance, smart-home cameras, and robots via embedded hardware remains outside this work’s scope.The proposed hardware would anonymize images before processor loading or network transmission.
2 Related work
Related work spans privacy-preserving action recognition, spatial action detection, face recognition, adversarial attacks, and GANs. The paper distinguishes its explicit pixel-level face modifier from prior feature-only privacy methods and hand-crafted low-resolution transforms.
- Privacy-Preserving Recognition: Privacy-preserving action recognition removes identity information from testing videos while retaining reliable action recognition, but relatively few studies address this setting.This differs from protecting privacy in training data.
- Privacy-Preserving Recognition: Prior low-resolution action-recognition methods rely on hand-crafted video downsampling and are not guaranteed to be optimal for privacy-preserving recognition.Examples include extreme resolutions such as 16x12 videos.
- Action Detection: Spatial action detection localizes actions with bounding boxes in each video frame and categorizes them.The paper uses Faster R-CNN as its frame-level action detector.
- Face Recognition: Face recognition methods include softmax classification, center loss, triplet loss with hard-instance mining, and metric-learning combinations.The paper uses the face-recognition module from.
- Network Attacking: The paper’s modified images attack a face identifier like adversarial examples, but its goal is to dramatically change face content rather than make imperceptible perturbations.This distinguishes the task from conventional network attacks.
- Generative Adversarial Networks: GANs use adversarial loss to generate realistic data, and the paper extends this formulation to explicit pixel-level face anonymization without requiring a target visual domain.Prior privacy work cited here suppresses user information in video features without modifying image pixels.
- Generative Adversarial Networks: The architecture combines a face modifier, an action detector, and an adversarial face classifier to learn privacy-preserving action detection.The face classifier encourages non-trivial anonymization while the action detector learns under the modifications.
3 Approach
The approach jointly learns a face modifier and spatial action detector while adversarially training a face classifier to resist identity obfuscation. It combines privacy, action-detection, and structure-preservation objectives using video and face datasets.
- Framework: The framework learns a face modifier, face classifier, and action detector through a multi-task objective using training videos V and face images F.The modifier anonymizes faces, the classifier identifies them, and the detector recognizes actions despite modification.
- Photorealistic Loss: An L1 loss preserves basic face structure such as pose and brightness, while a relatively small λ prevents the modified face from remaining too similar to the original.The full objective sums detection, adversarial, and L1 losses; training alternates updates of the modifier, discriminator, and detector.
- Action Detection Loss: The modifier replaces detected face regions in each frame while leaving the rest of the frame unchanged, including repeated replacement for multiple people.The resulting modified frame is v′ = v − r_v + M(r_v).
- Action Detection Loss: The action detector is trained on modified frames with ground-truth action boxes and categories using Faster-RCNN’s RPN and Fast-RCNN classification and regression losses.Faster-RCNN supplies four component losses: RPN classification, RPN regression, Fast-RCNN classification, and Fast-RCNN regression.
- Adversarial Classification: Adversarial training alternates updates so the modifier fools face identification while the discriminator learns to classify anonymized and original faces correctly.The modifier minimizes the classification loss, whereas the discriminator maximizes it; using both modified and original images encourages greater visual change.
4 Implementation
The implementation combines dedicated face detection, face modification, spatial action detection, and face identity-classification networks. Training uses Adam with task-specific learning rates for 12 epochs.
- Face Detection: Faces are detected with SSH, filtered at probability greater than 0.8, and processed with MTCNN to remove false positives.The two-stage process produces clean, highly accurate face bounding boxes.
- Face Modification: The face modifier uses nine residual blocks, instance normalization, and bilinear resizing to 256 × 256.Its architecture is adapted from Johnson et al.’s image-translation network.
- Spatial Action Detection: Spatial action detection uses Faster-RCNN with a ResNet-101 backbone and end-to-end training, resizing images differently for JHMDB and DALY.The shorter image length is set to 340 pixels for JHMDB and to 600×800 pixels for DALY.
- Face Identity Classification: Face identity classification uses SphereFace-20 initialized from a CASIA-WebFace pretrained model with aligned, keypoint-cropped faces.Cropping uses a differentiable non-parametric grid generator and sampler.
- Training Details: Training uses Adam for 12 epochs with learning rates of 0.001 for Faster-RCNN and 0.0003 for the face modifier and classifier.The learning rate is dropped by 1 10 after the seventh epoch.
5 Results
The evaluation compares privacy protection and action detection across datasets, baselines, and qualitative studies. The learned modifier improves the privacy–utility trade-off, preserves face-involving actions, and produces identity changes that appear systematic and effective against human recognition.
- Metrics and Datasets: The evaluation uses action-detection mAP at IoU 0.5 on DALY and JHMDB, plus face verification on LFW after training with CASIA-WebFace.DALY and JHMDB include actions involving the face area; face verification uses cosine similarity between extracted features.
- Baselines: Hand-crafted baselines increase face-recognition error but harm action-detection performance on both DALY and JHMDB.The baselines include extreme downsampling, masking, noise, and super-pixel modification.
- Quantitative Results: The learned method achieves significantly better action detection than the baselines while remaining close to un-anonymized detection, and outperforms most baselines on face verification.It is inferior only to 8 × 8 down-sampling and masking for face verification, while substantially improving the joint trade-off.
- Quantitative Results: Action-detection gains are largest for drinking, phoning, playing harmonica, and taking photos or videos, whose actions involve the face region.For cleaning floor, cleaning windows, and ironing, the method only marginally improves over or performs worse than baselines.
- Qualitative Results: Qualitative examples show realistic identity changes while preserving pose, clothing, background, and actions such as brushing teeth, playing harmonica, and applying makeup.The modifier changes local details including gender, hairstyle, nose, glasses, eyebrows, age, and ethnicity.
- Qualitative Results: Modified images of the same person remain visually consistent across pictures and video frames, suggesting systematic rather than instance-independent identity changes.The paper hypothesizes that gradient ascent reshapes the face-identity manifold systematically more easily than perturbing each face separately.
- User Study: In a study of 400 answers from 10 subjects, Q1 accuracy was 53.3%, celebrities were correctly identified only 19.75% of the time, and all but two subjects felt protected.The results indicate that modified images were difficult for people to identify and were generally perceived as privacy-protective.
- Ablation Study: The trained face classifier retained 94.75% verification accuracy on original LFW faces but achieved only 66.95% on modified faces.This indicates that the classifier remained effective on unmodified faces despite being fooled by anonymized inputs.
6 Conclusion
The paper presents an adversarial approach that jointly learns face anonymization and activity detection. Experiments show that the learned anonymizer confuses humans and machines in face identification while producing reliable action detection.
- The paper introduces a face anonymizer and activity detector trained through an adversarial learning formulation.
- The learned anonymizer confuses both humans and machines in face identification.
- The approach produces reliable action detection alongside face anonymization.