PrivacyNet: Semi-Adversarial Networks for Multi-attribute Face Privacy

1 INTRODUCTION

• Face images contain auxiliary attributes such as age and gender, which machine-learning systems can extract and users may wish to keep private.The paper frames this concern around profiling and possible extraction without users’ consent.Source: §1 · p. 1
• Earlier SAN models concealed gender while retaining satisfactory matching accuracy, but did not provide multi-attribute privacy.PrivacyNet is presented as extending this gender-focused approach to race, gender, and age.Source: §1 · pp. 1–2
• PrivacyNet targets selective, collective perturbations of age, gender, and race while retaining biometric matching performance.The model is proposed for choosing which attributes to conceal and how many to conceal.Source: §1 · pp. 1–2
• PrivacyNet uses a GAN-based model with cycle-consistency loss, unlike earlier regular convolutional autoencoders,,.The paper identifies this architectural distinction as a difference from prior work.Source: §1 · p. 2

2 RELATED WORK

• Sim and Zhang’s multi-modal discriminant analysis selectively suppresses attributes, but does not explicitly preserve matching performance, severely diminishing biometric utility.Their decomposition uses gender, age, and ethnicity axes, leaving identity information as a residual.Source: §2 · p. 2
• Adversarial privacy perturbations can fail to generalize to unseen attribute classifiers, a limitation described as critical for real-world privacy applications,,.The cited approaches generate examples based on a specific attribute classifier.Source: §2 · p. 2
• Face-representation methods remove sensitive information from recognition vectors, but such vectors may be uninterpretable and incompatible with future recognition software,, [68],,.PrivacyNet instead applies perturbations directly to face images.Source: §2 · p. 3
• SAN methods obfuscate gender while retaining matching utility, with generalizability enhanced by diversifying auxiliary classifiers or combining diverse SAN models,,.These methods remain focused on a single attribute: gender.Source: §2 · p. 3
• GAN face-editing methods target attributes loosely connected to identity, while gender and race are more closely tied to identity and harder to modify without affecting recognition,,,,.DIAT also considers gender transfer, but its reported eyeglass-attribute transfer significantly decreased face-verification performance [80].Source: §2 · p. 3
• PrivacyNet is presented as the first multi-attribute privacy method to generalize to unseen attribute classifiers while preserving recognition utility across unseen face matchers.It modifies face images to obfuscate soft-biometric attributes while maintaining recognition capability.Source: §2 · p. 3

3.1 Problem Formulation

• Perturbations should reduce unseen classifiers’ performance on selected attributes while preserving other attributes and arbitrary face matchers’ performance.The formulation compares attribute-classifier performance on perturbed and original images and requires matching performance to remain substantially unaffected.Source: §3.1 · p. 3

3.2 PrivacyNet

• PrivacyNet uses a conditional generator, source discriminator, attribute classifier, and auxiliary face matcher in a cycle-consistent GAN.The generator maps an input face toward target attribute labels; the cycle constraint reconstructs the original image from the modified one.Source: §3.2 · p. 3, Figure 2 · p. 4

3.3 Neural Network Architecture of PrivacyNet

• Dsrc and Dattr share convolutional layers but use separate output layers, while the auxiliary matcher is a pre-trained VGG-Face model.The source-discriminator head returns a scalar score; the attribute-classifier head returns class probabilities.Source: Figure 3 · p. 5, §3.3 · p. 5

3.4 Datasets

• CelebA and MORPH training subsets are combined for a more balanced race distribution, while MUCT, RaFD, and UTK-face are used only for evaluation.The CelebA and MORPH splits are subject-disjoint; the datasets lack ground-truth labels for at least one of age or race.Source: §3.4 · p. 6
• Age ground truth is available only in MORPH and UTK-face; other datasets use commercial age predictions, and age grouping does not fully capture aging-related texture changes.The stated objective is to disturb arbitrary age predictors, not synthesize faces in particular age groups.Source: §3.4 · p. 6
• Gender is treated as binary, and race analysis is limited to Caucasian and African-descent labels, with RaFD excluded from race prediction analysis.Other race groups are omitted because they are under-represented in training data.Source: §3.4 · p. 6
• Matching analysis excludes UTK-face because it lacks subject information and evaluates three face matchers: M-COTS, DR-GAN, and SE-Net.DR-GAN and SE-Net were trained on VGGFace2.Source: §3.4 · p. 6

4 EXPERIMENTAL RESULTS

• PrivacyNet was trained on joint CelebA and MORPH training subsets, and evaluation used five datasets, three gender classifiers, a race classifier, and an age predictor.The model can generate up to 15 outputs per image, depending on which attributes are selected for perturbation.Source: §4 · p. 6, §4.1 · p. 8

4.1 Perturbing Facial Attributes

• Gender-perturbed outputs raise EER to 20–40% for G-COTS and above 60% for AFFACT and IntraFace, while unselected gender information remains usable.For G-COTS, PrivacyNet increases EER more than the face-mixing approach and the model in; baseline-GAN produces larger deviations but degrades matching utility.Source: §4.1 · p. 8, Figure 5 · p. 9
• When race is selected for perturbation, EER rises to about 20% on CelebA and UTK-face and about 10% on MORPH and MUCT; unselected outputs remain near 1%.Method performs slightly better when perturbing race, but its gender perturbation also affects race predictions.Source: §4.1 · p. 8, Figure 6 · p. 10
• Age perturbation with target A2 produces the largest MAE—around 20 years on RaFD and 15 years on the other four datasets—while unselected outputs remain near 5 years.A1 changes are similar to unperturbed cases; the authors suggest age-pattern non-stationarity may limit these perturbations and propose ordinal regression as a possible improvement.Source: §4.1 · pp. 8–9, Figure 7 · p. 10

4.2 Retaining the Matching Utility of Face Images

• PrivacyNet’s ROC and CMC results closely match those from original images across evaluated datasets, while baseline-GAN shows larger deviations and lower matching performance.Removing the matching loss also reduces true matching rate and identification accuracy; the matching-loss coefficient can tune the utility–obfuscation trade-off.Source: §4.2 · pp. 9–10

• PrivacyNet combines a SAN module with GANs to selectively perturb age, gender, and race while preserving other attributes and face-matching utility.Source: §5 · p. 11
• Three unseen face matchers and three unseen attribute classifiers showed attribute perturbation without adversely impacting matching utility.Source: §5 · p. 11
• Modified images may contain artifacts, and human observers might distinguish them from unmodified faces.Source: §5 · p. 11