PaperlayerView exact revision on arXiv

Source-linked AI summary

This Person (Probably) Exists. Identity Membership Attacks Against GAN Generated Faces

Ryan Webster, Julien Rabin, Loic Simon, Frederic Jurie

arXiv:2107.06018v1cs.CVcs.AI

TL;DR

It remains unclear whether GAN-generated faces are novel in identity, a question relevant to privacy when generated images resemble people in training data. This paper tests identity-level membership through a generator-focused attack, finding that more identities reduce detectable overfitting but diversity alone does not protect overrepresented people.

  • Problem

    How GANs generalize in identity remains unclear, despite assumptions that generated faces are entirely original and prior attacks targeting other forms of leakage.

    Source: §1 · p. 1

  • Method

    The paper defines an identity membership attack that uses generated faces to test whether query samples share identities with GAN training data, without requiring exact training images.

    Source: §1 · p. 3, §4 · p. 11

  • Results

    More identities reduce detectable overfitting, but dataset diversity alone does not protect against bias, and more iterations seemed to exacerbate attack success without necessarily improving image quality.

    Source: §4 · p. 11

  • Takeaways & Limitations

    A high-precision attack on even a small subset remains troubling because privacy aims to protect all individuals equally.

    Source: §2.4 · p. 7

    The attack depends strongly on identification-network performance and requires overlap between YG and YF; its precision is zero without overlap.

    Source: §4 · p. 13

Abstract

from arXiv · show

Recently, generative adversarial networks (GANs) have achieved stunning realism, fooling even human observers. Indeed, the popular tongue-in-cheek website {\small \url{ http://thispersondoesnotexist.com}}, taunts users with GAN generated images that seem too real to believe. On the other hand, GANs do leak information about their training data, as evidenced by membership attacks recently demonstrated in the literature. In this work, we challenge the assumption that GAN faces really are novel creations, by constructing a successful membership attack of a new kind. Unlike previous works, our attack can accurately discern samples sharing the same identity as training samples without being the same samples. We demonstrate the interest of our attack across several popular face datasets and GAN training procedures. Notably, we show that even in the presence of significant dataset diversity, an over represented person can pose a privacy concern.

1 Introduction

GANs produce visually convincing images, but how they generalize—and whether their generated faces are truly novel in identity—remains unclear. This paper introduces an attack that tests whether identities represented in GAN training data can be detected, even when the generated image is not an exact training sample.

Introduction

  • GANs achieve high visual quality, yet how they generalize remains unclear, and the belief that their generated images are entirely original in identity is unsettled.Earlier work used discriminator overfitting for LOGAN, leaving generator-side leakage less explored; interpolation and pose or expression generalization offer evidence of novelty but do not resolve identity-level resemblance.Source: §1 · pp. 1–2
  • Privacy concerns include releasing generators trained on sensitive data and using generated datasets as substitutes, despite unsatisfactory privacy-quality tradeoffs reported for GAN differential privacy mechanisms.The paper gives biometric, medical, and art-data scenarios as examples of possible harms.Source: §1 · pp. 2–3
  • Earlier generative-model attacks examined direct sample leakage, but GAN faces may resemble training identities strongly even when the images themselves differ.The paper contrasts verbatim reproduction by GLO with its proposed focus on subtler identity resemblance.Source: §1 · p. 3

Identity Membership Attacks

  • The proposed identity membership attack asks whether identities represented in GAN training data also appear among query samples, rather than whether the exact query images were used.The paper identifies faces as its central case, while noting that identity can also apply to painters, biometric data, and medical images.Source: §1 · p. 3

Face Generation

  • Face datasets can contain repeated instances of individuals and uneven identity representation, including datasets whose per-identity sample counts vary widely.The paper notes that generation datasets such as FFHQ may also have such biases without specific safeguards.Source: §1 · pp. 3–4, Figure 2 · p. 4

2 Identity Membership Attack

The attack tests whether generated faces reveal identities present in a GAN’s training data, even when the attacker’s classifier uses separate image instances. It counts classifier-predicted identities among generated samples and evaluates identity-level membership decisions, with visual checks showing resemblance in low-diversity and dataset-bias scenarios.

2 Identity Membership Attack

  • The attack targets whether query individuals’ identities appeared in GAN training, rather than whether the exact query images did.Generated samples are compared with training-set images within the identified person’s identity.Source: §2 · p. 4, Figure 3 · p. 5

2.1 Attack Assumptions

  • The protocol uses separate GAN-training and classifier-training images while requiring their datasets to share some identities.The attacker’s classifier recognizes identities in its own dataset, and its predictions on generated faces help detect generator overfitting.Source: Figure 3 · p. 5

2.2 Attack Algorithm

  • The attacker generates faces, assigns each the classifier’s most likely identity, and flags identities predicted at least T times.The sample count K must be large enough; the natural threshold is T0 = λ when K = λ×|YF|, while T1 = 10λ is also tested.Source: §2.2 · pp. 5–6

2.3 Classifier Training

  • The attacker’s identity classifier uses VGGFace features and a trained fully connected layer, reaching 86.1% top-1 accuracy on VGGFace2 and 94.7% on CASIA-Webface.To keep classifier-training images separate from GAN-training images, the authors remove 53 identities shared by VGGFace and VGGFace2.Source: §2.3 · p. 6

2.4 Attack Evaluation

  • Attack evaluation treats identities shared by the GAN and classifier datasets as positives and measures identity-level precision and recall.The authors prioritize high precision even at low recall as a privacy concern, and summarize precision and recall with F1.Source: §2.4 · p. 7

2.5 Visual Evaluation

  • In low-diversity and dataset-bias scenarios, generated faces highly resemble training-set images of the same identities.The authors say these scenarios also correspond to high accuracy in guessing training identities.Source: §2.5 · p. 7

3 Identity Membership Attack Evaluation

The evaluation varies identity diversity and sample imbalance across face datasets and GANs, finding that both affect identity-membership attack success. Even high diversity does not prevent attacks against overrepresented identities, while continued StyleGAN training can worsen privacy without improving FID.

3.1 Experimental Protocol

  • The protocol defines diversity by the number of training identities and bias by unequal samples per identity, and curates CASIA-WebFace to at least 80 samples per identity.The evaluation spans real-world training settings and uses disjointness for blind attacks.Source: §3.1 · p. 8

3.2 Setting 1: Low bias and varying diversity

  • With low bias, the evaluation varies identity count from 30 to 880 while fixing samples per identity and compares LSGAN and StyleGAN on VGGFace2 and CASIA-WebFace.Precision, recall, and F1 are reported, with random identification as the baseline and frequency thresholds trading off precision and recall.Source: §3.2 · p. 8

3.3 Setting 2: Varying bias and high diversity

  • In the high-diversity setting, the study varies the number of overrepresented identities and evaluates attacks on that biased subset as its bias ratio decreases.The remaining identities are distributed across 2,000 identities and fixed at 40,000 samples; evaluation uses VGGFace2 with LSGAN and StyleGAN.Source: §3.3 · pp. 8–9

3.4 Analysis

  • The recovered-identity distribution has a lighter tail as identity count and sample count increase, or as the bias ratio decreases.Frequent recovered identities in the tail are training identities prone to membership attack, and their samples can be extracted for visual overfitting assessment.Source: §3.4 · p. 9
  • Even with more than 2,000 identities, attacks retain high F1 under bias, while sufficient diversity in low-bias data reduces attacks to near guessing.Insufficient diversity leaves models susceptible even in the blind setting; LSGAN appears more susceptible than StyleGAN until a dataset-size threshold.Source: §3.4 · p. 9, Table 1 · p. 10

3.5 Early Stopping

  • Beyond about 17,000 StyleGAN iterations, continued training is detrimental to privacy without improving FID.The paper identifies this as an interaction between FID and training iterations, suggesting that early stopping can avoid this observed privacy cost.Source: §3.5 · p. 10

4 Discussion and Conclusion

The discussion reports that identity membership attacks reveal training-data signals in generated faces: greater identity diversity reduces detectability, but dataset bias can preserve successful attacks. Attack success also increases with training iterations without necessarily improving image quality, motivating diversity, low bias, and early stopping as empirical safeguards.

4 Discussion and Conclusion

  • As training iterations increase, attack success can worsen while image quality does not necessarily improve.The authors note that attack precision continues to gain while FID converges.Source: §4 · p. 11, Figure 7 · p. 14
  • The attack is black-box, targets the generator directly, and does not require exact training samples or generator architecture and parameters.It also requires no additional training based on the generator.Source: §4 · p. 11
  • More identities reduce detectable overfitting, but dataset bias can leave attacks successful even with more than 2,000 identities.The biased-data setting remains detectable despite significant diversity.Source: §4 · p. 11, Figure 5 · p. 12
  • The attack's effectiveness depends on the identification network and on overlap between query and training identities; with no overlap, precision is zero.The authors also identify generalization beyond faces as an important direction.Source: §4 · pp. 12–13
  • More sophisticated privacy approaches, including GAN-objective changes or distillation, are left for future work.The discussion also notes that early stopping may mitigate the identified problems.Source: §4 · p. 14
  • The authors recommend sufficient identity diversity, low dataset bias, and early stopping as empirical tools for training safer models.Early stopping is suggested when FID stops changing or reaches a threshold.Source: §4 · p. 14

Related papers

Loading 2107.06018v1…