PaperlayerView exact revision on arXiv

Source-linked AI summary

FaceLinkGen: Rethinking Identity Leakage in Privacy-Preserving Face Recognition with Identity Extraction

Wenqi Guo, Shan Du

arXiv:2602.02914v2cs.CV

TL;DR

PPFR evaluations commonly treat privacy as resistance to pixel-level reconstruction, but the paper argues this does not capture identity-level leakage. FaceLinkGen distills identity embeddings from protected templates for linkage and uses them for face regeneration without recovering original pixels. It achieves over 98.5% matching accuracy and above 96% regeneration success across three frequency-domain PPFR systems, with leakage remaining feasible under constrained assumptions.

  • Problem

    Existing PPFR evaluations emphasize pixel-level reconstruction metrics such as PSNR and SSIM, leaving identity-level leakage insufficiently captured.

  • Method

    FaceLinkGen uses distillation to extract identity embeddings from protected templates and a diffusion generator to regenerate identity-consistent faces without recovering original pixels.

  • Results

    Over 98.5% matching accuracy and above 96% regeneration success are achieved on three frequency-domain PPFR systems.

  • Takeaways & Limitations

    The results support adopting identity-centric evaluation alongside existing pixel-level metrics in PPFR research.

  • Takeaways & Limitations

    The near-zero-knowledge proxy attack relies on high-pass-filter coupling specific to frequency-domain PPFR methods, although CanFG remains vulnerable with oracle access.

Abstract

from arXiv · show

Transformation-based privacy-preserving face recognition (PPFR) aims to verify identities while hiding facial data from attackers and malicious service providers. Existing evaluations mostly treat privacy as resistance to pixel-level reconstruction, measured by PSNR and SSIM. We show that this reconstruction-centric view fails. We present FaceLinkGen, an identity extraction attack that performs linkage/matching and face regeneration directly from protected templates without recovering original pixels. On three recent PPFR systems, FaceLinkGen reaches over 98.5\% matching accuracy and above 96\% regeneration success, and still exceeds 92\% matching and 94\% regeneration in a near zero knowledge setting. These results expose a structural gap between pixel distortion metrics, which are widely used in PPFR evaluation, and real privacy. We show that visual obfuscation leaves identity information broadly exposed to both external intruders and untrusted service providers.

1 Introduction and Related Works

Transformation-based PPFR is designed to preserve face-recognition utility while hiding raw facial data, but its reconstruction-centric privacy evaluation misses identity leakage. FaceLinkGen instead extracts identity for linkage and regeneration directly from protected templates, showing that visual or pixel-level obfuscation can coexist with identity exposure.

  • Motivation: Transformation-based PPFR converts faces into protected templates intended for server-side identity matching without exposing raw facial data.The primary adversary is a curious or malicious service provider receiving the template.
  • Existing evaluation: PSNR and SSIM dominate PPFR privacy evaluation by measuring resistance to pixel-level or local reconstruction.This evaluation tradition is inherited from image privacy and compression literature.
  • Existing evaluation: Pixel-level reconstruction does not reliably indicate identity consistency: visually similar images may depict different people, while same-identity images may have low pixel similarity.Figures 2 and Table 1 illustrate the mismatch between pixel-level and identity-level similarity.
  • Failure of reconstruction-centric attacks: Pixel-level attack objectives can produce blurry or average-looking outputs because protection discards nuisance factors needed to reproduce the exact enrollment image.This traps attackers into targeting a specific registration image even when identity-revealing information remains accessible.
  • FaceLinkGen: FaceLinkGen performs identity linkage and face regeneration from protected templates without recovering original pixels.The paper argues that a straightforward distillation pipeline exposes identity information even when PSNR and SSIM suggest unreconstructability.

2 Threat Model

The threat model returns to PPFR’s intended adversary: an untrusted service provider with oracle access to the conversion process, while distinguishing this insider from external wiretappers. The attack does not depend on server-side verification behavior or extensive online querying.

  • Adversary scope: PPFR was originally intended to protect face data from a remote recognition server that can match protected representations without reverting them to original images.Network-level eavesdropping is treated as a separate concern addressed by secure communication protocols.
  • Adversary scope: External wiretapping represents a later framing that departs from PPFR’s original insider-centric design intent.The paper separates this framing from the service-provider threat it evaluates as intended.
  • Adversary scope: The primary threat is a service provider with oracle access to the conversion process but no knowledge of its internal architecture or parameters.This insider-focused model is less informed than prior assumptions that reveal the conversion architecture or other internals.
  • Attack assumptions: The attack avoids server-side behavior and therefore does not require thousands of online verification queries or continuous similarity scores.This matters because deployed systems may return only binary decisions or quantized scores.

3 Methods

FaceLinkGen models protected templates as retaining identity information while discarding nuisance factors, then distills an identity embedding and regenerates a new face by supplying sampled nuisance information. The attack therefore targets identity extraction rather than exact pixel recovery.

  • Representation: PPFR templates suppress nuisance information z_N while preserving identity information z_I through a lossy transformation.This representation explains why recovering the original image is ill-posed while identity utility remains available.
  • Identity extraction: FaceLinkGen trains a student model to map protected templates into an identity embedding space aligned with a frozen teacher’s embeddings from original images.The training objective maximizes cosine similarity between student and teacher representations.
  • Face regeneration: The attack uses a diffusion generator to synthesize a face from the extracted identity representation and sampled stochastic noise rather than recovering the original nuisance factors.The generated face preserves identity while varying non-identity attributes.
  • Representation: Identity extraction remains possible because recovering discarded nuisance information is not required for successful attack output.Visual distortion of nuisance information therefore does not prevent identity extraction in the evaluated frequency-domain systems.
  • Model assumptions: The attack is independent of the PPFR server’s recognition backbone as long as templates retain identity-discriminative features learnably aligned with an accessible embedding space.ArcFace is used as a convenient teacher and compatible generator interface, not because it is the server’s actual model.

4 Attack Vectors

FaceLinkGen supports two attack vectors from protected templates: embedding-space linkage and diffusion-based face regeneration. Linkage can connect public faces to templates or connect templates across leaked databases through nearest-neighbor embedding search.

  • Attack vectors: The linkage attack uses an extracted identity embedding, whereas regeneration additionally uses a diffusion generator to synthesize identity-consistent faces.These are distinct attack vectors enabled by the same identity extraction framework.
  • Linkage attack: Face-to-template linkage associates a real-world face with a protected identity, while template-to-template linkage links protected templates belonging to the same person.The paper notes that ISO/IEC 24745 requires resistance to template-to-template search but not face-to-template search.
  • Linkage attack: The adversary obtains a query embedding from either the student or teacher model, computes embeddings for protected templates, and performs nearest-neighbor search.Cosine similarity is used to compare embeddings.
  • Regeneration attack: Diffusion-based generation produces a realistic face corresponding to the protected template without reconstructing the original enrollment image.The generator takes a facial embedding and generates an image whose embedding matches it.

5 Experiments and Results

The experiments evaluate FaceLinkGen on three frequency-domain PPFR methods using linkage and regeneration attacks. Protected templates retain identity information, achieving high matching and regeneration success despite strong frequency-channel disruption.

  • Experimental Setup: FaceLinkGen evaluates PartialFace, MinusFace, and FracFace using protected templates and a student model trained without dataset or architecture overlap.The attack uses CASIA-WebFace for distillation and evaluates linkage and regeneration across multiple datasets.
  • Linkage Attack: Above 70% closed-set 1-to-N linkage success, frequently exceeding 80%, approaches the dataset’s theoretical maximum under WebFace noise.The original-image-to-original-image linkage score of 0.88 establishes the performance upper bound.
  • Regeneration Attack: Over 97% first-attempt regeneration success was achieved on all three datasets, while five-attempt success ranged from 97.9% to 100%.Even at the strictest verification threshold, success remained above 90%.
  • Protection Claims: Near-total identity recovery persists despite FracFace’s high frequency-channel protection claims.FracFace’s metric counts distorted or disrupted frequency channels, whereas FaceLinkGen measures identity recovery through commercial verification.
  • Embedding Analysis: Embedding similarity to the source image exceeds similarity to another image of the same person, independently supporting identity leakage.This cross-check reduces dependence on Arc2Face and commercial verification APIs.

6 What If the Attacker Knows Almost Nothing?

The near-zero-knowledge experiment tests whether frequency-domain PPFR identity leakage persists without knowledge of the protection mechanism. A generic high-pass proxy attack remains effective across all three systems, though this proxy is specific to frequency-domain methods.

  • Attacker Assumptions: The attacker uses only 30 paired image-template samples for validation and has zero knowledge of the underlying protection mechanisms.This setting is stricter than the cited black-box scenario that assumes knowledge of the conversion process.
  • Proxy Attack: A generic Gaussian-blur-based high-pass filter serves as a universal proxy because the evaluated templates visibly preserve high-frequency information while obfuscating low-frequency information.The proxy avoids system-specific modeling and does not depend on DCT or DWT implementations.
  • Results: One model attacking all three systems achieved over 92% 1-to-1 matching and over 94% regeneration success@5 on Face++.On LFW, 1-to-1 linkage remained around 92-96%, close to the main experiments.
  • Interpretation: Identity-consistent regeneration and reliable linkage remain feasible under extremely constrained attacker assumptions for frequency-domain PPFR methods.The evaluated systems share strong coupling between their output representations and simple high-pass filtering operations.
  • Scope: The high-pass proxy attack is specific to frequency-domain methods, but CanFG remains vulnerable under the oracle-access threat model.This indicates that identity leakage is not unique to frequency-domain design.

7 Similarity Distribution

The evaluation measures identity leakage by comparing protected templates with face embeddings and reports that templates remain strongly tied to their source identities. This exposes weaknesses in defenses judged primarily by reconstruction or frequency-channel disruption.

  • Similarity-based identity leakage: FaceLinkGen evaluates identity leakage through ArcFace-space cosine similarity between normal images and protected templates.The analysis compares same-person image similarity with image-to-template similarity, independently of face generation or commercial verification APIs.
  • Defense evaluation: High frequency-channel disruption fails to prevent identity extraction, with FaceLinkGen achieving near-total recovery through commercial-grade verification.The comparison contrasts FracFace’s defensive claim with direct evaluation of actual identity leakage.
  • Defense evaluation: Table 7 reports regeneration success@5 and 1-to-1 linkage success, with matching accuracy defined by the optimal threshold.These metrics evaluate regeneration and linkage under assumption-constrained settings.
  • Similarity-based identity leakage: The template-to-source similarity exceeds the similarity between another image of the same person across all three methods.This indicates that templates preserve identity while retaining source-instance bias rather than acting as universal identity prototypes.

8 Soft Identity Leakage: Beyond Unique Identifiers

Protected templates can retain soft biometric attributes beyond unique identity information. Tests show recoverable age, gender, and race signals, challenging designs that preserve such attributes for auxiliary tasks.

  • Motivation: Soft biometric attributes such as skin color, age, and gender are sensitive data that can enable profiling and discrimination.The paper argues that privacy-preserving systems should prevent recovery of these attributes from templates.
  • Measured leakage: Gender is identified with at least 82% accuracy, while Age MAE ranges from 6.1 to 7.5 years across 500 FairFace test images.Race accuracy ranges from 0.50 to 0.60 across seven classes, above the 14.3% random baseline.
  • Measured leakage: The above-chance recovery of race, age, and gender indicates that protected templates retain information sufficient for downstream attribute extraction.The paper distinguishes template retention from the general presence of demographic information in ArcFace embeddings.
  • Measured leakage: Table 8 includes original-image ArcFace embeddings as a baseline for evaluating soft biometric leakage.The baseline supports comparison between attribute prediction from protected templates and original-image representations.
  • Design implication: Intentionally preserving soft biometrics for auxiliary tasks gives attackers additional information for reconstruction and profiling without improving recognition performance.The paper identifies this design choice in CanFG and FaceAnonyMixer as problematic for PPFR.

9 Transfer to Attacking De-Identification Systems

The paper extends its identity-extraction perspective from PPFR templates to perception-consistent de-identification systems. A student model recovers identity information that adversarial protection suppresses only in the original recognizer’s embedding space.

  • System setting: Perception-consistent de-identification preserves human-recognizable faces while disrupting machine-based face recognition.These systems use adversarial noise or synthetic makeup rather than removing identity information from the visual signal.
  • System setting: Both PPFR and perception-consistent de-identification can leak identity because each preserves sufficient machine-readable or human-perceivable identity information.The paper characterizes these as symmetric vulnerabilities involving recoverable mutual information.
  • Experimental setting: TIP-IM is evaluated on 2,082 images from 408 identities in a subset of CASIA-WebFace.The paper distinguishes dodging from impersonation when assessing privacy-oriented de-identification.
  • Results: TIP-IM’s perfect protection against ArcFace is almost completely undermined when the distillation pipeline recovers identity information.The reported similarities compare the original FR model with a student model operating on protected images.

10 Future Directions

The paper recommends strengthening PPFR defenses and broadening vulnerability assessments. Proposed directions include secret-key or cryptographic hardening and redesigning de-identification around divergence between human and machine facial perception.

  • Research priorities: Future PPFR work should prioritize stronger defensive mechanisms and broader vulnerability assessments.This recommendation frames the paper’s proposed research directions.
  • Cryptographic and key-based hardening: Secret keys can make face conversion require both biometric data and a key, preventing conversion without the secret.The paper presents key-based hardening as a multi-factor authentication approach and also suggests formal cryptography.
  • Alternative de-identification: De-identification should explore hiding identity from humans rather than preserving it visually while fooling machines.The proposal relies on reported divergence between human and machine perception of facial similarity.

11 Conclusion

FaceLinkGen exposes identity-level leakage in frequency-domain PPFR despite strong pixel-level privacy evaluations. The findings support identity-centric evaluation alongside existing metrics and exploration of stronger protection mechanisms.

  • Over 98.5% matching accuracy and above 96% regeneration success were achieved on three frequency-domain PPFR systems.
  • Identity information can be extracted from protected templates for linkage and face regeneration without recovering original pixels.
  • The findings suggest that pixel-level evaluation results overstate the identity privacy provided by these systems.
  • The paper encourages identity-centric evaluation alongside existing metrics and exploration of stronger protection mechanisms, including cryptographic approaches with more formal privacy guarantees.
Loading 2602.02914v2…