How Private is Private? A Comparative Study for Face De-Identification

• A review of over 100 FDeID papers found more than 60 datasets and 20 evaluation metrics across prior protocols.Source: Figure 1 · p. 2
• HiFD compares pretrained-estimator outputs on original and de-identified faces, directly measuring surviving information without per-image annotations.Source: §1 · p. 2
• The hierarchy separates readily perceived macro cues, fragile micro cues, and highly modification-sensitive imperceptible signals such as rPPG.Source: §1 · p. 3
• The paper contributes UTILFACE, a three-level utility hierarchy, HiFD, and a comparative study of adversarial, GAN-based, and diffusion-based methods.Source: §1 · p. 3

2.1 FDeID Methods

• Methods span pixelation [82], k-Same [18, 19, 58, 63], GANs, and diffusion [24] [73], yet utility evaluation remains fragmented.Source: §2.1 · pp. 3–4

2.2 Evaluation Metrics and Protocols

• Identity, utility, and quality use heterogeneous metrics and dataset-specific annotations, impeding systematic comparison and reproducibility; no metric unifies all three dimensions.Source: §2.2 · p. 4

2.3 Facial Signal Analysis

• Facial signals range from perceptible coarse attributes to fragile micro-dynamics and sub-pixel physiology; the paper formalizes them as three evaluation levels.Source: §2.3 · p. 4

3.1 UTILFACE Benchmark

• Four large-scale sources pass through identity-aware cleaning, resolution enhancement, perceptual-quality filtering, and demographic balancing to form UTILFACE.Source: §3.1 · pp. 4–5
• UTILFACE contains 99,928 images across 2,069 identities and reduces the largest-to-smallest ethnicity-category ratio from 53× to 2.3×.Source: §3.1 · p. 5, Figure 3 · p. 5

3.2 HiFD Metric

• HiFD compares pretrained estimators' outputs on original and de-identified faces, measuring consistency without per-image ground-truth labels.Source: §3.2 · pp. 5–6
• HiFD scores macro, micro, and imperceptible utility alongside identity suppression and image quality, using task-specific consistency measures and estimator ensembles.Source: §3.2 · p. 6
• A weighted harmonic mean aggregates scoreable dimensions, penalizing imbalanced components; configurable profiles prioritize privacy, balanced evaluation, or clinical utility.Source: §3.2 · p. 6

4.1 Experimental Setup

• The study evaluates 12 adversarial, GAN-based, and diffusion-based methods, using UTILFACE for privacy, L1 attributes, gaze, and quality, with DFME and PURE for other utility measures.HiFD scores compare pretrained estimators’ outputs on original and de-identified faces; the default aggregation uses a balanced profile.Source: §4.1 · pp. 6–7

4.2 Main Results

• Adversarial, GAN-based, and diffusion methods occupy different privacy–utility trade-offs: PGD and MI-FGSM retain U1 above 0.92 but have ¯P below 0.14, while CIAGAN reaches ¯P 0.45 with low U1 and landmark scores.Adv-Makeup preserves macro cues but has ¯P 0.01; DiffAM scores higher than WeakenDiff on U2, U3, Q, and overall HiFD (0.394 versus 0.332).Source: §4.2 · pp. 7–8
• Utility generally falls from L1 to L3, especially for generative methods; at threshold 0.80, U3 preservation implies U1 and U2 preservation, but U1 implies U3 for only 22% of methods.The weighted harmonic mean penalizes near-zero components, so strong macro-cue scores cannot compensate for weak L3 utility.Source: §4.2 · p. 8, Figure 4 · p. 8
• Application profiles reshape rankings: TIP-IM still leads under Privacy-First, while adversarial methods dominate Clinical; no profile rescues methods with near-zero identity suppression.The profiles assign extra weight to privacy or physiological-signal preservation, respectively.Source: §4.2 · p. 8, Table 4 · p. 8
• Across methods, gender gaps stay at or below 0.035 and ethnicity ranges at or below 0.064, far smaller than inter-method variation.The authors identify method choice, rather than subject identity, as the dominant axis of FDeID quality variation.Source: §4.2 · pp. 8–9, Figure 5 · p. 9

4.3 HiFD Validation

• On RAF-DB, consistency scores tracked retained macro-expression accuracy closely; on MMPD, their rPPG rankings largely matched contact-sensor outcomes, while ¯P ≥0.45 corresponded to 93–96% de-identification at FAR 10−3.Consistency can still mislead for near-tie posteriors or estimators degenerate on de-identified inputs.Source: §4.3 · p. 9, Figure 6 · p. 10

4.4 How Private is Private?

• TIP-IM leads with HiFD 0.607, yet no evaluated method preserves privacy and utility across all five axes; every other method scores below 0.52.The authors therefore recommend reporting HiFD alongside its constituent components rather than a single privacy or utility number.Source: §4.2 · p. 8, §4.4 · p. 10

5 Conclusion

• Across 12 representative methods, macro cues are mostly retained, micro cues are partially degraded, and rPPG is silently destroyed by virtually all generative approaches.Most adversarial methods also show limited identity suppression against an ensemble of modern recognizers; no current method is simultaneously private and utility-preserving across all three levels.Source: §5 · p. 10
• UTILFACE is a curated, demographically balanced corpus assembled from four large-scale sources, paired with HiFD, which combines identity suppression, three-level utility preservation, and image quality without per-image attribute annotations.HiFD computes consistency between estimator outputs on original and de-identified faces.Source: §5 · p. 10
• HiFD inherits any bias in its pretrained estimators, and the released protocol scores L1 and L2 with one estimator per attribute.Ensembling for privacy and U3, and a substitution study, bound but do not remove this dependence; improved underlying models translate directly into more faithful scores.Source: §5 · p. 10

A UTILFACE Benchmark: Extended Details

• The deferred curation details include per-source statistics, the identity-deduplication threshold, and the pose-conditioned routing between GFPGAN and CodeFormer.The section also provides the empirical NIQE < 5.0 cutoff, the composite ranking used for hierarchical stratified sampling, and source-dataset licenses.Source: A UTILFACE Benchmark: Extended Details · p. 16
• The operational details also cover the NIQE < 5.0 quality cutoff and the composite ranking that drives hierarchical stratified sampling.Source-dataset licenses are included as part of the deferred details.Source: A UTILFACE Benchmark: Extended Details · p. 16
• Per-source statistics, the identity-deduplication threshold, and source-dataset licenses are among the implementation details supplied here.The section completes details deferred for space from the benchmark construction account.Source: A UTILFACE Benchmark: Extended Details · p. 16

A.1 Source Dataset Statistics

• No single source dataset provides both the identity diversity and image coverage needed for reliable privacy evaluation, motivating aggregation of four large-scale face datasets.The sources differ substantially in scale, identity count, and images per identity.Source: §A.1 · p. 16, Table 5 · p. 16

A.2 Identity-Aware Cleaning

• Cleaning removes images with zero or multiple detected faces, non-RGB formats, heavy blur, and cross-dataset duplicate identities.Identity pairs with ArcFace cosine similarity above 0.55 are merged across sources or have the lower-quality instance removed; this threshold yielded under 1% false merges while capturing over 90% of cross-dataset duplicates on a held-out set.Source: §A.2 · pp. 16–17

A.3 Resolution Enhancement

• Surviving low-resolution images are enhanced to 448×448 or 512×512, with upsampling capped at 4× and restoration chosen by estimated yaw.Faces above 30° yaw are routed to CodeFormer; the remainder go to GFPGAN, and samples undetectable by MTCNN after both attempts are discarded.Source: §A.3 · p. 17

A.4 Perceptual Quality Filtering

• Images with NIQE ≥5.0 are discarded; the cutoff was chosen because 95% of FFHQ images fall below NIQE = 5.0.This filter removes approximately 58% of the pool after Stage 2, disproportionately from the heavily blurred WebFace42M subset.Source: §A.4 · p. 17

A.5 Hierarchical Stratified Sampling

• Identity-level hierarchical stratified sampling addresses skew by selecting identities across 14 gender-by-ethnicity strata and retaining all images for each selected identity.The preceding pool overrepresented males, White subjects, Happy expressions, frontal poses, and ages 45–55; ranking also up-weights underrepresented expression, pose, and age categories by 1.5–4×.Source: §A.5 · p. 17

A.6 Licenses and Release

• UTILFACE is released as identity-indexed scripts and selected identity keys and filenames, not as a redistributed image archive, under the source datasets’ research-use terms.The authors acknowledge that source images were collected from public web imagery without individual consent, and state that HiFD is not a guarantee of anonymity against human observers or future recognizers.Source: §A.6 · pp. 17–18

B.1 Consistency Function Definitions

• HiFD defines consistency functions for an original face or video clip and its de-identified counterpart; the extended section gives closed forms for L1, L2, and L3 functions and normalization constants.Source: §B.1 · p. 18, B HiFD Metric: Extended Details · p. 18
• For L1, age uses truncated absolute error, ethnicity and macro-expression compare full posterior distributions, gender uses confidence-weighted binary posteriors, and landmarks use normalized mean error.Landmark error is capped at τNME = 0.10 and normalized by inter-ocular distance measured on the original image; the L1 aggregate is the unweighted mean of five consistency scores.Source: §B.1 · pp. 18–19
• For L2, gaze uses angular error normalized by a 90° cap, while micro-expression consistency is video-based and shares the categorical-task consistency function.Image-only U2 reduces to gaze; video-based results report gaze and micro-expression subscores separately before the final composite.Source: §B.1 · p. 19
• For L3, rPPG consistency compares blood-volume waveforms and derived heart rates, assigning zero to uncorrelated or anti-correlated signals and using a 20 bpm cap for heart-rate error.The method averages over an ensemble of rPPG estimators to reduce estimator-specific bias.Source: §B.1 · p. 19
• Privacy is measured as embedding-space dissimilarity and averaged across recognition models to reduce dependence on any single backbone.The quality score combines reference-based LPIPS perceptual fidelity and normalized inverse NIQE naturalness, with α = 0.5 by default.Source: §B.1 · pp. 19–20

B.2 Normalization-Constant Choices

• HiFD’s normalization constants are specified a priori and shared across methods, so they do not introduce method-specific bias.The section summarizes the constants and their rationale in Table 6.Source: §B.2 · p. 20, Table 6 · p. 20

B.3 Aggregation-Choice Ablation

• The comparison applies three aggregation schemes with uniform weights to the same five component scores: ¯P, Q, U1, U2, and U3.The weighted harmonic mean is justified because a method that fails identity suppression should not rank highly regardless of its utility profile.Source: §B.3 · pp. 20–21
• A near-zero component drives the harmonic mean toward zero, whereas the arithmetic mean largely hides it and the geometric mean scores it too leniently.For Adv-Makeup, ¯P = 0.012 contributes a reciprocal term of about 83, overwhelming the other four terms, each at most about 1.91.Source: §B.3 · p. 21
• Adv-Makeup ranks first under the arithmetic mean despite ¯P = 0.012, but falls to rank 12 under the harmonic mean, reversing its incorrect ranking against TIP-IM.The harmonic scores for Adv-Makeup and DeID-rPPG are an order of magnitude below TIP-IM’s, while the arithmetic mean places Adv-Makeup first and TIP-IM third.Source: §B.3 · p. 21, Table 7 · p. 21

C.1 Validation of the Consistency Paradigm

• Consistency predicts expression-label survival with AUC 0.981; mean consistency correlates with retained ground-truth accuracy at Spearman 0.944 and Pearson 0.992.Among 20,096 pairs with consistency above 0.8, 1.06% have a flipped true label; on contact-sensor data, ranking by −∆MAE correlates with U3 at Spearman 0.944 on MMPD and 0.993 on PURE.Source: §C.1.1 · p. 22, Table 8 · p. 22, §C.1.2 · p. 22

C.2 Operational Privacy at Fixed False-Accept Rates

• At FAR ≤10−3, ¯P ≈0.5 maps to 93–97% de-identification, while ¯P ≈0.01 maps to 0%; ¯P and success correlate at Spearman 0.783–0.804.The study calibrates ¯P against fixed false-accept thresholds for a recognizer threat model; human same-person judgments reproduce rankings within, but not across, paradigms.Source: §C.2 · p. 23, Table 9 · p. 23

C.3 Statistical Uncertainty

• TIP-IM ranks first in all 2,000 resamples, with a 0.097 margin over the runner-up (95% CI [0.086, 0.108]); video-axis intervals are wider than image-axis intervals.Intervals capture identity- and subject-sampling variability, not estimator or seed variability; image-only scores retain the same top three and correlate with the full ranking at Spearman 0.888.Source: §C.3 · pp. 23–24, Table 10 · p. 24

C.4 Estimator Substitution and Multi-Model Averaging

• Estimator substitutions and multi-model averaging preserve the published ranking exactly: HiFD Spearman is 1.000, with zero rank changes and the same top three.Alternative-estimator sub-scores correlate at 0.937–0.979, and averaging changes the composite by at most 0.002 per method.Source: §C.4 · p. 24, Table 11 · p. 24

C.5 Weight-Profile Sensitivity

• TIP-IM remains rank 1 as privacy weight wP sweeps from 0.2 to 0.6; at w3 = 0.6, Chameleon leads by 0.006, a margin within bootstrap uncertainty.The sweep varies one weight while splitting the remaining weight equally across the other four axes.Source: §C.5 · p. 25, Figure 7 · p. 25, Table 12 · p. 25

C.6 Quality Axis: Reference-Free Alternatives

• Replacing the published quality term with reference-free Qnr yields Spearman 0.951 with the same top three, while removing quality yields 0.979.The published Q and reference-free Qnr rank methods almost independently (Spearman 0.126), capturing source fidelity and freedom from artifacts, respectively.Source: §C.6 · p. 25, Table 13 · p. 26

D.1 Qualitative Comparisons

• The qualitative grids compare 12 methods across paradigms on two UTILFACE samples spanning pose and lighting variation.Rows group methods by paradigm and columns by method.Source: §D.1 · p. 26, Figure 8 · p. 26

D.2 Demographic Fairness Under Profile Reweighting

• Under Privacy-First and Clinical profiles, TIP-IM remains the top-ranked method, and the aggregate East-Asian-peak/Black-trough ethnicity ordering persists.TIP-IM’s gender gap is at most 0.021 and ethnicity range at most 0.019 across the two profiles; reweighting changes which component is amplified, not which subgroup is favored.Source: §D.2 · pp. 26–27, Figure 10 · p. 27, Figure 11 · p. 28

D.3 L3 Replication on the MMPD Dataset

• On MMPD, PURE’s U3 ordering replicates at Spearman 0.986, with the same top three and bottom three methods, and contact-sensor columns give the same ranking.Adversarial methods average U3 = 0.566 on MMPD versus 0.424 for generative methods; the ordering remains separated across illumination and motion conditions.Source: §D.3 · pp. 28–29

E Human Perceptual Study

• The human study tests whether naturalness, attribute preservation, and identity judgments align with selected estimator-based HiFD axes, not with human perception in general.It specifically tests whether naturalness tracks reference-free quality rather than LPIPS, whether human attribute judgments track U1, and whether privacy scores track identity matching.Source: E Human Perceptual Study · p. 29

E.1 Protocol

• Raters judge unpaired images for naturalness and artifacts, then compare original/de-identified pairs for four attributes and identity; 22 of 24 collected raters remain after prespecified exclusions.Thirty identities were sampled across UTILFACE’s 14 gender-by-ethnicity strata; the study also specifies rater-bootstrap intervals and method-level correlation analyses.Source: §E.1 · pp. 29–30

• 3.32/5: originals’ naturalness was modest; PGD, Adv-Makeup, and MI-FGSM were statistically indistinguishable from originals, and restoration keeps UtilFace’s naturalness ceiling below 5.Raters correctly declined to call the restored UtilFace images unedited photographs.Source: §E.2 · pp. 30–31
• 0.000 (p = 1.00): LPIPS had no detectable rank association with human naturalness; BRISQUE’s 0.517 association was not significant against method sampling.HiFD (Balanced) correlated −0.62 with human naturalness, the intended trade-off rather than a disagreement.Source: §E.2 · p. 31
• 0.762 [0.567, 0.848]: human attribute preservation tracked U1, while CIAGAN’s human score was 0.159 [0.085, 0.244] versus its U1 of 0.468.TI-DIM’s human preservation score was 0.881 versus U1 of 0.757.Source: §E.2 · p. 31
• −1.000: human same-person rates matched fixed-FAR privacy rankings exactly within adversarial and generative paradigms, despite a pooled correlation of −0.549 (p = 0.07).The authors characterize P̄ as a machine-recognizer privacy score, not a model of human adversaries.Source: §E.2 · p. 31
• Across all sixteen threshold combinations, no method both hid identity from humans (<0.5 same-person) and preserved attributes (>0.7); only CIAGAN cleared the identity bar but destroyed attributes.Source: §E.2 · p. 31
• The authors limit conclusions to three specific claims and one replication, not general human validation; method correlations rest on twelve points and raters were convenience-sampled.Naturalness Krippendorff’s α was 0.42, below the conventional 0.667.Source: §E.2 · p. 31

F.1 Hardware and Software Environment

• The inference jobs used AMD MI250X GPUs, rocm/6.0.3, PyTorch 2.3.1, and global random seed 42 across NumPy, PyTorch, and Python.Estimators ran in inference mode with a fixed batch order.Source: §F.1 · p. 32

F.2 Pretrained Estimator Checkpoints

• Table 16 lists every pretrained HiFD checkpoint, released by its original authors under research-only licenses.Source: §F.2 · p. 32