Source-linked AI summary
FaceKit: a Toolkit for Interpretable Facial Phenotyping, Synthetic Image Generation and Privacy Analysis in Rare Diseases
Hongzhuo Chen, Zhanliang Wang, Florent Pollet, Mian Umair Ahsan, Joshua Bie, Tzung-Chien Hsieh, Peter Krawitz, Cong Liu, Wendy K Chung, Chunhua Weng, Gamze Gürsoy, Kai Wang
TL;DR
FaceKit addresses the lack of reproducible quantitative facial measurements despite standardized phenotype terminology. It converts frontal photographs into inspectable geometric features scored against normative references; in a curated database, 62 of 84 assessable mapped terms deviated in the predicted direction.
Problem
Facial features are often described qualitatively, and existing image-based approaches do not consistently link them to reproducible, interpretable measurements.
Source: §1 · p. 2Method
FaceKit converts facial images into structured geometric measurements, reports z-scores against a normative reference, and supports additional geometric features.
Source: §1 · p. 2Results
62 of 84 assessable HPO terms mapped to measurements that deviated from a healthy reference in the direction predicted by the term.
Source: §5 · p. 25Takeaways & Limitations
FaceKit provides inspectable geometric quantities for recording facial phenotype rather than relying solely on free-text descriptions or uninspectable model outputs.
Source: §5 · p. 24The privacy audit used a generator fitted only to the training partition because the released generator had seen every available image and had no held-out reference.
Source: §4 · p. 24
Abstract
from arXiv · showhide
Many rare genetic diseases are associated with recognizable craniofacial features. However, traditional approaches for describing facial morphology rely largely on qualitative clinical observation and free-text descriptions, which are often subjective, non-standardized, and difficult to reproduce across observers and institutions. Although the Human Phenotype Ontology (HPO) provides controlled terms for describing facial features, these terms are typically categorical rather than quantitative and may vary depending on examiner experience and interpretation. Here, we present FaceKit, a computational framework for quantitative facial phenotyping from frontal facial photographs. FaceKit extracts standardized measurements of facial landmarks and derived 120 morphological features, then reports feature-level z-scores representing deviation from population reference distributions. The reference distributions are built from the FairFace dataset spanning diverse ancestral groups. We evaluated FaceKit on a curated subset of the GestaltMatcher Database covering 50 rare-disease cohorts. In addition to quantitative facial analysis, FaceKit includes synthetic facial image generation to support rare disease model development and data augmentation. We also performed privacy evaluation to assess whether synthetic images reveal identifiable information from real patient photographs and could compromise patient privacy. Across disease case studies, FaceKit-derived quantitative measurements captured known facial features associated with rare genetic disorders and provided objective support for clinical phenotyping. Together, these results establish FaceKit as a useful tool for quantitative phenotyping, and has the potential to improve rare disease diagnosis, support genotype-phenotype studies, and enable more reproducible clinical characterization across diverse patient populations.
1 Introduction
Facial morphology is clinically useful in rare-disease characterization, but current descriptions and image-based approaches do not consistently provide reproducible, interpretable measurements. FaceKit addresses this gap with a unified pipeline for quantifying facial features from images.
Prior approaches
- Expert observation and HPO terminology have traditionally described facial patterns, while neural networks and vision-language models have also been used for rare-disease recognition and phenotype-term identification.The paper cites neural network work and vision language model work [3].Source: §1 · p. 2
Gap in existing approaches
- HPO standardizes phenotype vocabulary, but many facial features remain qualitative and are not linked to reproducible image measurements.Learning-based models often output disease rankings, probabilities, or phenotype labels without readily expressible underlying facial characteristics.Source: §1 · p. 2
FaceKit
- FaceKit computes 120 interpretable facial measurements, reports each as a z-score against a normative reference, and supports disease-aware feature selection and extensible feature definitions.The toolkit integrates landmark extraction and visualization, frontal-pose filtering, geometric normalization, and cohort average-face generation.Source: §1 · p. 2
2 Methods
FaceKit links image preprocessing and interpretable geometric phenotyping with cohort-conditioned image generation and assessments of phenotype preservation and privacy. Its methods standardize facial landmarks and measurements, compare synthetic images with real cohorts, and audit the generator’s similarity to training data.
2 Methods
- FaceKit combines image preprocessing, landmark-based phenotyping, cohort-conditioned synthetic-image generation, phenotype-preservation testing, and privacy assessment.The workflow can support downstream machine-learning tasks and generative-model development, and evaluates whether generated images retain cohort geometry or resemble training photographs.Source: §2 · p. 2, Figure 1 · p. 3
2.1 Image Preprocessing
- Images are screened for frontal pose using yaw and pitch limits of 15° and a roll limit of 10°, with landmark overlays available for inspection.FaceKit extracts 478 three-dimensional landmarks and stores their coordinates with image identifiers and dimensions.Source: §2.1 · pp. 3–4
2.2 Geometric Phenotype Extraction
- FaceKit corrects head pose using a canonical depth profile, then normalizes landmarks and derives 120 inspectable geometric features across facial regions.Measurements are expressed as ratios to anatomically defined reference lengths; the canonical depth table performed best among the tested alternatives.Source: §2.2 · pp. 4–5
2.3 Normative scoring
- Each measurement is expressed as its departure from a control mean in control standard deviations, using a reference built from frontal FairFace images pooled across ancestry groups.The distributed reference uses 2,051 FairFace images, of which 886 passed the same frontal-pose gate; users may provide their own controls.Source: §2.3 · p. 5
2.4 Average-face generation
- Average-face generation aligns cohort images to a common landmark configuration and combines them into a visual summary of shared facial morphology.Users can adjust output resolution and optionally apply edge smoothing, a transparent background, or blending with a destination face.Source: §2.4 · p. 5
2.5 Synthetic Facial Image Generation
- FaceKit supports user-trained or pre-trained cohort-conditioned StyleGAN3 generators, with optional image restoration and colorization before training.The study’s generator was trained on 4,559 prepared images from 50 cohorts and produces 256 × 256 images.Source: §2.5 · p. 6
2.6 Phenotype preservation of synthetic images
- Phenotype preservation is tested by comparing real and synthetic measurements within cohorts and comparing each cohort’s effects against pooled other cohorts.The comparisons use Cohen’s d, effect-sign agreement, correlations between synthetic and real effect vectors, and variance ratios.Source: §2.6 · p. 6, Figure 2 · p. 7
2.7 Privacy assessment of the pre-trained generator
- Privacy assessment separately evaluates identity similarity and photograph-appearance similarity using nearest-neighbor comparisons against cohort training images.Identity uses patient-disjoint training and held-out partitions and multiple recognition models; appearance uses LPIPS, while nearest-neighbor adversarial accuracy summarizes set proximity.Source: §2.7 · pp. 7–9
3 Results
The results assess FaceKit’s quantitative phenotyping across disease cohorts and reference conditions, alongside the cohort-level characteristics and privacy of its synthetic images.
3.1 Quantitative measurements reproduce annotated facial phenotypes
- Across 84 assessable HPO terms, 62 (73.8%) deviated in the predicted direction; among 78 statistically testable terms, 39 separated from reference after correction.Thirty-three significant terms changed as predicted and six in the opposite direction; for hypertelorism in Crouzon syndrome, the mapped ratio was elevated by 1.04 reference SD (q = 4.2×10−22) and was the most deviant of 120 measurements.Source: §3.1 · p. 10
3.2 Depth for pose correction is better taken from a canonical face than from the image
- On 4,559 GMDB images, canonical depth reduced mean absolute pitch correlation from 0.425 to 0.303 while increasing ICC from 0.370 to 0.391 and AUC from 0.703 to 0.714.Residual landmark error was 4.82% uncorrected, 5.06% under the flat-face assumption, and 0.00% with the canonical table.Source: §3.2 · p. 11, Table 1 · p. 12
3.3 The measurements transfer to an independent cohort and degrade predictably with image resolution
- In RDFace, same-condition images were closer by 1.34 standardized units than different-condition images, and separation persisted after removing flagged repeat patients and across resolution gates.The result was 10.7 standard deviations below the stratified null (p = 5.0 × 10−4); it remained 4.8 standard deviations below the null at 224 pixels, with 51 images from 15 conditions.Source: §3.3 · p. 14, Figure 6 · p. 15
- The catalogue was stable to about 128 pixels, but median absolute measurement shift rose from 0.06 reference SD at 128 pixels to 0.23 at 64 pixels.At 128 pixels, 36 of 120 measurements shifted by more than 0.1 reference SD; the authors identify 128 pixels as the operating floor.Source: §3.3 · p. 14, Figure 6 · p. 15
3.4 Age and image resolution change the measurements but not the reference
- Age and resolution conditioning did not materially change phenotyping conclusions: age conditioning reproduced 24 versus 27 terms, while resolution conditioning changed one verdict, from 33 to 34.Age conditioning had exact McNemar p = 0.375 and resolution conditioning p = 1.0; age-sensitive measurements showed no greater gain, and the resolution-sensitivity association was ρ = +0.17 (p = 0.134).Source: §3.4 · pp. 15–16, Figure 7 · p. 16
3.5 Ancestry enters through the reference, not through the disease effect
- An ancestry-conditioned reference increased mean absolute effects from 0.474 to 0.536, with greater gains for more ancestry-sensitive measurements.The higher-sensitivity half gained a median +0.121 versus −0.012 for the less-sensitive half (Mann–Whitney p = 0.0098); the headline reproduced-term count rose from 29 to 34 (exact McNemar p = 0.125).Source: §3.5 · p. 17
- For 22q11.2 deletion syndrome, four of five significant ancestry interactions among 120 measurements disappeared on a log scale, while canthal_to_pupillary_ratio remained significant.The four lip terms shifted from q = 0.0029 to 0.042 on the raw scale to q = 0.22 to 0.74 on the log scale; canthal_to_pupillary_ratio had partial η2 = 0.019.Source: §3.5 · p. 17
- For 22q11.2 deletion, proportional disease deviations were similar across three ancestry groups, although reference baselines differed; FaceKit z-scores should therefore use ancestry-conditioned references when ancestry is recorded.For the catalogue’s mostly ratio measurements, the authors recommend log-scale group comparisons; skin tone’s interaction with vermilion-border placement under different acquisition conditions remained uncontrolled.Source: §3.5 · p. 18, Figure 8 · p. 18
3.6 Synthetic images reproduce cohort-level geometric phenotypes
- The median absolute Cohen’s d between synthetic and real measurements was 0.111; 74.1% of pairs were below |d| = 0.2 and 98.6% below |d| = 0.5.Among cohorts mapped to at least ten measurements, median d ranged from 0.041 to 0.210; it was 0.076 for cohorts with at least 100 training patients and 0.181 for cohorts with 13 to 19.Source: §3.6 · p. 19, Figure 9 · p. 20
- Across 1,013 syndrome–measurement pairs, synthetic-versus-real effect directions agreed for a median 86.7% of measurements, with median Spearman ρ = 0.837 and Pearson r = 0.866.The 183 sign disagreements (18.1%) were concentrated near zero real effects: 83.6% had real |d| < 0.2. Agreement weakened in cohorts with fewer training patients, with median Spearman ρ of 0.604 for cohorts with 13 to 19 patients versus 0.945 for cohorts with 60 to 99.Source: §3.6 · p. 19, Figure 9 · p. 20
- Synthetic measurement variance was lower than real-image variance for 58.1% of pairs, with a median ratio of 0.94; narrowing was concentrated in the eye (0.91), mouth and lips (0.92), nose (0.92), and chin and jaw (0.94).Eyebrow and forehead ratios were 1.04 each, and the philtrum ratio was 1.16; the reduced variation in central facial features should be considered when using synthetic images for data augmentation.Source: §3.6 · p. 19, Figure 9 · p. 20
3.7 Synthetic images show no detectable excess similarity to training identities
- At all five tested operating points, no recognition model flagged synthetic faces above the held-out reference; at 1% false acceptance, rates were 0.17% for ArcFace, 0.13% for AdaFace, and 0.15% for LVFace.Adversarial-accuracy privacy losses were +0.013 for ArcFace, +0.020 for AdaFace, and +0.012 for LVFace; the authors interpret the small shift as movement toward the training partition as a whole, not reproduction of individual patients.Source: §3.7 · pp. 21, 23, Table 2 · p. 21, Figure 10 · p. 22
- LPIPS did not distinguish same-person from different-person images well, yet synthetic images were closer to training photographs: 8.6% fell below the 5% reference threshold and 26.4% below the 20% threshold.Because LPIPS responds to texture, pose, and illumination rather than identity, this proximity reflects inherited cohort acquisition characteristics, not evidence that particular patients were reproduced.Source: §3.7 · pp. 21–22, Figure 10 · p. 22
4 Discussion
FaceKit offers an interpretable workflow for geometric phenotyping, but its measurements represent only part of facial phenotype and are not diagnostic. Synthetic-cohort reliability varies with training-cohort size, while privacy conclusions are bounded by the audit design and the toolkit’s measurements have additional technical and validation limits.
4 Discussion
- Most mapped measurements deviated from a normative reference in the direction encoded by their corresponding HPO term, though only a subset reached statistical significance; FaceKit is a partial geometric representation, not an HPO-term detector or diagnostic tool.Source: §4 · p. 23
- Synthetic images reproduced cohort geometry, but agreement was weakest for cohorts with 13–19 patients and could not be separated from sampling noise.The correlation between synthetic and real cohort effects declined to 0.604 in the smallest cohort tier; such cohorts should be checked against real images before synthetic images are used in their place.Source: §4 · p. 23
- The privacy assessment supports distributing the pre-trained generator, but its identity and appearance findings come with distinct qualifications.The released generator’s privacy was inferred from an audited generator fitted to 3,602 rather than 4,559 images; the audit used specific recognition models and a perceptual metric, not a formal privacy guarantee.Source: §4 · pp. 23–24
- Geometric measurements depend on landmark accuracy and are sensitive to pose, expression, resolution, cropping, and acquisition conditions; appearance-only traits and some normalization choices also limit feature coverage.The catalogue is stable to about 128 pixels of short side, but below that acquires systematic offsets; measurements should not be treated as equivalent to calibrated three-dimensional measurements.Source: §4 · p. 24
- Clinical interpretation requires further validation across larger cohorts and demographic and imaging subgroups; the reported ancestry-invariance result rests on one syndrome and groups of 31–352 patients.The smaller ancestry groups cannot exclude effects large enough to matter clinically.Source: §4 · p. 24
5 Conclusion
FaceKit turns frontal photographs into inspectable geometric measurements on a normative scale. The results support measurement and cohort comparison, while clinical interpretation remains uncalibrated and the method is not intended to diagnose.
5 Conclusion
- FaceKit converts a frontal photograph into 120 inspectable geometric measurements reported on a normative scale, rather than an unexamined model output or free-text description.The measurements’ definitions are open to inspection.Source: §5 · p. 24
- 62 of 84 assessable HPO terms mapped to measurements deviated from a healthy reference in the direction predicted by the term, with failures traceable to offsets, normalization, or appearance-based phenotypes.The pose-correction comparison favored depth from a fixed canonical face over per-image detector depth, which lowered intra-patient reliability below no correction.Source: §5 · p. 25
- The generator reproduced cohort geometry, with modestly narrower within-cohort variation and weaker agreement for cohorts with fewer training patients; its outputs showed no identity signal beyond held-out real photographs in the reported audit.The generator privacy comparison was irrespective of how many patients a cohort contributed.Source: §5 · p. 25
- FaceKit is an instrument for cohort-level measurement and structured downstream analysis, not diagnosis, and its quantitative ranges are not yet calibrated for clinical interpretation.Clinical use requires reference distributions validated across broader ancestry, age, and acquisition conditions, alongside improved landmark detection and normalization.Source: §5 · p. 25
8 Author’s Contribution
The author contributions assign study conception, implementation, experiments, analysis, writing, software testing, dataset provision, and study guidance across the named contributors.
8 Author’s Contribution
- H.C. and K.W. conceived the study; H.C. and F.P. implemented the method and experiments, while H.C. and Z.W. analyzed results and wrote the manuscript.J.B. tested the software; T.C.H. and P.K. provided the dataset; M.U.A., I.M.C., C.L., W.K.C., C.W., and G.G. advised, and K.W. guided execution.Source: §8 · p. 25