PaperlayerView exact revision on arXiv

Source-linked AI summary

Deep Face Representations for Differential Morphing Attack Detection

Ulrich Scherhag, Christian Rathgeb, Johannes Merkle, Christoph Busch

arXiv:2001.01202v2cs.CR

TL;DR

Prior morphing attack detection studies often used limited or unrealistic image distributions, making real-world robustness difficult to assess. This paper builds a more realistic database and evaluates differential MAD using deep face representations. The proposed deep-representation methods outperform prior differential schemes and show robustness to varied post-processing.

  • Problem

    Prior MAD evaluations often used limited morphing tools, omitted realistic post-processing or probe variation, and raised concerns about overfitting and generalization to challenging scenarios.

    Source: §I · p. 1

  • Method

    The authors build a database from FERET and FRGCv2 with varied morph tools and post-processing, then evaluate differential MAD classifiers trained on deep representations of reference–probe pairs.

    Source: §I · p. 2

  • Results

    Deep-representation differential MAD outperforms previously proposed differential schemes and is highly robust to compression, resizing, and print-scan post-processing.

    Source: §I · p. 2, §VI · p. 13

  • Takeaways & Limitations

    Realistic evaluation data should account for probe as well as reference-image quality, since both strongly influence MAD detection performance.

    Source: §VI · p. 13

    Strong illumination and focus variation in probes negatively affects detection performance, making such conditions especially challenging for differential MAD.

    Source: §VI · p. 13

Abstract

from arXiv · show

The vulnerability of facial recognition systems to face morphing attacks is well known. Many different approaches for morphing attack detection have been proposed in the scientific literature. However, the morphing attack detection algorithms proposed so far have only been trained and tested on datasets whose distributions of image characteristics are either very limited (e.g. only created with a single morphing tool) or rather unrealistic (e.g. no print-scan transformation). As a consequence, these methods easily overfit on certain image types and the results presented cannot be expected to apply to real-world scenarios. For example, the results of the latest NIST Face Recognition Vendor Test MORPH show that the submitted MAD algorithms lack robustness and performance when considering unseen and challenging datasets. In this work, subsets of the FERET and FRGCv2 face databases are used to create a large realistic database for training and testing of morphing attack detection algorithms, containing a large number of ICAO-compliant bona fide facial images, corresponding unconstrained probe images, and morphed images created with four different tools. Furthermore, multiple post-processings are applied on the reference images, e.g. print-scan and JPEG2000 compression. On this database, previously proposed differential morphing algorithms are evaluated and compared. In addition, the application of deep face representations for differential morphing attack detection algorithms is investigated. It is shown that algorithms based on deep face representations can achieve very high detection performance (less than 3\%~\mbox{D-EER}) and robustness with respect to various post-processings. Finally, the limitations of the developed methods are analyzed.

I. INTRODUCTION

Prior MAD evaluations often used limited or unrealistic image distributions, leaving reported performance difficult to apply to real-world scenarios. The paper contributes a realistic database, deep-representation differential detection, and comparative evaluation.

I. INTRODUCTION

  • Prior evaluations often omitted realistic post-processing and probe variation, while training and test sets commonly reused morphing tools, raising concerns about overfitting.NIST FRVT MORPH results found that no submitted algorithm performed satisfactorily across all test scenarios.Source: §I · pp. 1–2
  • The paper also contributes a conceptual categorization of published MAD approaches, including technical considerations, tradeoffs, and a detailed survey of differential schemes.Source: §I · p. 2
  • The new database combines ICAO-compliant bona fide and morphed references, unconstrained probes, four morphing tools, and realistic post-processings.Probe images vary in pose, expression, illumination, and focus; reference images receive JPEG2000 compression or print-scan transformations.Source: §I · p. 2
  • The proposed differential MAD extracts deep representations from reference–probe pairs and trains machine-learning classifiers to detect morph-induced alterations.The representations come from state-of-the-art face recognition systems.Source: §I · p. 2
  • Deep-representation MAD outperforms previously proposed differential schemes, a finding also confirmed by the NIST FRVT MORPH report.The evaluation applies a commercial and two open-source deep face recognition systems.Source: §I · p. 2

II. RELATED WORK

Single-image MAD examines a potentially morphed image alone and includes texture, forensic, and deep-network approaches. These methods target different traces left by morphing.

A. Single Image MAD

  • Single-image MAD includes texture descriptors, forensic image analysis, and deep neural networks, which target different potential morphing artefacts.Source: A. Single Image MAD · p. 2
  • Texture descriptors detect smoothing, ghosting, half-shade effects, and shifted or distorted regions produced when images are averaged or misaligned.Examples include Local Binary Patterns and Binarized Statistical Image Features.Source: A. Single Image MAD · pp. 2–3
  • Forensic approaches can use changes to sensor-pattern noise, double-compression artefacts, or inconsistencies in illumination and color as morph traces.Source: A. Single Image MAD · p. 3
  • Deep networks are either trained or retrained for MAD, or existing networks’ embeddings are used; the latter were not trained on morphed faces.Training MAD networks requires high-variance data to avoid overfitting to algorithm- and database-specific artefacts.Source: A. Single Image MAD · p. 3

B. Differential MAD

Differential MAD compares a potentially morphed reference with a trusted live capture, either through feature comparisons or by reversing the morphing process. The paper builds a database from FERET and FRGCv2 because no public database then met its realism requirements.

B. Differential MAD

  • Differential MAD compares features from a trusted live capture and a possible morph, or attempts to reverse the morphing process through explicit or GAN-based demorphing.Feature-based methods may estimate differences between feature vectors or landmark distances.Source: B. Differential MAD · p. 3

III. CREATION OF MAD DATABASE

  • The database design calls for ICAO-compliant reference images, multiple morphing tools, realistic post-processing, and unconstrained probes resembling degraded eGate captures.Probe variation may include pose, expression, illumination, background, appearance, and age.Source: §III · p. 3

A. Selection of the Facial Image Database

  • FERET and FRGCv2 were selected because they provide suitable references and less-constrained probes, including variations in pose, expression, illumination, sharpness, and background.Source: A. Selection of the Facial Image… · p. 3
  • The databases span only three and two years, respectively, limiting reference–probe age variation compared with the longer gaps possible in bona fide border-control checks.The authors note that large age differences are more expected for bona fide authentication than for attacks using current images of an attacker and accomplice.Source: A. Selection of the Facial Image… · pp. 3–4

B. Selection of Image Candidates

The image-selection procedure distinguishes passport-like references from less-constrained probes, then limits image reuse while constructing the morph set. Selection criteria also constrain the available sample and involve some subjective judgment.

B. Selection of Image Candidates

  • Reference images meet ICAO requirements except face alignment, which is adjusted using detected eye coordinates and a fixed 180-pixel intra-eye distance.Source: B. Selection of Image Candidates · p. 4
  • Probe images are recognizable but may be partially illuminated or slightly out of focus; FERET probes vary in expression and pose, while FRGCv2 adds background, illumination, and sharpness variation.Probe images are converted to grayscale.Source: B. Selection of Image Candidates · p. 4
  • The image-selection criteria limit the number of usable images, and the selection is partly subjective.The selected-image list is made available to support reproducibility.Source: B. Selection of Image Candidates · p. 4

C. Selection of Images and Morph Creation

  • Where possible, bona fide references are kept disjoint from morph inputs, and references and probes come from different recording sessions to add image and temporal variation.Source: C. Selection of Images and Morph… · p. 4
  • Morph inputs are paired in sorted order subject to same-sex and one-glasses-image criteria, and each input image is used for only one morph.The inputs are discarded after morph generation to avoid repeated image parts in MAD training.Source: C. Selection of Images and Morph… · p. 4
  • Table II reports database composition, with the number of morph images multiplied by the number of morphing tools used.Source: Table II · p. 5

D. Morphing Algorithms

The database uses four morphing tools that produce different visual artefacts and image quality. Their varied outputs support evaluating MAD across morph quality levels.

D. Morphing Algorithms

  • The four tools span low-artifact FaceFusion, neck- and hair-artifact FaceMorpher, edge-landmark OpenCV, and background-composited UBO-Morpher.No manual post-processing was applied to the morphs.Source: D. Morphing Algorithms · p. 5
  • The varied morphing tools enable investigation of how high- and low-quality morphs affect MAD training and testing.The morphs were generated without manual post-processing.Source: D. Morphing Algorithms · p. 5
  • Each database contains four sets of tool-generated morphs, and the bona fide and morphed images are ICAO-normalized to 720×960 pixels.The sets comprise 4×529 morphs for FERET and 4×964 for FRGCv2.Source: D. Morphing Algorithms · p. 5

E. Post-Processing

The section describes varied passport-image post-processings and evaluates face-recognition vulnerability to morphs. Recognition systems were highly vulnerable to equal-weight morphs, particularly on FERET.

E. Post-Processing

F. Validation of Attack Potential

  • ArcFace and COTS were highly vulnerable to MAs50, especially on FERET; MAs25 success chances fell, while FaceFusion and UBO-Morpher morphs passed more often.The vulnerability results are summarized in Table IV.Source: Table IV · p. 7, F. Validation of Attack Potential · p. 7
  • Vulnerability was assessed on NPP images with ArcFace and COTS; thresholds targeted 0.1% FMR, and score distributions compare genuine, impostor, and morph matches.Preliminary studies found recognition performance only slightly affected by the tested post-processings.Source: F. Validation of Attack Potential · pp. 6–7, Figure 6 · p. 7
  • MAs50 use equal 50/50 contributions; MAs25 use 25% attacker and 75% accomplice contributions, considered potentially more realistic if equal-weight morphs are readily spotted.Source: F. Validation of Attack Potential · p. 7
  • When the attacker contributes 25%, the greater morph-to-attacker-probe distance means differential MAD generally detects these attacks more often than equal-weight MAs.Source: F. Validation of Attack Potential · p. 8

IV. MAD BASED ON DEEP FACE REPRESENTATIONS

The paper uses pretrained deep face representations to build differential MAD classifiers. Difference vectors separate bona fide and morphed attempts, with separation varying by feature extractor and dataset.

IV. MAD BASED ON DEEP FACE REPRESENTATIONS

  • The approach targets the large intra-class variance of realistic probes, which existing differential MAD algorithms struggle to handle.Morph representations may deviate from attacker probe representations, while features not trained on morphs may reduce tool-specific overfitting risk.Source: §IV · p. 8
  • Retraining deep networks risks overfitting because the available MAD database has limited training images and far fewer subjects.The concern also applies when only lower layers are retrained.Source: §IV · p. 8
  • Pretrained ArcFace, Eyedea, and FaceNet networks serve as feature extractors, avoiding direct retraining of their high-complexity models on the MAD data.Their representations are extracted from the lowest layer.Source: §IV · p. 8
  • The system subtracts probe and reference deep representations, then uses an RBF-kernel SVM to classify bona fide attempts versus morph attacks.SVM showed the best accuracy among the tested classifiers.Source: §IV · p. 8, Figure 8 · p. 8
  • Difference vectors from ArcFace and Eyedea separate morphs from bona fide attempts on both databases; ArcFace nearly error-free on FERET, while FaceNet is expected to have higher error rates.The separation is illustrated by MDS-reduced scatter plots.Source: §IV · pp. 8–9, Figure 9 · p. 9

V. EXPERIMENTS

The experiments benchmark differential MAD accuracy without post-processing, test robustness to post-processing, and examine score distributions. Training and testing are separated by source database.

V. EXPERIMENTS

  • Training on FERET and testing on FRGCv2, or vice versa, enforces source-database separation and variation between training and test image characteristics.Source: §V · p. 9
  • Each training set contains images with only one of four post-processings and morphs from only one of four tools.The experiments do not combine multiple post-processings or morphing tools in a training set.Source: §V · p. 9
  • Detection accuracy is reported as D-EER, where APCER equals BPCER; BPCER10 is also reported at APCER=10%.Source: §V · p. 9

A. Detection Accuracy in the Absence of Post-Processing and Comparison with other MAD Algorithms

On non-post-processed images, deep-representation differential MAD—especially ArcFace—outperforms most evaluated methods, while performance varies with probe quality and morph characteristics.

A. Detection Accuracy in the Absence of Post-Processing and Comparison with other MAD Algorithms

  • The evaluation compares ArcFace, Eyedea, and FaceNet representations with texture, landmark, and demorphing methods adapted for differential detection.Texture methods use reference–probe histogram differences; landmark methods use Dlib or Wing Loss landmarks.Source: A. Detection Accuracy in the Absence… · p. 9
  • All evaluated MAD algorithms perform much worse on FRGCv2 than FERET, which the authors attribute to lower-quality, more variable probes.FRGCv2 probes vary more in illumination, background, and sharpness; the authors conclude probe quality strongly affects detection.Source: A. Detection Accuracy in the Absence… · p. 10
  • ArcFace-based MAD achieves D-EERs of 1%–7%, while Eyedea-based MAD reaches 3%–17% and FaceNet features are less suitable.The comparison is on NPP images using different morphing tools for training and testing.Source: A. Detection Accuracy in the Absence… · p. 10, Table V · p. 10
  • Texture methods perform near randomly on FRGCv2, landmark methods are generally poor, and demorphing with COTS recognition performs comparatively well there.Demorphing is slightly worse than ArcFace for higher-quality morphs and comparable for lower-quality morphs on FRGCv2, but falls behind Eyedea on FERET.Source: A. Detection Accuracy in the Absence… · p. 10
  • The authors assume color probes would yield comparable detection performance to grayscale probes for deep representations and methods that convert images to grayscale.This is an assumption based on the expected limited effect of grayscale probes on deep face recognition performance.Source: A. Detection Accuracy in the Absence… · pp. 10–11
  • Higher-quality FaceFusion and UBO-Morpher morphs are harder to detect, and matching the training and test morphing tools yields the best results but only limited tool influence.The authors associate the harder morphs with automatic post-processing that replaces artefact-rich regions outside the face.Source: A. Detection Accuracy in the Absence… · p. 11

B. Robustness against Image Post-Processing

Deep-representation MAD remains robust across the tested reference-image post-processings, with ArcFace leading the evaluated systems.

B. Robustness against Image Post-Processing

  • The robustness evaluation separately applies four reference-image post-processings during training and testing, training with FaceMorpher and OpenCV and testing with FaceFusion and UBO-Morpher.The test morphing tools were chosen because higher-quality morphs were harder to detect and training-tool influence was limited.Source: B. Robustness against Image… · p. 11
  • The comparison excludes the other previously considered MAD algorithms because they did not achieve comparable performance.The demorphing footnote notes that its outer region is replaced with the attacker’s image, which is expected to increase similarity to the attacker’s probe.Source: B. Robustness against Image… · p. 11
  • Training on post-processed images has no noticeable effect, while testing on them can slightly affect performance; the proposed deep-representation methods are not severely influenced overall.ArcFace remains far ahead in detection performance, followed by Eyedea.Source: B. Robustness against Image… · p. 11

C. Analysis of Score Distributions

ArcFace score distributions separate bona fide attempts from morphing attacks, but separation differs by database and error type.

C. Analysis of Score Distributions

  • FERET scores separate bona fide attempts and morphing attacks better than FRGCv2 scores, with distributions remaining similar across training morphing tools and test post-processings within each database.The figure plots decision-score densities; low scores indicate bona fide attempts and high scores indicate morphing attacks.Source: C. Analysis of Score Distributions · p. 12, Figure 10 · p. 12
  • FRGCv2 errors include bona fide samples confidently scored as attacks, whereas FERET errors occur mostly when morphs receive bona fide-like scores.The authors connect FRGCv2 false alarms with probe expression variation and headgear, and note that such variation is realistic in border control.Source: C. Analysis of Score Distributions · p. 12
  • Good-quality morphs that closely resemble the sample can evade detection, while expression changes and headgear in probes are associated with wrongly classified bona fide attempts.Examples are identified for attack errors on FERET and bona fide errors on FRGCv2.Source: C. Analysis of Score Distributions · p. 12, Figure 11 · p. 12, Figure 12 · p. 12
  • Despite confidently misclassified samples that threshold adjustment is unlikely to correct, both databases show clearly separated score distributions and stable error rates across a considerable range of thresholds.The authors describe this threshold stability as making the algorithm suitable for practical application.Source: C. Analysis of Score Distributions · p. 12

D. NIST Face Recognition Vendor Test MORPH

The NIST evaluation reports strong performance on unseen datasets, while the conclusions identify robustness, data variation, and classifier choice as important boundaries and findings.

D. NIST Face Recognition Vendor Test MORPH

  • On unseen datasets in NIST FRVT MORPH, the proposed method achieved rates comparable to this work and outperformed other submitted methods by orders of magnitude.The submitted system used ArcFace and trained its SVM on bona fide and morphed references from FERET and FRGCv2, including all morphing tools used in the work.Source: D. NIST Face Recognition Vendor Test… · p. 13

VI. CONCLUSION

  • Deep-representation MAD is reported as robust to compression, resizing, and print-scan, with training-set post-processing not significantly affecting performance.The authors state that scanned images are therefore not necessary for training.Source: §VI · p. 13
  • Morphs with obvious artefacts are generally detected more accurately, and performance slightly degrades when training and evaluation use different morphing algorithms.Source: §VI · p. 13
  • Strong probe variations in illumination and focus challenge differential MAD; detection is higher on the FERET subset than FRGCv2 in the reported experiments.The FRGCv2 probes also vary in focus, while FERET probes have only slight expression and pose variation.Source: §VI · p. 13
  • SVM classifiers are generally the most competitive among the tested classifiers across most experiments.The comparison includes AdaBoost, Gradient Boosting, Random Forest, and SVM.Source: §VI · p. 13
  • For MAD, open-source ArcFace features may be better suited than commercial-system features, and realistic testing data matter despite privacy-related barriers to database exchange.The authors state that both reference and probe image quality strongly influence detection performance.Source: §VI · p. 13
Loading 2001.01202v2…