PaperlayerView exact revision on arXiv

Source-linked AI summary

Revisiting Face Recognition for Monozygotic Twins: The Celeb Twins Test Set

Michael Zang, Haiyu Wu, Mrinal Sharma, Kevin W. Bowyer

arXiv:2609.01141v1cs.CVcs.AI

TL;DR

Distinguishing monozygotic twins remains an open face-recognition problem. This paper analyzes CTTS-80 and current matchers, finding that visible skin marks have almost no effect on matcher embeddings.

  • Problem

    Distinguishing monozygotic twins remains challenging and is an open problem in face recognition.

    Source: §8 · p. 24

  • Method

    The paper analyzes CTTS-80 matcher performance and compares embeddings for original versus skin-mark-erased images.

    Source: §5 · p. 9, §6 · p. 10

  • Results

    Positive and negative pair distances nearly totally overlapped for original and skin-mark-erased images, showing visible marks had almost no effect on embeddings.

    Source: §6 · p. 10

  • Takeaways & Limitations

    CTTS enables research comparing face matchers on MZ-twin recognition, with scope to focus on skin marks and asymmetry.

    Source: §8 · p. 24

    The natural facial differences between real MZ twins that separate positive and negative pairs remain uncharacterized in the face-recognition literature.

    Source: §7 · p. 21

Abstract

from arXiv · show

Past literature on face recognition for monozygotic (("identical") twins points to facial marks and mirror asymmetry as possible directions for improved accuracy of twins recognition. The Celeb Twins Test Set (CTTS) contains web-scraped image pairs for 80 sets of celebrity twins. It is the only twins test set with meta-data for twins with distinguishing skin marks and possible mirror asymmetry. CTTS is organized in the manner of face verification test sets such as LFW, CALFW, CPLFW, CFP-FP, and AgeDB-30. Current deep CNN matchers can achieve over 76% accuracy in classifying CTTS same-person / different-person image pairs. We show that current matchers do not make use of skin marks, or asymmetry, and discuss reasons for this. Finally, we discuss the feasibility of using generative AI tools such as Grok, ChatGPT and Gemini to create images of imagined monozygotic twins as a means to increase representation of twins in face recognition training sets.

1. Introduction

The introduction frames monozygotic twin recognition as a major challenge and identifies limited, insufficiently annotated twin datasets as a barrier to research. It presents CTTS-80 as a larger test set with metadata on distinguishing skin marks and possible mirror asymmetry.

1. Introduction

  • With a non-twin-calibrated 0.0001 FMR threshold, most algorithms had 0.98–0.99 FMR for accepting one twin as the other; the lowest was 0.475.The report’s threshold was selected to give a 0.0001 (1-in-10,000) false match rate for non-twins.Source: §1 · p. 1
  • ND-Twins draws from ND Twins Days, and neither it nor the larger source set records distinguishing marks or mirror-twin status; skin-mark datasets exist but are not twin-focused.The paper identifies limited experimental datasets for both training and testing as a barrier to twin-recognition research.Source: §1 · p. 1
  • CTTS-80 contains 21,120 pairs across 80 celebrity-twin sets—over 3.5 times the 6,000 pairs in listed benchmarks—with distinguishing marks in over half and 7 mirror-twin sets.The listed comparisons include ND-Twins, LFW, CALFW, CPLFW, CFP-FP, AgeDB-30, Hadrian, and Eclipse.Source: §1 · p. 2
  • The paper evaluates matcher accuracy on CTTS, tests whether matchers use skin marks and mirror asymmetry, and examines generative AI for creating imagined-twin training images.The planned analyses are described as Sections 5–7.Source: §1 · p. 2

2. MZ Twins: Biological Background

Monozygotic twins are not literally identical in facial appearance: differences may arise from skin marks, variable asymmetry, and aging. Mirror-twin status does not guarantee a useful facial asymmetry, and skin marks are useful for distinguishing MZ twins generally.

2. MZ Twins: Biological Background

  • About 1 in 4 MZ twins are mirror twins, whose reversed features vary by pair and may not create distinguishing facial asymmetry.Mirror twins generally result from embryo splitting around 7 to 10 days after conception, compared with around four days for most MZ twins; no DNA test identifies mirror-twin status.Source: §2 · p. 2
  • Mirror twins generally do not have skin marks in mirrored locations, while moles and freckles can distinguish MZ twins generally.Birth marks may or may not appear in mirrored locations; non-mirror MZ twins generally do not have skin marks in the same location.Source: §2 · p. 2
  • Opposite-handedness identifies some mirror twins, but only an estimated 50% to 75% are opposite-handed.The authors use known opposite-handedness to categorize some CTTS twins as mirror twins, while some uncategorized pairs may also be mirror twins.Source: §2 · p. 2
  • Age-related DNA mutations can distinguish MZ twins through sequencing, but there is no reason to believe these mutations change facial appearance.Facial aging may differ due to environmental conditions, health habits, or other factors.Source: §2 · p. 2
  • MZ twins generally have small facial differences, and differences tend to increase with age.The cited NIST study found lower average facial similarity between MZ twins with increased age.Source: §2 · p. 2

3. Human Ability to Distinguish Twins

Research on human twin recognition suggests facial marks can help distinguish monozygotic twins, while performance varies across people and may improve with training. Results also depend on viewing conditions and the methods used to compare humans with algorithms.

3. Human Ability to Distinguish Twins

  • 93% mean human accuracy (78%–100% range) was reported for judging twin image pairs, with marks cited more often on correctly classified pairs.In the study, “moles / scars / freckles” were selected for over 40% of correctly classified pairs and 15% of incorrectly classified pairs; eyes, nose, and lips showed the reverse pattern.Source: §3 · p. 3
  • At 30 ms, twins and close friends were no more accurate at identifying one twin than the other, while a DCNN exceeded nearly all tested human participants across conditions [39].The authors speculate that 30 ms may be too brief to use local features or mirror asymmetry. Parde et al. tested students, and the matcher may not represent current competitive ResNet-based matchers.Source: §3 · p. 3
  • Facial-mark matching yielded equal error rates of about 24%–35% with manual annotations and about 27% with automatic annotations.The approach used high-resolution images and bipartite graph matching to compare sets of broadly defined marks, unlike current deep CNN matchers.Source: §3 · p. 3
  • Face-recognition scores correlated more strongly for monozygotic twin pairs (0.70) than for same-gender dizygotic pairs (0.29), suggesting ability is strongly influenced by genetics.The authors suggest comparisons between algorithm and human accuracy should use best-performing humans rather than group averages.Source: §3 · p. 3
  • Training improved twin discrimination: after reaching 90% correct at labeling images, subjects rated same-person pairs as more similar and twin pairs as more different.A second experiment with a fixed amount of training produced a similar result; the authors speculate subjects learned local features distinguishing those twins.Source: §3 · p. 3
  • The literature review identifies facial marks as useful, finds no explicit reports of asymmetry-feature use, and concludes that ability varies across people and can improve with training.The training claim is presented alongside evidence that face-recognition ability is influenced by genetics.Source: §3 · p. 3

4. Automated Face Recognition of MZ Twins

Research on automated recognition of monozygotic twins has explored facial marks, asymmetry, age-related features, and deep matchers, but large-scale evaluation still finds twins difficult to distinguish. The literature also lacks a standard test set, limiting comparisons across studies.

4. Automated Face Recognition of MZ Twins

  • Asymmetry features may help distinguish twins, but the limited work emphasizing them assumes a controlled frontal pose.Le et al. also used a two-step approach: classify the twin set, then distinguish the individuals using Gabor features from specific face regions.Source: §4 · pp. 4–5
  • Mahalingam and Ricanek found twin discriminability increased with age, while CASIA accuracy was lower, possibly because some subjects were children.Le et al. connected region features to the possibility that MZ twins acquire unique facial features as they age.Source: §4 · p. 4
  • FGFV’s best classifier and feature combination exceeded 72% positive/negative pair accuracy, but its pair categories perfectly tracked image source, risking source-based classification [11].FGFV contains 455 positive pairs from LFW and 455 negative pairs of twins cropped from the same web image.Source: §4 · p. 4
  • Sami et al. [42] used a separate facial-similarity metric to define look-alikes and concluded that such pairs were rare in the populations studied.Their study used the WV Twins Days dataset and reported twin discrimination results across multiple deep CNN matchers.Source: §4 · p. 4
  • 61% accuracy on twins was unchanged by fine-tuning, while accuracy was 77% on positive pairs and 45% on negative pairs, in Hsu and Tsai.Their evaluation used 182 images of 54 twin sets and fine-tuned on 31 twin pairs.Source: §4 · pp. 4–5
  • At a 1-in-10,000 FMR threshold, the largest evaluation found high false matches and that algorithms could not distinguish identical and fraternal same-sex twins as different people.The report also notes that facial landmarks can be distinctive and useful, but are few and may not be observable.Source: §4 · p. 5
  • Facial marks appear useful for distinguishing twins, and were relatively more important for twin than non-twin accuracy when manually annotated.Brahmbhatt et al. reported 96% accuracy using 319 images from 74 twin sets, selecting each person’s best image.Source: §4 · pp. 4–5
  • The ND Twins Days dataset is the most widely used MZ-twin resource, but it is over 15 years old and too small for effective deep CNN training; the field also lacks a standard verification test set.Other work explored landmark-based recognition and feature-, score-, or decision-level fusion.Source: §4 · pp. 4–5

5. The Celeb Twins Test Set

The Celeb Twins Test Set (CTTS) is a cross-validated face-verification benchmark of celebrity monozygotic twins, with metadata on distinguishing skin marks and possible mirror twins. Current matchers perform less well on CTTS than on the traditional five benchmarks, with substantial variation across twin sets.

Dataset construction

  • CTTS includes 80 twin sets: 43 have potentially distinguishing skin marks, 7 have opposite handedness indicating mirror twins, and each face image is normalized to 112×112 pixels.Image selection required at least 12 images per twin, faces about 112×112 pixels or larger, and high-confidence identity labels, favoring celebrities with greater online presence.Source: §5 · p. 5
  • With 12 images per individual, each twin set contributes 132 positive and 132 randomly selected negative pairs; the 10 folds contain 21,120 pairs total, over 3.5 times the cited comparison sets.Each fold contains 264 pairs for each of eight twin sets.Source: §5 · p. 5

Evaluation protocol

  • The reported measure is average accuracy over ten folds, using a threshold selected on nine folds to classify pairs in the held-out fold.Unlike the NIST approach, which selects a threshold for 1-in-10,000 false-match rate on non-twin pairs, CTTS selects it using other twin pairs.Source: §5 · pp. 5, 9
  • CTTS folds are twin-set-disjoint, so a set's image pairs do not contribute to the threshold used to score that set, reducing optimistic bias.Source: §5 · p. 9

Results

  • CTTS-80 and ND-Twins accuracies are lower than those on Trad 5, Hadrian, and Eclipse; twin-set results have higher standard deviations, limiting firm comparisons across matchers and training sets.Accuracy is more stable across matchers and training sets for CTTS than for ND-Twins, likely because CTTS contains more image pairs.Source: §5 · p. 9, Table 1 · p. 9

Limitations

  • Some established celebrity twins may appear in web-scraped matcher training sets, but their overlap with CTTS is likely too small a fraction of training data to significantly affect accuracy.Source: §5 · p. 9

Results

  • Across the 80 twin sets, 7 had accuracy in (50%, 60%), 19 in [60%, 70%), 26 in [70%, 80%), 11 in [80%, 90%), and 17 in [90%, 100%).Accuracy reflects both separation of positive and negative distance distributions and whether the threshold selected from the other nine folds generalizes to that twin set.Source: §5 · p. 10
  • Well-separated pair-distance distributions are illustrated with mostly frontal, minimally occluded faces, while poorly separated examples show occlusion, pose variation, or atypical expressions; this suggests image acquisition may improve separation.Source: §5 · p. 10

6. Do Current Matchers Encode Skin Marks?

Skin-mark-erasure experiments show that visible marks have little or no effect on matcher embeddings, and disabling horizontal-flip augmentation does not significantly improve overall CTTS accuracy. The authors suggest training-set composition and flip-based processing may help explain these results.

Skin-mark experiments

  • Over half of CTTS twin sets have distinguishing skin marks, but pose, cosmetics, resolution, or blur can obscure them, including at 112x112 pixels.The passage also notes examples where marks remain visible at 112x112 pixels.Source: §6 · p. 10, Figure 9 · p. 14
  • Erasing Tamera Mowry’s skin mark barely changed the pair-distance distributions, and original-versus-erased image distances clustered near zero.The original and erased-image positive and negative distance distributions were nearly totally overlapped, except for the 12 negative pairs comparing an image with and without the mark.Source: §6 · p. 10, Figure 10 · p. 15
  • The same negligible skin-mark effect appeared for the Hassan and Kaczynski twins, whose original and erased-image distance distributions were essentially the same.For each set, differences between original and erased versions of the same image were zero.Source: §6 · p. 10, Figure 12 · p. 17, Figure 14 · p. 19

Why marks may be ignored

  • The authors suggest that few training identities are likely monozygotic twins and horizontal-flip augmentation may discourage encoding mark location or asymmetry.Averaging embeddings from original and flipped images at test time may also make gains from encoding mark location harder to detect.Source: §6 · pp. 10, 20

Asymmetry-aware matcher

  • 78.58% ±3.7 CTTS accuracy for the asymmetry-aware matcher was not statistically significantly different from the original model’s accuracy.For Mowry twins, distribution overlap increased to 11 pairs from 6 with the original matcher, so the skin mark did not reduce overlap.Source: §6 · p. 20
  • Across seven known mirror-twin sets, asymmetry-aware overlap increased for three, stayed the same for one, and decreased for three; the largest changes were decreases for two sets.The authors suggest some mirror twins may have useful facial asymmetry, but note that this remains poorly understood.Source: §6 · p. 20

7. Twins Training Images from Generative AI?

Initial tests found that Grok and ChatGPT did not generate identity-preserving images and distance distributions that distinguish same-person from different-person pairs for imagined monozygotic twins, while Gemini declined the task. The section concludes that generating useful twins training images requires knowledge of real twins’ natural facial differences that is not yet established.

Training data scale

  • A 20% rule of thumb implies needing images for 60,000–200,000 twin sets to train a twins-aware matcher, an infeasible target using real monozygotic twins.The estimate applies the 20% training-identity share suggested by work on sunglasses and facial-hair conditions to WebFace 12M’s 600,000 and WebFace 42M’s 2 million identities.Source: §7 · p. 20

Experimental approach

  • The authors tested imagined twins from Grok, ChatGPT, and Gemini using the same prompt for fictional 35-year-old white female monozygotic twins, requesting variation in hairstyle, expression, and age.The experiment asked whether generated images could augment other training data and yield pair-distance distributions resembling those for real twins.Source: §7 · p. 20

Grok-imagined twins

  • Grok’s imagined twins had wholly overlapping same-person and different-person distance distributions, implausible faces, and images that failed to preserve individual identity.Their same-person distances began at or above the range for same-person pairs of real twins, and the imagined twins appeared literally identical rather than naturally different.Source: §7 · pp. 20–21, Figure 16 · p. 22

ChatGPT-imagined twins

  • ChatGPT’s imagined twins had wholly overlapping same-person and different-person distance distributions, despite plausible-looking images and largely plausible same-person distances.The images varied in hairstyle and expression but showed less pose and age variation than Grok’s; the findings suggest both tools fail to represent how monozygotic twins differ naturally.Source: §7 · p. 21, Figure 17 · p. 23

Gemini-imagined twins

  • Gemini repeatedly said it could not produce the requested images of imagined monozygotic twins.Its responses cited generation constraints and recommended using a labeled dataset of real human twins for experiments on pair similarity.Source: §7 · p. 21

Open problem

  • The study identifies missing knowledge of real twins’ typical facial differences as a fundamental problem for generating useful imagined-twin training data.The literature does not yet establish which facial features vary, or by how much, in ways that let matchers separate positive and negative pairs; existing results may also lead generative models to treat twins as literally identical.Source: §7 · p. 21

8. Conclusions and Discussion

The discussion frames MZ-twin recognition as an open challenge and CTTS as a resource for studying it. Facial cues, training-set composition, and synthetic-twin generation remain possible but unresolved routes to improvement.

Conclusions and Discussion

  • CTTS accuracy is much lower than on benchmarks targeting age, pose, hairstyles, or illumination, and it enables matcher comparisons focused on skin marks and asymmetry.The paper identifies twin discrimination as an open face-recognition problem.Source: §8 · p. 24
  • All 12 matcher versions in Table 1 exceed 70% accuracy, and their embeddings capture some difference between MZ twins.If twin appearances were literally identical, the expected accuracy would be 50%.Source: §8 · p. 24
  • Although some CTTS-80 twins are distinguishable by skin marks at 112x112 resolution, current matchers do not use skin marks or asymmetry.Training matchers to use these cues likely requires a significant fraction of such twins in training data and could improve accuracy.Source: §8 · p. 24
  • Adding MZ-twin images to training sets could improve accuracy on twins and non-twins, but the necessary training-set composition, size, and feasibility of acquiring enough images remain open questions.Source: §8 · p. 24
  • Current generative AI tools cannot create useful imagined-twin images, though future work could model natural twin variation or generate a realistic imagined twin relative to a real person.Source: §8 · p. 24
Loading 2609.01141v1…