PaperlayerView exact revision on arXiv

Source-linked AI summary

Detection of false positives by human reviewers depends on the face recognition algorithm and its security configuration

Esha Edakoth, John J. Howard, Isabelle M. Shuggi, Yevgeniy B. Sirotin, Robert Wood

arXiv:2610.33045v1cs.CVcs.HC

TL;DR

Human reviewers’ ability to catch commercial face-recognition false positives may depend on the algorithm and security setting. The study tests this using face pairs from four comparable commercial algorithms, finding that reviewers can reduce false positives but become less effective as security tightens. Reviewer performance also varies across algorithms, supporting consideration of human-review compatibility in algorithm selection.

  • Problem

    Recent false positives have raised questions about human reviewers’ effectiveness in face-recognition workflows, including across commercial algorithms and operational security settings.

    Source: INTRODUCTION · p. 2

  • Method

    The study sampled false-positive face pairs from four comparable commercial algorithms at operationally relevant thresholds and measured reviewer detection performance.

    Source: Face Pair Selection Methodology · p. 5

  • Results

    Human reviewers reduced algorithmic false positives but became less effective as security settings tightened; at the highest security setting, untrained reviewers detected ~80% of false positives.

    Source: DISCUSSION · p. 18

  • Takeaways & Limitations

    Algorithm selection and human-oversight strategies should account for differences in reviewer compatibility across algorithms and security configurations.

    Source: DISCUSSION · pp. 18, 21

    The task reflects verification rather than identification, which requires review of a candidate list.

    Source: DISCUSSION · p. 20

Abstract

from arXiv · show

Integration of artificial intelligence (AI) technology into identity workflows, such as automated face recognition (FR), has enhanced operational capabilities, with FR algorithms achieving super-human scale and accuracy. However, these gains can make human oversight of FR more difficult, raising questions about the effectiveness of these human-AI teams. We measured human reviewers' ability to detect false positive errors made by commercial FR algorithms across varying security configurations. Using a novel dataset, the Difficult Facial Image Comparison Test (DFICT), derived from false positives made by four comparable commercial face recognition algorithms, we measured the ability of 801human reviewers to detect algorithmic false positives across three security configurations, representing operationally relevant biometric thresholds. Our findings show that human reviewers can reduce algorithmic false positives, but performance declines at stricter security thresholds. Further, reviewer performance varied by algorithm indicating that some algorithms are more compatible with human oversight. We suggest that biometric systems should be designed to account for human reviewers' variable effectiveness, particularly in high-security applications. Our findings should be considered during algorithm selection and inform human oversight strategies to minimize AI-related risks in operational FR deployments.

INTRODUCTION

Deep-learning face recognition has improved automated performance, but recent false positives have renewed questions about human oversight. The introduction situates those questions in human face-comparison abilities and the security-dependent operation of commercial systems.

INTRODUCTION

  • Recent false positives from face-recognition systems have raised questions about the effectiveness of human-AI teams in face-recognition workflows.Deep learning has improved automated FR performance, while humans are typically assigned supervisory review without standardized facial-comparison training.Source: INTRODUCTION · p. 2

Human Face Recognition

  • Humans can compare unfamiliar faces, but performance varies; training that directs attention to specific facial features can improve matching accuracy.Many biometric workflows use reviewers without formal facial-comparison training or time for comprehensive morphological comparison.Source: Human Face Recognition · p. 2

Face Recognition Technologies

  • FR systems compare numerical similarity scores against thresholds selected for security or facilitation, with stricter settings trading false-positive and false-negative rates.Modern systems can exceed human face-recognition performance, but results are affected by image quality and demographics.Source: Face Recognition Technologies · pp. 2–3
  • Verification compares one probe with one reference, whereas identification searches a reference database and requires higher thresholds for the same false-positive risk.Either process can use different levels of automation and human oversight.Source: Face Recognition Technologies · p. 3

Human-AI Teaming in Face Recognition Workflows

Human-AI face-recognition teams must account for teammate capabilities and how their tasks depend on one another. Because human review follows an algorithmic decision, evaluating reviewer capability relative to the algorithm can guide team design.

Human-AI Teaming in Face Recognition Workflows

  • Human reviewers may add cues algorithms ignore, but they can also lower team performance by overriding accurate decisions or missing errors, especially with degraded images.Different algorithms can also change which face pairs reviewers encounter and thereby affect reviewer performance.Source: Human-AI Teaming in Face Recognition… · p. 4
  • Human-AI workflow subsystems can be assigned different automation levels according to use-case requirements, producing teams with different task interdependence.The framework covers data capture, signal processing, storage, comparison, and decision subsystems.Source: Human-AI Teaming in Face Recognition… · p. 4
  • Sequential interdependence means a reviewer waits for the algorithm’s comparison and decision before reviewing the match.This contrasts with pooled interdependence, where human and algorithm decisions are made independently and fused.Source: Human-AI Teaming in Face Recognition… · p. 4
  • Task allocation assigns human and AI roles according to their capabilities, and assessing human capability relative to the AI can guide team adjustments.Adjustments may include changes in task allocation, algorithm selection, or training.Source: Human-AI Teaming in Face Recognition… · p. 4

Contributions

The study tests whether human reviewers detect commercial FR false positives, how security settings affect detection, and whether algorithms differ in suitability for human-AI teaming. It introduces a sampling method and reports that reviewer performance depends on both security setting and algorithm.

Contributions

  • The study asks whether reviewers can detect current commercial FR false positives, how security settings affect detection, and whether algorithms differ in teaming suitability..Source: Contributions · p. 5
  • The study introduces S-FPRI, a method for selecting face pairs to assess human-AI team performance at operationally relevant settings.The evaluation uses face pairs selected from four comparable commercial algorithms and human volunteers who had not seen the algorithmic decisions.Source: Contributions · p. 5
  • Reviewers became less likely to catch FR false positives at stricter security settings, while performance also varied by algorithm.The authors identify some algorithms as potentially more suitable for human-AI teaming.Source: Contributions · p. 5

Face Pair Selection Methodology

The methodology selects false-positive face pairs from four comparable commercial algorithms using security-related false-match-rate intervals. It samples challenging non-mated pairs within each interval and adds mated pairs to the reviewer surveys.

Face Pair Selection Methodology

  • S-FPRI sets four algorithm-specific thresholds at FMRs of 1e-3, 1e-4, 1e-5, and 1e-6, enabling comparisons despite different similarity-score scales.The four algorithms had false non-match rates at or below 1% at each evaluated FMR setting.Source: Face Pair Selection Methodology · p. 5
  • The three highest-scoring non-mated pairs in each interval were selected to provide challenging comparisons, with intervals ordered from lower to higher security.Figure 1 marks the lower, medium, and higher FPRI samples on each algorithm’s similarity-score distribution; mated pairs were directly above the 1e-6 FMR setting.Source: Face Pair Selection Methodology · p. 6, Figure 1 · p. 7
  • Mated pairs were added to limit habituation to non-mated pairs, and each survey contained 12 questions for one algorithm.Across the four surveys, the selected material included 70 individuals and 39 unique face pairs.Source: Face Pair Selection Methodology · p. 7, Figure 2 · p. 8

Survey Design

Untrained volunteers reviewed face pairs from one face recognition algorithm in a short survey, rating each pair on a certainty scale. The survey included non-mated pairs from three false-positive-rate intervals and mated pairs.

Survey Design

  • Each untrained reviewer rated 12 pairs from one algorithm: three non-mated pairs from each FPRI and three mated pairs.Most volunteers completed the task in about 10 minutes, with no time limit.Source: Survey Design · p. 9
  • Figure 3 shows an example survey question using a non-mated face pair, whose subjects gave permission to share their images.Source: Figure 3 · p. 9
  • Pairs appeared one at a time in random order on a 7-point scale from certainty that the people differed to certainty that they were the same.Reviewers had to choose a response, with “I am not sure” available.Source: Survey Design · p. 9

Volunteer Demographics

The study retained 801 volunteers after excluding 13 of the 814 who consented. Volunteers self-reported age, sex, and race, and reviewer demographics are presented in Table 1.

Volunteer Demographics

Data Analysis

The analysis used signal detection theory to evaluate reviewer discrimination and response bias across decision thresholds. It also estimated threshold-independent sensitivity using the ROC curve.

Data Analysis

  • Signal detection theory measures sensitivity (d′) and criterion (c), assessing discrimination between mated and non-mated pairs and cognitive response bias, respectively.Source: Data Analysis · p. 10
  • At six reviewer decision thresholds listed in Table 2, the analysis calculated TPRθ and FPRθ from responses, then used them to calculate d′ and c.Responses above a threshold counted as “same,” and those below it as “different”; “I am not sure” counted as different.Source: Data Analysis · p. 11, Table 2 · p. 11
  • AUC provided a threshold-independent sensitivity measure, calculated for each FPRI under Gaussian signal and noise distribution assumptions.The calculation used the intercept and slope of the best-fit line between the z scores of TPRθ and FPRθ; d′ across thresholds was a supplementary AUC metric.Source: Data Analysis · pp. 11–12

Aggregated Across Algorithms

Reviewers’ ability to detect non-mated pairs declined as their similarity scores increased across the false positive rate intervals (FPRIs). They correctly classified most mated pairs, but showed a bias toward labeling pairs as different, with the highest FPRI producing the weakest performance.

Aggregated Across Algorithms

  • For non-mated pairs, responses shifted toward “same person” at medium and higher FPRIs, whereas most reviewers correctly labeled mated pairs “same person,” with varying certainty.Figure 4 encodes different-person responses as negative, same-person responses as positive, and uncertainty as zero; absolute response size indicates certainty.Source: Aggregated Across Algorithms · p. 12, Figure 4 · p. 12
  • FPR increased significantly from lower to medium to higher FPRIs, while reviewers’ TPR on mated pairs was 0.67 and their FPRs on non-mated pairs ranged from 0.085 to 0.21.The comparison used reviewer decision threshold θ = 0.5; “not sure” responses were classified as different.Source: Aggregated Across Algorithms · p. 13, Figure 5 · p. 13
  • Sensitivity and criterion were assessed across FPRIs at reviewer threshold θ = 0.5, with the reported sensitivity confidence intervals overlapping for the lower and medium intervals.Table 3 presents sensitivity and criterion by FPRI.Source: Aggregated Across Algorithms · pp. 13–14, Table 3 · p. 14
  • Reviewers performed worse on non-mated pairs from the higher FPRI than from the lower and medium FPRIs, while sensitivity confidence intervals overlapped for the lower and medium intervals.Az and sensitivity decreased as non-mated-pair similarity increased; the ROC curve shifted right at the higher FPRI.Source: Aggregated Across Algorithms · pp. 13–15, Figure 6 · p. 14

Disaggregated by Algorithm

Reviewer performance varied across the four algorithms, especially at the higher FPRI. Algorithm C’s high-FPRI pairs were the most difficult to distinguish from mated pairs, while performance was more similar across algorithms at lower and medium FPRIs.

Disaggregated by Algorithm

  • Reviewer TPR on mated pairs was comparable across algorithms, while FPR increased more for all algorithms from the medium to higher FPRI than from the lower to medium FPRI.The TPR comparison across algorithms had p = 0.076.Source: Disaggregated by Algorithm · p. 15, Figure 7 · p. 15
  • At the higher FPRI, reviewer FPR was significantly lower for algorithm A than B and C, and for D than B and C; no algorithm differences were found at the medium FPRI.At the lower FPRI, algorithm A’s FPR was significantly higher than algorithm C’s.Source: Disaggregated by Algorithm · pp. 15–16, Figure 7 · p. 15
  • At the higher FPRI, algorithm C had lower reviewer sensitivity than the other algorithms, and its ROC curve shifted markedly right relative to the others.Across algorithms, Az and sensitivity decreased from lower to higher FPRIs; algorithm C showed the largest Az and d’ decrease from medium to high.Source: Disaggregated by Algorithm · pp. 16–17, Figure 8 · p. 17
  • At the slightly strict reviewer threshold, sensitivity and criterion confidence intervals overlapped across algorithms for the lower and medium FPRIs, but not for algorithms C and D at the higher FPRI.Algorithm C had lower sensitivity and criterion than the other algorithms at the higher FPRI.Source: Disaggregated by Algorithm · p. 16, Table 4 · p. 16

DISCUSSION

Human review can reduce face-recognition false positives, but reviewer effectiveness declines at stricter security settings and varies across algorithms. The discussion argues that human-review compatibility should inform biometric algorithm selection and evaluation.

DISCUSSION

  • At the highest security setting, untrained reviewers detected ~80% of algorithmic false positives (d’ ~ 1).At FMR = 1e-5, detection rates dropped by more than half versus lower thresholds. Reviewer false-positive rates rose from under 10% at lower security FPRI (FMR setting between 1e-3 and 1e-4) to over 20% at higher security FPRI (FMR setting between 1e-5 and 1e-6).Source: DISCUSSION · pp. 18, 20
  • Reviewer false-positive rates varied nearly twofold across four commercial algorithms, despite comparable FMR and FNMR performance.The authors argue that compatibility with human review should be considered alongside algorithmic accuracy when selecting systems for manual-review workflows.Source: DISCUSSION · pp. 18, 21
  • Biometric accuracy standards focus on algorithms, while existing human-review guidance addresses procedures rather than total human-AI system performance.The authors argue that evaluation frameworks should incorporate human-AI team performance because reviewers and algorithms are sequentially interdependent.Source: DISCUSSION · pp. 18–19
  • The S-FPRI method selected uniformly high-quality image pairs by non-mated similarity scores to compare current commercial algorithms on algorithmic false positives.Across FPRIs, sensitivity was d’ 1.24 to 1.81; the high-FPRI value of d’ = 1.24 was below GFMT2 short-form values of d’ 1.64-1.75, indicating greater challenge for reviewers.Source: DISCUSSION · p. 19
  • Reviewers saw neither algorithm accuracy nor decisions, although operational reviewers often receive algorithm information that could reduce false-positive detection.The authors note that information type and amount can influence human decisions, while conversational AI may allow richer teammate communication.Source: DISCUSSION · p. 20
Loading 2610.33045v1…