PaperlayerView exact revision on arXiv

Source-linked AI summary

Review of the Fingerprint Liveness Detection (LivDet) competition series: 2009 to 2015

Luca Ghiani, David A. Yambay, Valerio Mura, Gian Luca Marcialis, Fabio Roli, Stephanie A. Schuckers

arXiv:1609.01648v1cs.CV

TL;DR

Fingerprint sensors can be attacked with artificial replicas, so liveness detection seeks to distinguish live from fake traits. This review tracks LivDet's evaluations of algorithms and systems across editions. Reported classification accuracy rose from about 70% in 2011 to 90% in 2015, while the authors note that explaining the benefits and limitations of current features remains an issue.

  • Problem

    Fingerprint sensors are vulnerable to spoof attacks using artificial replicas, making detection of live versus fake traits a relevant research problem.

    Source: §1 · p. 2

  • Method

    The paper reviews LivDet's competition history, evaluation approaches, and results for fingerprint liveness algorithms and systems.

    Source: §1 · p. 3, §6 · p. 41

  • Results

    Classification accuracy increased from about 70% in LivDet 2011 to 90% in LivDet 2015.

    Source: §6 · p. 42

  • Takeaways & Limitations

    The review reports strong performance improvement across competitions, including good performance on 500 and 1000 dpi devices and some robustness to unseen attacks.

    Source: §6 · p. 42

    Explaining the benefits and limitations of currently used features remains an unresolved issue, in the authors' view.

    Source: §6 · p. 42

Abstract

from arXiv · show

A spoof attack, a subset of presentation attacks, is the use of an artificial replica of a biometric in an attempt to circumvent a biometric sensor. Liveness detection, or presentation attack detection, distinguishes between live and fake biometric traits and is based on the principle that additional information can be garnered above and beyond the data procured by a standard authentication system to determine if a biometric measure is authentic. The goals for the Liveness Detection (LivDet) competitions are to compare software-based fingerprint liveness detection and artifact detection algorithms (Part 1), as well as fingerprint systems which incorporate liveness detection or artifact detection capabilities (Part 2), using a standardized testing protocol and large quantities of spoof and live tests. The competitions are open to all academic and industrial institutions which have a solution for either softwarebased or system-based fingerprint liveness detection. The LivDet competitions have been hosted in 2009, 2011, 2013 and 2015 and have shown themselves to provide a crucial look at the current state of the art in liveness detection schemes. There has been a noticeable increase in the number of participants in LivDet competitions as well as a noticeable decrease in error rates across competitions. Participants have grown from four to the most recent thirteen submissions for Fingerprint Part 1. Fingerprints Part 2 has held steady at two submissions each competition in 2011 and 2013 and only one for the 2015 edition. The continuous increase of competitors demonstrates a growing interest in the topic.

1. Introduction

Fingerprint sensors are vulnerable to artificial replicas, motivating liveness detection that uses additional information to distinguish live from fake traits. This paper reviews how LivDet competitions have evolved and the trends in their results.

1. Introduction

  • Artificial fingerprint replicas can circumvent fingerprint sensors, creating a need for countermeasures.Fingerprints are widespread, but scanners have been shown vulnerable to attacks using artificial replicas.Source: §1 · p. 2
  • Liveness detection uses information beyond standard verification data to assess whether a fingerprint image is authentic.Software approaches process the collected image, while hardware approaches use additional sensors.Source: §1 · p. 2
  • LivDet compares software-based algorithms and integrated fingerprint systems using standardized testing protocols and spoof and live tests.Part 1 evaluates algorithms; Part 2 evaluates systems incorporating liveness or artifact detection.Source: §1 · p. 3

2. Background

The background describes fingerprint spoofing, approaches to liveness detection, and the rationale for LivDet as a public benchmark. The competition series evaluates algorithms and systems across successive editions.

Background

  • Fingerprint countermeasures include hardware sensing and software analysis, with liveness systems classifying presented fingerprints as live or fake.Software approaches analyze image features such as ridge and valley patterns.Source: §2 · pp. 3–4
  • Artificial fingers can be made cooperatively by casting a negative impression or non-cooperatively from a digitized latent fingerprint.The latter route can use a fingerprint image etched onto a printed circuit board to create a spoof cast.Source: §2 · pp. 4–5
  • LivDet 2009 through 2015 established a recurring benchmark for liveness detection, with later editions adding integrated-system evaluation.The editions are identified as LivDet 2009, 2011 [2], 2013 [3], and 2015 [4].Source: §2 · pp. 5–6
  • The authors motivate LivDet by noting that earlier spoof-detection studies often used small, nonpublic datasets and nonuniform protocols.They identify these concerns in.Source: §2 · p. 5
  • LivDet was established to provide public datasets and third-party testing with a sequestered test set, addressing earlier studies' small, nonpublic datasets and inconsistent protocols.Its evaluations cover software algorithms and integrated systems.Source: §2 · pp. 5–6
  • A quick Google Scholar search for works citing publications related to the first three LivDet competitions found 65 results for 2009, 41 for 2011, and 26 for 2013.Source: §2 · p. 7

3. Methods and Datasets

LivDet uses separate protocols for algorithm and system evaluations, with datasets varying across editions and devices. The methods also examine image quality, evaluation metrics, and challenges involving spoof-generation methods and sensor resolution.

3.2. Part 2: Systems Submissions

  • Part 1 evaluates algorithms on pre-generated datasets, while Part 2 tests submitted systems through systematic data collection.Systems output an image and a liveness score from 0 to 100, with 50 as the live/spoof threshold; nonresponses are treated differently for live and spoof attempts.Source: §3 · p. 7, §3.2 · pp. 11–12

3.1. Part 1: Algorithm Datasets

  • Eighteen datasets were completed across the four competitions, including 15 fingerprint and three iris liveness datasets.Part 1 datasets comprise live and spoof images from three to four devices; materials and devices varied across editions.Source: §3.1 · p. 7

3.3. Image Quality

  • The image-quality analysis applies NFIQ to assess all competition fingerprints and examine how removing lower-quality images affects liveness protocols.NFIQ assigns image quality on a scale from 1 (highest) to 5 (lowest).Source: §3.3 · pp. 12–13

3.4. Performance Evaluation

  • Performance is reported using Ferrlive, Ferrfake, average classification error, equal error rate, and accuracy.Ferrlive and Ferrfake are the rates of misclassified live and fake fingerprints, respectively.Source: §3.4 · p. 13

3.5. Specific challenges

  • Recent editions introduced challenges involving latent-derived spoof images, previously unseen spoof materials, and a 1000 dpi sensor.Two 2013 datasets used latent fingerprints; the 2015 test set included two spoof materials absent from training.Source: §3.5 · p. 13

3.5.1. LivDet 2013 Consensual vs. Semi-Consensual

  • The 2013 semi-consensual method generated spoofs from digitized latent fingerprints by binarizing, thinning, printing, and casting the fingerprint image.Gelatin or silicone was applied to the printed image and separated after solidification.Source: §3.5.1 · p. 14
  • Consensual spoofs closely resemble live fingers, whereas latent-derived spoofs lose detail and are easier to distinguish.Obtaining a consensual cast may be harder than obtaining a latent fingerprint.Source: §3.5.1 · p. 14

230 3.5.2. LivDet 2015 Hidden Materials and 500 vs 1000 dpi

The 2015 tests included spoof materials absent from training and a 1000 dpi sensor. Results did not support the hypothesis that doubling resolution would improve performance.

3.5.2. LivDet 2015 Hidden Materials and 500 vs 1000 dpi

  • The 2015 tests included spoof materials absent from training to assess algorithm robustness to unknown attacks.The materials included liquid Ecoflex and RTV for three datasets, and OOMOO and gelatin for Crossmatch.Source: 2303.5.2. LivDet 2015 Hidden Materials… · pp. 14–15
  • The Biometrika HiScan-PRO captured images at 1000 dpi, compared with approximately 500 dpi for most prior competition datasets.The authors hypothesized that doubling resolution could benefit feature extraction and performance, but results did not confirm it.Source: 2303.5.2. LivDet 2015 Hidden Materials… · p. 15

4. Examination of Results

This section examines reported results across LivDet system and algorithm competitions, including error rates and the effects of dataset conditions, spoof quality, and score-level fusion choices.

4.1. Trends of Competitors and Results for Fingerprint Part 1: Algorithms

  • Part 1 submissions rose from 4 in 2009 and 3 in 2011 to 11 in both 2013 and 2015, while errors declined most clearly in 2013 and 2015.LivDet 2013 had nine participants and 2015 had ten; error rates changed little from 2009 to 2011, then fell in each subsequent competition.Source: §4.1 · p. 15, Table 5 · p. 16
  • Mean ACE results indicate that LivDet 2011 was harder with high-quality spoofs, while 2013 Biometrika and Italdata spoofs made from latent fingerprints were easier to detect.For Biometrika and Italdata, the 2013 average error was 10%, although standard deviation was similar to 2011; the authors caution that different participants and algorithms limit this comparison.Source: §4.1 · p. 18
  • LivDet 2015 mean error rates were 11%–16% against high-quality spoofs, while higher Biometrika resolution did not ensure the best performance and small Digital Persona images generally reduced accuracy.The reported error rates suggest slow but steady advancement in liveness and artifact detection.Source: §4.1 · pp. 22–23
  • With FerrLive = 1%, the best algorithm’s Ferrfake ranged from 2.66% to 19.10% across materials, and no analyzed algorithm generalized reliably to unseen spoof materials.Performance dropped on never-seen materials, with the size of the drop varying unpredictably by material.Source: §4.1 · p. 24, Table 10 · p. 25, Table 11 · p. 25
  • The 2015 algorithm score plots showed low correlation in many doubtful cases, suggesting that algorithms may detect complementary spoofs.The authors infer complementarity from the spread of algorithm scores, rather than aligned diagonal patterns.Source: §4.1 · p. 29
  • The study fused three of four better algorithms using summed probabilities, multiplied probabilities, or majority vote; the Oracle response marks a pattern correct if any classifier does.The Oracle indicates maximum potential accuracy, which improves as classifiers become more complementary.Source: §4.1 · p. 29
  • Score-level fusion improved DET results especially with the sum rule, but performance declined on Digital Persona and depends on careful classifier and combination-method selection.Averaging can let a strong correct classifier counterbalance two weaker, incorrect ones.Source: §4.1 · p. 34

4.2. Trends of Competitors and Results for Fingerprint Part 2: Systems

  • Part 2 submissions remained limited: two systems entered in both 2011 and 2013, followed by one in 2015.The testing required a full fingerprint recognition device with integrated liveness detection.Source: §4.2 · p. 34
  • System results improved sharply from 2011 to 2013; 2015 FerrLive remained below 2011 but above 2013, and FerrFake fell below 2% excluding playdoh errors.The best 2011 results were worse than the worst 2013 results; most 2015 FerrFake error came from playdoh.Source: §4.2 · pp. 34, 36
  • In 2013, Anonymous1 achieved 0.0% error, identifying every spoof correctly; its FerrLive was 1.4% and FerrFake 0.0%.Dermalog’s 2013 FerrLive was 11.8% and FerrFake 0.6%.Source: §4.2 · p. 35
  • In 2015, Anonymous2 recorded FerrLive 14.95% and FerrFake 6.29%; FerrFake was 11.09% for known recipes versus 1% for unknown recipes.This reversed the prior pattern in which known spoof types typically performed better than unknown types.Source: §4.2 · p. 35
  • Pink and lime green playdoh spoofs were accepted as live for almost 100% of collected images, driving 28% total playdoh error; brown and black caused a 6.9% Fake Non-Response Rate.Yellow and white playdoh spoofs were detected with high accuracy; the non-response rate was not counted as spoof detection error.Source: §4.2 · pp. 35–36

5. Quality analysis: A Lesson Learned from the LivDet Experience

The LivDet experience shows that spoof fabrication is difficult to control and that image quality is associated with detection error rates. The competition therefore uses carefully selected spoof materials and evaluates performance across quality levels.

5. Quality analysis: A Lesson Learned from the LivDet Experience

  • After ten years and more than 50,000 spoofs, the researchers assessed fake-fingerprint visual quality by mold and cast material, alongside fabrication ease.Visual quality was assessed using microscope analysis and the quality of images acquired by capture devices.Source: §5 · pp. 36–37
  • Some materials fail to reproduce ridge and valley flow without bubbles or altered ridge edges, and materials with these problems can also have low subjective quality.Many materials are difficult to manage when fabricating fake fingers.Source: §5 · pp. 37, 40
  • LivDet selects spoof materials by trading off fabrication quality and ease, so participants face spoofs made with the best and potentially most dangerous materials.The materials selection draws on the criteria assessed in Tables 16 and 17.Source: Table 16 · p. 38, Table 17 · p. 39, §5 · p. 40
  • Across four LivDet editions, FerrFake and FerrLive rates at different quality levels support a correlation between image quality and decreasing error rates.For Dermalog, adding lower-quality spoof images generally decreased FerrFake, while rates became less consistent when the worst-quality images were included.Source: §5 · pp. 40–41
  • Figure 20 presents error rates by quality level; in the Dermalog example, rates were less consistent when the worst-quality spoof images were included.The authors attribute this likely to variability in low-quality spoofs.Source: §5 · pp. 40–41, Figure 20 · p. 41

6. Conclusions

The LivDet competitions provide an independent assessment of fingerprint liveness algorithms and systems, and their results show improved classification accuracy from 2011 to 2015. The 2015 results also indicate strong performance across capture resolutions and some robustness to previously unseen spoof materials, while leaving substantial work before deployment.

6. Conclusions

  • LivDet was established to provide research centres and companies a fair, independent assessment of anti-spoofing algorithms and systems.The authors report increasing interest and recognition for the competition and the large amount of data it made publicly available.Source: §6 · p. 41
  • Classification accuracy rose from about 70% in LivDet 2011 to 90% in LivDet 2015.The 2015 result came under difficult consensual fingerprint-replication conditions and was comparable with the easier latent-mark replication task in the first two LivDet 2013 datasets.Source: §6 · p. 42
  • Submitted algorithms performed very well on both 500 and 1000 dpi devices, and some showed good robustness to never-before-seen spoofing attacks.The 2015 edition included a 1000 dpi capture device and evaluation against unknown spoof materials.Source: §6 · p. 42
  • The conclusion proposes that a future LivDet edition might explicitly include a section on algorithm and system fusion.The authors say liveness detection could further benefit from combining multiple features and approaches.Source: §6 · p. 42
  • Despite the reported progress, most submissions were not yet accurate enough for fingerprint-verification systems in many real applications.The authors say current error rates remain too high for those applications.Source: §6 · p. 42
  • The authors identify understanding fingerprint-replication physics and feature extraction as a future challenge for finding the characteristics most useful for classification.They note that concrete results on this challenging task may take many years.Source: §6 · p. 42
Loading 1609.01648v1…