PaperlayerView exact revision on arXiv

Source-linked AI summary

Labeled pupils in the wild: A dataset for studying pupil detection in unconstrained environments

Marc Tonsen, Xucong Zhang, Yusuke Sugano, Andreas Bulling

arXiv:1511.05768v1cs.CV

TL;DR

Pupil detection in head-mounted eye images is difficult under everyday conditions, and available datasets have limitations in size, realism, or image quality. LPW presents a varied, annotated video dataset and evaluates current detection algorithms. The evaluation finds that none performs well across all dataset conditions.

  • Problem

    Head-mounted eye images are affected by changing light, reflections, occlusions, and shadows, while available datasets have limitations in size, realism, or image quality.

    Source: §1 · p. 1

  • Method

    LPW provides manually annotated high-quality eye-region videos recorded from varied participants and everyday locations, and benchmarks pupil detection algorithms.

    Source: §1 · p. 1, Ground truth annotation · p. 2

  • Results

    None of the examined algorithms performed well on all dataset parts, and detection accuracy in at least half of cases was insufficient for good eye tracking.

    Source: §4 · p. 4

  • Takeaways & Limitations

    Evaluations highlight lighting, image resolution, and vision aids as key challenges for head-mounted pupil detection.

    Source: §5 · p. 4

    Participants' viewing behavior was controlled by a gaze target and was therefore not natural.

    Source: §4 · p. 4

Abstract

from arXiv · show

We present labelled pupils in the wild (LPW), a novel dataset of 66 high-quality, high-speed eye region videos for the development and evaluation of pupil detection algorithms. The videos in our dataset were recorded from 22 participants in everyday locations at about 95 FPS using a state-of-the-art dark-pupil head-mounted eye tracker. They cover people with different ethnicities, a diverse set of everyday indoor and outdoor illumination environments, as well as natural gaze direction distributions. The dataset also includes participants wearing glasses, contact lenses, as well as make-up. We benchmark five state-of-the-art pupil detection algorithms on our dataset with respect to robustness and accuracy. We further study the influence of image resolution, vision aids, as well as recording location (indoor, outdoor) on pupil detection performance. Our evaluations provide valuable insights into the general pupil detection problem and allow us to identify key challenges for robust pupil detection on head-mounted eye trackers.

1 Introduction

Pupil detection in head-mounted eye images remains difficult under everyday imaging conditions, while existing datasets leave gaps in size, realism, or image quality. LPW addresses these limitations with high-quality, annotated videos recorded across diverse participants and indoor and outdoor conditions.

1 Introduction

  • Ambient light, corneal reflections, pupil occlusions, and shadows make robust pupil detection challenging in head-mounted eye images.The cited examples include strong shade, eyelid occlusion, reflections on glasses, and make-up.Source: §1 · p. 1, Figure 1 · p. 1
  • Existing head-mounted pupil datasets are limited in size, confined to controlled settings, lack fully outdoor recordings, or contain low-quality images.The earlier Swirski dataset contains 600 close-up images from two participants recorded in a laboratory; a later dataset includes driving and shopping but not fully outdoor settings.Source: §1 · p. 1
  • LPW introduces 66 high-quality eye-region videos from 22 participants, covering diverse indoor and outdoor illumination, ethnicities, vision aids, and make-up.Videos were recorded at about 95 FPS and manually annotated with pupil ellipse and center positions.Source: §1 · p. 1

2 Labelled pupils in the wild (LPW) dataset

LPW recordings were designed to vary lighting, camera position, and participant appearance across everyday locations. The dataset combines high-speed eye videos with manual pupil annotations using procedures adapted to image conditions.

2 Labelled pupils in the wild (LPW) dataset

  • The collection aimed to vary lighting, eye-camera position, and participant appearance by recording participants at different locations while they followed a moving gaze target.Source: §2 · p. 2
  • The 22 participants represented five ethnicities and five eye colors, and included people with impaired vision, glasses, contact lenses, and strong eye make-up.Source: Participants · p. 2
  • The eye tracker recorded at 95 FPS using a high-speed Pupil Pro and a replacement scene camera capable of up to 149 Hz.The eye tracker itself recorded eye videos at 120 Hz.Source: Apparatus · p. 2
  • Recordings spanned randomly selected locations, including 34.3% outdoors, with varied illumination and camera angles; three roughly 20-second videos were recorded per participant.Participants could keep glasses or contact lenses on during recording.Source: Procedure · p. 2
  • For clear pupils, annotators seeded a similar-intensity connected-area search and defined the pupil center as that area's centroid.For noisy eye videos, they used scene-camera fixation-target tracking and manually annotated pupil positions, then refined annotations by fitting ellipses to boundary points.Source: Ground truth annotation · p. 2

3 Results

Across five algorithms, LPW reveals weak and uneven pupil-detection performance, varying by algorithm, scene conditions, vision aids, and resolution.

  • Roughly 30% of cases had <5px error for Pupil Labs, Swirski, and ExCuSe; about 70% had roughly 40px error for ExCuSe, Swirski, and Gradient.Source: Accuracy and Robustness · pp. 2–3, Figure 3 · p. 3
  • Pupil Labs was fairly accurate in almost 40% of cases but failed completely in the other 60%, while Isophote had the highest average error.Source: Accuracy and Robustness · p. 3
  • The hardest samples mainly involved strong shadows, eyelid occlusions, reflections from glasses, and strong make-up.Source: Accuracy and Robustness · p. 3
  • About 60% of indoor detections versus 50% of outdoor detections had errors of 50px or lower; outdoor sunlight can create pupil and iris reflections and shadows.Source: Indoor vs Outdoor · p. 3, Figure 4 · p. 4
  • Performance was significantly worse for participants wearing glasses, and make-up also greatly disturbed the examined algorithms, which detect dark blobs or strong edges.Source: Glasses and Makeup · p. 3
  • Swirski and ExCuSe improved when downsampling from 480p to 240p, while other algorithms dropped at lower resolutions; Swirski did not work at 30p.Source: Resolution · p. 4, Figure 4 · p. 4

4 Discussion

None of the examined algorithms performed well across the dataset, and detection accuracy was insufficient for good eye tracking in at least half of cases. The evaluation highlighted challenges from glasses, makeup, outdoor recording, and image resolution, while the high frame rate permits consideration of tracking-based algorithms despite controlled, non-natural viewing behavior.

4 Discussion

  • At least half of cases lacked sufficient detection accuracy for good eye tracking, and none of the examined algorithms performed well across the entire dataset.Source: §4 · p. 4
  • Glasses and makeup were severe problems for current algorithms, and outdoor images were more difficult than indoor images; image resolution was also evaluated.Source: §4 · p. 4
  • The high video frame rate supports considering tracking-based algorithms, but gaze-target control means participant viewing behavior was not natural.Source: §4 · p. 4

5 Conclusion

LPW provides a large set of ground-truth annotated eye-region videos for developing and evaluating pupil detection algorithms. Evaluations showed fundamental limitations in current algorithms and identified lighting, image resolution, and vision aids as challenges.

5 Conclusion

  • LPW is a ground-truth annotated eye-region video dataset intended for developing and evaluating pupil detection algorithms.Source: §5 · p. 4
  • The dataset contains 66 high-quality videos and 130,856 frames from 22 participants, recorded in everyday locations at about 95 FPS.Source: §5 · p. 4
  • Evaluations demonstrated fundamental limitations of current algorithms and highlighted challenges related to lighting, image resolution, and vision aids.Source: §5 · p. 4
Loading 1511.05768v1…