Source-linked AI summary
Masked Face Recognition Dataset and Application
Zhongyuan Wang, Guangcheng Wang, Baojin Huang, Zhangyang Xiong, Qi Hong, Hao Wu, Peng Yi, Kui Jiang, Nanxi Wang, Yingjiao Pei, Heling Chen, Yu Miao, Zhibing Huang, Jinbi Liang
TL;DR
Mask wearing has reduced the effectiveness of conventional face recognition, while deep-learning approaches lack publicly available masked-face training data. The paper constructs three masked-face datasets and develops a multi-granularity model, which achieves 95% recognition accuracy and supports contactless authentication applications.
Problem
Conventional face recognition is ineffective for masked faces, and no publicly available masked-face dataset existed despite deep-learning methods requiring many training samples.
Method
The paper constructs MFDD, RMFRD, and SMFRD and trains a face-eye-based multi-granularity model using public, simulated, and real-world masked-face data.
Results
95% recognition accuracy was achieved, exceeding the reported industry results of 85% and over 90%.
Takeaways & Limitations
The datasets and algorithm support contactless face authentication in community access, campus management, and enterprise resumption scenarios.
Abstract
from arXiv · showhide
In order to effectively prevent the spread of COVID-19 virus, almost everyone wears a mask during coronavirus epidemic. This almost makes conventional facial recognition technology ineffective in many cases, such as community access control, face access control, facial attendance, facial security checks at train stations, etc. Therefore, it is very urgent to improve the recognition performance of the existing face recognition technology on the masked faces. Most current advanced face recognition approaches are designed based on deep learning, which depend on a large number of face samples. However, at present, there are no publicly available masked face recognition datasets. To this end, this work proposes three types of masked face datasets, including Masked Face Detection Dataset (MFDD), Real-world Masked Face Recognition Dataset (RMFRD) and Simulated Masked Face Recognition Dataset (SMFRD). Among them, to the best of our knowledge, RMFRD is currently theworld's largest real-world masked face dataset. These datasets are freely available to industry and academia, based on which various applications on masked faces can be developed. The multi-granularity masked face recognition model we developed achieves 95% accuracy, exceeding the results reported by the industry. Our datasets are available at: https://github.com/X-zhangyang/Real-World-Masked-Face-Dataset.
I. BACKGROUND
Widespread mask wearing has made conventional face recognition ineffective for important authentication settings. The paper addresses the need for masked-face recognition data and methods that work with incompletely exposed faces.
- Mask wearing has undermined face recognition in applications including community access, attendance, train-station gates, and mobile payment.
- Figure 1 contrasts pairs of normal face images with pairs of masked face images.
- Deep-learning face recognizers depend on massive training datasets, but no publicly available masked-face dataset existed.
II. PROPOSED DATASETS
The paper introduces three complementary masked-face datasets for detection and recognition. They differ in construction and support requirements, with recognition datasets requiring matched masked and unmasked images for each subject.
- Masked-face detection needs masked images, whereas recognition requires multiple masked and unmasked images of the same subjects.
- The paper proposes MFDD, RMFRD, and SMFRD as three types of masked-face datasets.
- MFDD: MFDD contains 24,771 masked face images annotated for mask presence and masked-face coordinates.
- SMFRD: Figure 2 presents samples from a set of simulated masked face images.
- RMFRD: RMFRD includes 5,000 masked images and 90,000 unmasked images of 525 people collected and filtered from Internet resources.
III. MASKED FACE RECOGNITION
Masked-face recognition is easier in controlled settings with frontal, cooperative subjects than in uncontrolled surveillance conditions. The proposed model combines diverse datasets with attention to visible facial regions and reports substantially higher accuracy.
- Controlled settings provide high-quality frontal images, while uncontrolled conditions combine uncertain distance, pose, occlusion, and lighting with lower recognition accuracy.
- The method excludes mask interference and prioritizes useful exposed face features.
- The model combines public, simulated, and real-world masked-face data in a face-eye-based multi-granularity recognition model.
- 95% recognition accuracy was achieved, up from an initial 50%, by weighting visible features such as contour, ocular and periocular details, and forehead.
IV. APPLICATION STATUS AND PROSPECT
The reported model reaches 95% masked-face recognition accuracy, although masked-face recognition remains below regular face recognition. The datasets and algorithm are intended for contactless authentication and related operational settings.
- 95% masked-face recognition accuracy exceeds the reported industry results of 85% and over 90%.
- Masked-face recognition remains less reliable than regular facial recognition, which is reported as exceeding 99% accuracy.
- The datasets and algorithm target contactless face authentication in community access, campus management, and enterprise resumption scenarios.
- The paper argues that masked-face recognition needs will persist because people frequently wear masks during haze weather.