Source-linked AI summary
SCUT-FBP5500: A Diverse Benchmark Dataset for Multi-Paradigm Facial Beauty Prediction
Lingyu Liang, Luojun Lin, Lianwen Jin, Duorui Xie, Mengru Li
TL;DR
Facial beauty prediction requires benchmarks that accommodate classification, regression, and ranking rather than a single supervised-learning paradigm. The paper introduces SCUT-FBP5500, a diverse 5500-face dataset with multiple labels, evaluates shallow and deep predictors, and reports improved FBP performance while identifying further gains from data augmentation as future work.
Problem
Existing facial beauty benchmarks were built under specific computational constraints, although FBP has been formulated as classification, regression, and ranking.
Method
The paper constructs SCUT-FBP5500 with diverse faces, beauty scores, score distributions, and 86 facial landmarks, then evaluates shallow and deep FBP models.
Results
The dataset supports multiple FBP paradigms, and evaluations with shallow and deep models indicate improved FBP performance; most score standard deviations fall within 0.6–0.7.
Takeaways & Limitations
SCUT-FBP5500 provides a diverse benchmark adaptable to appearance-based or shape-based classification, regression, and ranking tasks.
Abstract
from arXiv · showhide
Facial beauty prediction (FBP) is a significant visual recognition problem to make assessment of facial attractiveness that is consistent to human perception. To tackle this problem, various data-driven models, especially state-of-the-art deep learning techniques, were introduced, and benchmark dataset become one of the essential elements to achieve FBP. Previous works have formulated the recognition of facial beauty as a specific supervised learning problem of classification, regression or ranking, which indicates that FBP is intrinsically a computation problem with multiple paradigms. However, most of FBP benchmark datasets were built under specific computation constrains, which limits the performance and flexibility of the computational model trained on the dataset. In this paper, we argue that FBP is a multi-paradigm computation problem, and propose a new diverse benchmark dataset, called SCUT-FBP5500, to achieve multi-paradigm facial beauty prediction. The SCUT-FBP5500 dataset has totally 5500 frontal faces with diverse properties (male/female, Asian/Caucasian, ages) and diverse labels (face landmarks, beauty scores within [1,~5], beauty score distribution), which allows different computational models with different FBP paradigms, such as appearance-based/shape-based facial beauty classification/regression model for male/female of Asian/Caucasian. We evaluated the SCUT-FBP5500 dataset for FBP using different combinations of feature and predictor, and various deep learning methods. The results indicates the improvement of FBP and the potential applications based on the SCUT-FBP5500.
I. INTRODUCTION
Facial beauty prediction spans classification, regression, and ranking, but existing datasets typically support only specific computational paradigms. SCUT-FBP5500 addresses this gap with diverse faces, labels, and benchmark evaluations for multi-paradigm FBP.
- Facial beauty prediction has been formulated as classification, regression, or ranking, making it intrinsically a multi-paradigm computation problem.
- Existing FBP datasets often impose specific computational constraints, limiting model flexibility and cross-paradigm comparison.
- SCUT-FBP5500 contains 5500 frontal faces with diverse gender, race, age, landmarks, beauty scores, and score distributions.
- The dataset supports appearance-based and shape-based classification, regression, and ranking models across Asian and Caucasian male and female subsets.
- Evaluations include hand-crafted-feature shallow predictors and state-of-the-art deep learning models, with results indicating improved FBP performance on the more diverse dataset.
A. Face Images Collection
SCUT-FBP5500 comprises 5500 frontal, unoccluded neutral-expression faces aged 15–60, organized into four race-and-gender subsets. The section also visualizes beauty-score distributions and reports their outlier statistics.
- Face Images Collection: 5500 frontal, unoccluded faces aged 15–60 with neutral expressions make up SCUT-FBP5500.The images were collected mainly from the Internet, with some Asian faces from DataTang and some Caucasian faces from the 10k US Adult database.
- Face Images Collection: The dataset contains 2000 Asian females, 2000 Asian males, 750 Caucasian females, and 750 Caucasian males.
- Face Images Collection: Table II reports the outlier number and portion of beauty scores for the four race-and-gender groups.
- Face Images Collection: Gaussian and piecewise curves visualize beauty-score distributions for Caucasian and Asian females and males.The groups are labeled CF, CM, AF, and AM.
B. Facial Beauty Scores and Facial Landmarks
The dataset provides volunteer-assigned beauty scores and manually verified facial landmarks, supporting both perceptual and geometric analysis. Score variability is examined across the four race-and-gender subsets.
- Facial Beauty Scores and Facial Landmarks: Beauty scores from 1 to 5 were assigned by 60 volunteers aged 18–27 through a web-based crowdsourcing system.A score of 5 denotes the most attractive rating, and the four subsets were labeled separately.
- Facial Beauty Scores and Facial Landmarks: 86 facial landmarks mark significant components including the eyes, eyebrows, nose, and mouth.Active-shape-model detections were manually modified by volunteers to improve landmark accuracy.
- Facial Beauty Scores and Facial Landmarks: Standard-deviation distributions are shown separately for Caucasian female, Asian female, Caucasian male, and Asian male faces.
- Facial Beauty Scores and Facial Landmarks: Box figures summarize standard deviations for CF, AF, CM, and AM subsets.
- Facial Beauty Scores and Facial Landmarks: Benchmark analysis covers beauty scores, labelers, and facial landmarks across the four gender-and-race groups.
A. Distribution of Beauty Scores
Beauty-score labels were analyzed across the four demographic subsets, with outlier filtering and distribution fitting used to characterize the score data. The subsets’ score distributions were approximately modeled by two Gaussian components.
- Outlier Processing: The average score from all 60 labelers was treated as ground truth, and scores differing by more than 2 were removed as outliers for visualization.The original and preprocessed distributions were reported as mostly similar.
- Outlier Processing: The number and proportion of outliers were small across the four subsets, indicating a reliable labeling process for beauty scores.
- Distribution Fitting: The beauty-score distributions for Asian and Caucasian female and male subsets were approximately fitted by a mixed distribution with two Gaussian components.Both Gaussian and piecewise fitting schemes were used after preprocessing.
B. Standard Deviation of Beauty Scores
Labeler-score variability was generally moderate, while agreement between male and female labelers differed by race. Asian-face scores showed persistently higher cross-gender correlation than Caucasian-face scores.
- Score Variability: Most standard deviations between individual labeler scores and the ground truth fell within the range [0.6, 0.7].The standard-deviation distribution was described as similar to a Gaussian distribution.
- Score Variability: The labeler-score standard deviations were summarized using both a histogram and a box figure.
- Cross-Gender Agreement: Male and female labelers showed persistently higher beauty-score correlation for Asian faces than for Caucasian faces.The authors relate this pattern to better perception of same-race facial beauty.
D. PCA Analysis of Facial Geometry
PCA was used to visualize facial-landmark geometry, while shallow FBP experiments combined geometric or appearance features with predictors across demographic subsets and the whole dataset.
- D. PCA Analysis of Facial Geometry: PCA visualized the mean and five principal components of 86-point facial geometry for Asian female and Asian male faces.Caucasian faces were reported to have a similar landmark distribution.
- D. PCA Analysis of Facial Geometry: The analysis identified face shape as one of the main components influencing the geometry of facial beauty.
- Shallow Prediction: Shallow geometric-feature predictors used an 18-dimensional ratio vector with linear, Gaussian, and support vector regression models.Performance was measured with PC, MAE, and RMSE after 10-fold cross-validation for Asian and Caucasian male and female subsets.
- Shallow Prediction: Appearance-based shallow predictors were evaluated using Gabor features extracted with two sampling schemes and PCA-based dimensionality reduction.
B. Appearance Feature with Shallow Predictor
Appearance-based shallow prediction used sampled Gabor features, while the benchmark also compared AlexNet, ResNet-18, and ResNeXt-50 using PC, MAE, and RMSE.
- Gabor Feature Extraction: Forty Gabor feature maps were extracted from each image across five directions and eight angles for appearance-based FBP.
- Sampling Schemes: The 86-keypoints scheme produced a 3340-dimensional feature vector, whereas 64UniSample produced a 2560-dimensional vector.PCA reduced the extracted feature dimension before predictor training.
- Deep Model Comparison: AlexNet, ResNet-18, and ResNeXt-50 were compared using PC, MAE, and RMSE under 5-fold cross-validation and a 60% training–40% testing split.
V. FBP EVALUATION VIA DEEP PREDICTOR
Three CNN architectures were evaluated under cross-validation and a fixed train/test split. ResNeXt-50 performed best, while deep CNNs outperformed shallow predictors using hand-crafted features; cross-validation was slightly more accurate.
- Model and evaluation settings: AlexNet, ResNet-18, and ResNeXt-50 were evaluated using ImageNet-pretrained weights under two experimental settings.The settings were 5-fold cross-validation and training on 60% of samples with testing on the remaining 40%.
- Model comparison: ResNeXt-50 obtained the best performance among the evaluated CNN models in both experimental settings.Its performance exceeded that of ResNet-18 and AlexNet.
- Model comparison: Deep CNN models outperformed shallow predictors using hand-crafted geometric or appearance features.The comparison supports end-to-end feature learning for facial beauty prediction.
- Evaluation settings: 5-fold cross-validation produced slightly higher accuracy than the 60% training and 40% testing split.The paper suggests that the larger and more diverse training set used in cross-validation may explain this difference.
VI. CONCLUSION
The paper presents SCUT-FBP5500 as a diverse benchmark for multi-paradigm facial beauty prediction. Its varied faces and labels support different tasks and models, and evaluations indicate dataset reliability.
- Dataset: SCUT-FBP5500 contains 5500 frontal faces spanning sex, ethnicity, and age diversity.The dataset includes male and female Asian and Caucasian faces across ages.
- Dataset: The dataset provides face landmarks, beauty scores within, and beauty score distributions for different facial beauty prediction paradigms.These labels support appearance-based and shape-based classification, regression, and ranking models.
- Evaluation: Evaluations using varied feature-predictor combinations and deep learning models indicate the reliability of SCUT-FBP5500.The dataset is designed to adapt to different facial beauty prediction tasks.