Machine Learning-Based Facial Beauty Prediction and Analysis of Frontal Facial Images Using Facial Landmarks and Traditional Image Descriptors

Abstract

The beauty industry has seen rapid growth in multiple countries and due to its applications in entertainment, the analysis and assessment of facial attractiveness have received attention from scientists, physicians, and artists because of digital media, plastic surgery, and cosmetics. An analysis of techniques is used in the assessment of facial beauty that considers facial ratios and facial qualities as elements to predict facial beauty. Here, the facial landmarks are extracted to calculate facial ratios according to Golden Ratios and Symmetry Ratios, and an ablation study is performed to find the best performing feature set from extracted ratios. Subsequently, Gray Level Covariance Matrix (GLCM), Hu's Moments, and Color Histograms in the HSV space are extracted as texture, shape, and color features, respectively. Another ablation study is performed to find out which feature performs the best when concatenated with the facial landmarks. Experimental results show that the concatenation of primary facial characteristics with facial landmarks improved the prediction score of facial beauty. Four models are trained, K-Nearest Neighbors (KNN), Linear Regression (LR), Random Forest (RF), and Artificial Neural Network (ANN) on a dataset of 5500 frontal facial images, and amongst them, KNN performs the best for the concatenated features achieving a Pearson's Correlation Coefficient of 0.7836 and a Mean Squared Error of 0.0963. Our analysis also provides us with insights into how different machine learning models can understand the concept of facial beauty.