Machine Learning Based Emotion Level Assessment

Abstract

With recent advancements of technology, identification of emotions of humans via facial recognition is done with the application of numerous methods including machine learning and deep learning. In this paper, machine learning techniques are applied for identifying different levels of emotions of individuals for unannotated video clips using Facial Action Coding System. In order to archive the above, first, two methods were experimented to obtain a labeled image data set to train classification models where in the first method, clustering of images was done using Action Units(AU) identified from literature and the emotion levels of the images were determined through the resulted clusters and images are labeled according to the cluster they belonged to. In the second method, the image set is analyzed explicitly to identify AUs contributing to emotions rather than relying on those identified in literature and then the clustering of image set was done using those identified AUs to label the images similar to the first method. The two labeled data sets were used to train classification models with Random Forest, Support Vector Machine and K-Nearest Neighbour algorithms separately.Classification models showed better accuracy with data set produced using the second method. An overall F1 score, accuracy, precision and recall of 87% was obtained for the best classification model which is developed using the Random Forest algorithm to identify levels of emotions. Identifying the AU combinations related to emotions and developing a classification model for identifying levels of emotions are the major contributions of this paper. The results of this research would be especially useful to identify levels of emotions of individuals who are having issues in verbal communication.