Please use this identifier to cite or link to this item:
Title: A User-oriented Ensemble Method for Multi-Modal Emotion Recognition
Authors: Iddamalgoda, N
Thrimavithana, P
Fernando, H
Ratnayake, T
Priyadarshana, Y. H. P. P
Aththidiye, R
Kasthurirathna, D
Keywords: emotion recognition
ensemble methods
deep learning
machine learning
Issue Date: 12-Dec-2019
Publisher: SLAAI - International Conference on Artificial Intelligence
Series/Report no.: Sri Lanka Association for Artificial Intelligence (SLAAI);Pages 150-156
Abstract: Emotions play a vital role in mental and physical activities of human lives. One of the biggest challenges in Human-Computer Interaction is emotion recognition. With the resurgence in the fields of Artificial Intelligence and Machine learning, a considerable number of studies have been carried out in order to address the challenge of emotion recognition. The individual heterogeneity of expressing emotions is a key problem that needs to be addressed in accurately detecting the emotional state of an individual. The purpose of this work is to propose a novel ensemble method to predict the emotions using a multimodal approach. The presented multimodal approach with the modalities of facial expressions, voice variations and, speech and social media content, are used to identify seven emotional states: anger, fear, disgust, happiness, sadness, surprise and neutral emotion. In this study, for the facial expression-based emotion recognition and voice variation-based emotion recognition, Deep Neural Network models have been used, and for emotion recognition using speech and social media content, Multinomial Naïve Bayesian algorithm is used. The mentioned three modalities were integrated using a novel ensemble method that captures the heterogeneity of individuals in how they express their emotions. The proposed ensemble method was evaluated with respect to real states of human emotions of a sample user group and the experimental results suggest that the suggested ensemble method may be more accurate in recognizing emotions. Accurate recognition of emotions may have myriad applications in domains such as healthcare, advertising and human resource management.
Appears in Collections:Research Papers - Open Access Research
Research Papers - SLIIT Staff Publications

Files in This Item:
File Description SizeFormat 
A-User-oriented-Ensemble-Method-for-Multi-Modal-Emotion-Recognition.pdf663.79 kBAdobe PDFView/Open

Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.