Computer Vision and NLP based Multimodal Ensemble Attentiveness Detection API for E-Learning

Abstract

Attention is the fundamental element of effective learning, memory, and interaction. Learning however, with the evolvement of technologies in the modern digital age, has surpassed traditional learning systems to more convenient online or e-learning systems. Nevertheless, unlike in the traditional learning systems, attention detection of a student in an e-learning environment remains one of the barely explored areas in Human Computer Interaction. This study proposes a multimodal ensemble solution to detect the level of attentiveness of a student in an e-learning environment, with the use of computer vision, natural language processing, and deep learning to overcome the barriers in identifying user attention in e-learning. The proposed multimodal captures, processes, and predicts user attentiveness levels of individual students, which are subsequently aggregated through an ensemble model to derive an overall outcome of better accuracy than individual model outcomes. The final outcome of the ensemble model produces a range of percentages, within which the attentiveness level of the student lies during a single online lesson. This range is consequently delivered to the users through an Application Programming Interface.