This paper introduces a multimodal AI framework to predict student engagement in online learning videos by integrating various behavioral cues, personality models, and providing uncertainty estimates.

In plain terms

Predicting how engaged a student is during online learning videos is incredibly difficult because engagement is a complex mix of behaviors, emotions, and thoughts. The challenge is amplified by high individual differences among students and the subjective nature of judging engagement. To tackle this, the researchers developed a comprehensive AI system that combines various types of information, including subtle spatial and temporal features extracted from video, audio, and images. It also incorporates structured behavioral data like head pose, gaze (where someone is looking), facial expressions (action units), and emotions, integrating these via an advanced neural network architecture. Additionally, the system models student and instructor personalities to better account for individual variations and uses statistical methods to not only predict engagement but also to indicate how confident it is in its predictions. While all methods struggled on a challenging dataset, their framework achieved competitive performance and uniquely offers reliable quantification of prediction uncertainty, which is crucial for practical applications.