Recognizing Student Emotions using Brainwaves and Mouse Behavior Data

Recognizing Student Emotions using Brainwaves and Mouse Behavior Data

Judith Azcarraga (Center for Empathic Human-Computer Interactions, De La Salle University, Manila, Philippines) and Merlin Teodosia Suarez (Center for Empathic Human-Computer Interactions, De La Salle University, Manila, Philippines)
Copyright: © 2013 |Pages: 15
DOI: 10.4018/jdet.2013040101
OnDemand PDF Download:
$37.50

Abstract

Brainwaves (EEG signals) and mouse behavior information are shown to be useful in predicting academic emotions, such as confidence, excitement, frustration and interest. Twenty five college students were asked to use the Aplusix math learning software while their brainwaves signals and mouse behavior (number of clicks, duration of each click, distance traveled by the mouse) were automatically being captured. It is shown that by combining the extracted features from EEG signals with data representing mouse click behavior, the accuracy in predicting academic emotions substantially increases compared to using only features extracted from EEG signals or just mouse behavior alone. Furthermore, experiments were conducted to assess the prediction accuracy of the system at points during the learning session where several of the extracted features significantly deviate in value from their mean. The experiments confirm that the prediction performance increases as the number of feature values that deviate significantly from the mean increases.
Article Preview

Introduction

Students experience various emotions while engaged in learning. Such emotions, also referred to as academic emotions (Pekrun, 2002), may affect the flow of learning as well as the motivation to continue with the learning task. This has been the challenge for the human tutors and even for those who develop intelligent tutoring systems (ITS). Indeed, effective tutors, whether human tutors or computer-based intelligent tutoring systems, are those who are not only aware of the cognitive needs of the students but also of their affective needs.

With this in mind, recent research projects in the area of ITS have tried to address not only the cognitive needs of students but their affective needs as well. Such affective systems, also referred to as affective tutoring systems, consider the effect of emotions in the learning process of a learner as well as the typical emotional patterns under different learning scenarios. Academic emotions typically experienced by a learner while using a tutoring system are confidence (Arroyo et al., 2009; Azcarraga et al., 2011a, 2011b, 2011c; Ibañez et al., 2011), excitement (Arroyo et al., 2009; Azcarraga et al., 2011a, 2011b, 2011c; Ibañez et al., 2011), frustration (Arroyo et al., 2009; Azcarraga et al., 2011a, 2011b, 2011c; Burleson, 2006; Ibañez et al., 2011), interest (Arroyo et al., 2009; Azcarraga et al., 2011a, 2011b, 2011c; D’Mello & Graesser, 2009; Ibañez et al., 2011; Kapoor, Burleson & Picard, 2007) , flow/engagement (D’Mello & Graesser, 2009; Stevens, Galloway & Berka, 2007), boredom (D’Mello & Graesser, 2009) and confusion (D’Mello & Graesser, 2009).

Affective tutoring systems are capable of recognizing student affect based on tutorial information complemented with the user profile. Furthermore, these systems sometimes also include a combination of facial expression, gesture and physiological signals. In (Arroyo et al., 2009), affective states such as confident, frustrated, excited and interested are predicted with high accuracy using special devices such as a camera to capture facial expression, posture chair to monitor the level of engagement, pressure-sensitive mouse and skin-conductance sensor. Similarly, Burleson (2006) uses the same set of devices in order to predict student frustration and the need for help. Moreover, tutorial information such as conversational cues, posture and facial features are used in Autotutor to predict boredom, flow/engagement, confusion and frustration (D’Mello & Graesser, 2009).

Complete Article List

Search this Journal:
Reset
Open Access Articles: Forthcoming
Volume 15: 4 Issues (2017)
Volume 14: 4 Issues (2016)
Volume 13: 4 Issues (2015)
Volume 12: 4 Issues (2014)
Volume 11: 4 Issues (2013)
Volume 10: 4 Issues (2012)
Volume 9: 4 Issues (2011)
Volume 8: 4 Issues (2010)
Volume 7: 4 Issues (2009)
Volume 6: 4 Issues (2008)
Volume 5: 4 Issues (2007)
Volume 4: 4 Issues (2006)
Volume 3: 4 Issues (2005)
Volume 2: 4 Issues (2004)
Volume 1: 4 Issues (2003)
View Complete Journal Contents Listing