Text-Based Affect Detection in Intelligent Tutors

Introduction

Attempts to acquire a deep level understanding of conceptual information requires cognitive activities such as a systematic exploration of the problem space, generating self-explanations, making bridging inferences, asking diagnostic questions, causal reasoning, and critical thinking. These effortful deep reasoning and problem solving activities often lead to episodes of failure and the learner experiences a host of affective responses (Mandler, 1984; Stein & Levine, 1991). Negative emotions are experienced when expectations are not met, failure is imminent, and important goals are blocked. For example, confusion occurs when learners face obstacles to goals, contradictions, incongruities, anomalies, uncertainty, and salient contrasts (Festinger, 1957; Graesser & Olde, 2003; Piaget, 1952). Unresolved confusion can lead to irritation, frustration, anger, and sometimes even rage.

On the other hand, a learner may experience a host of positive emotions when misconceptions are confronted, challenges are uncovered, insights are unveiled, and complex concepts are mastered. Students who are actively engaged in the learning session may have a flow-like experience when they are so engrossed in the material that time and fatigue disappears (Csikszentmihalyi, 1990). They may also experience other positive emotions such as delight, excitement, and even one of those rare eureka (i.e. “a ha”) moments. Simply put, emotions are systematically affected by the knowledge and goals of the learner, as well as vice versa (Dweck, 2002; Mandler, 1984; Stein & Levine, 1991). Cognitive activities such as causal reasoning, deliberation, goal appraisal, and planning processes operate continually throughout the experience of emotion.

Given this inextricable link between emotions and learning, it is reasonable to hypothesize that computerized learning environments that are sensitive to the affective and cognitive states of a learner would positively influence learning. This is particularly true when deep learning is accompanied by confusion, frustration, anxiety, boredom, delight, flow, surprise and other affective experiences (D'Mello, Graesser, & Picard, 2007; Graesser, Chipman, King, McDaniel, & D'Mello, 2007; Lepper & Woolverton, 2002; Picard, 1997). An affect-sensitive learning environment would incorporate assessments of the students’ cognitive, affective, and motivational states into its pedagogical strategies to keep students engaged, boost self-confidence, heighten interest, and presumably maximize learning. For example, if the learner is frustrated, the Intelligent Tutoring System (ITS) might generate hints to advance the learner in constructing knowledge and make supportive empathetic comments to enhance motivation. If the learner is bored, the tutor might present more engaging or challenging problems for the learner to work on.

Affect-sensitive ITSs need to detect learners’ emotions before they can be responsive to their emotions. Affect detection does not need to be perfect, but must have a reasonable degree of accuracy. The affect detection problem is quite challenging because emotions are notoriously noisy, murky, and have fuzzy boundaries. This is compounded by the fact that there are individual differences and contextual influences in experience and expression. Despite these challenges, the aspiring affect-sensitive ITSs leverage recent advances in affective computing (Picard, 1997) to automatically detect learners’ affective states. The early systems primarily monitored physiological measures, such as skin conductance and heart rate, or bodily measures, such as facial expressions, posture, and gestures, and paralinguistic features of speech such as loudness, pitch, and formant frequencies (Pantic & Rothkrantz, 2003; Zeng, Pantic, Roisman, & Huang, 2009)