We describe a decision-theoretic tutor that helps students learn from Analogical Problem Solving (APS), i.e., from problem-solving activities that involve worked-out examples. This tutor incorporates an innovative example-selection mechanism that tailors the choice of example to a given student so as to trigger studying behaviors that are known to foster learning. The mechanism relies on a two-phase decision-theoretic process, as follows. First, a probabilistic user model corresponding to a dynamic Bayesian network simulates how a given student will use an example to solve a problem and what she will learn from doing so. Second, this simulation is quantified via an expected utility calculation, enabling the tutor to select the example with the highest expected utility for maximizing learning and problem-solving outcomes. Once an example is presented to a student, the user model generates an assessment of the student’s APS, enabling the selection mechanism to have up to date information on the student. Our empirical evaluation shows that this selection mechanism is more effective than standard selection approaches for fostering learning from APS. Here, we provide a comprehensive technical description of the example-selection mechanism, as well as an overview of its evaluation and a discussion of some of the challenges related to our decision-theoretic approach.
TopIntroduction
Intelligent Tutoring Systems (ITSs) are computer applications that employ artificial intelligence techniques to instruct students in an “intelligent way” (VanLehn, 1988). Although there isn’t an accepted definition of the term “intelligent”, a characteristic shared by many ITSs is that they possess knowledge and reasoning capabilities to adapt instruction to the needs of each individual student. This functionality is motivated by research demonstrating that students learn more effectively from tailored, one-on-one instruction, as compared to standard classroom instruction (Bloom, 1984). Achieving the goal of computer-based individualized instruction, however, is extremely challenging. To tailor pedagogical interventions to students’ needs, an ITS needs information on students’ states of interest, such as the evolution of knowledge as a result of interaction with a tutor (to provide appropriate instructional scaffolding), affective states (to generate empathic tutorial responses), and general learning and reasoning skills, or meta-cognitive skills (to provide instruction that promotes general learning abilities in addition to domain-dependent expertise). Information on the relevant student states can be difficult to obtain unobtrusively, making student modeling a process permeated with uncertainty. Uncertainty also permeates the selection of appropriate tutorial actions, because even if the tutor were able to obtain perfect information on the current state of the student, the effects of each available tutorial action cannot be unequivocally predicted from its theoretical underpinnings.
It is quite common for ITSs to deal with the uncertainty in student modeling by relying on formal approaches for reasoning under uncertainty (for an overview, see (Woolf, 2008)). Less common is to explicitly take uncertainty into account during action selection: most ITSs perform action selection based on ad-hoc heuristics. For instance, a common approach for deciding the content of a hint is to (1) a priori associate hints with the domain principles needed to solve the problem, (2) at run-time identify the principle the student is likely to need help on and display the associated hint (e.g., (Conati et al., 2002). Such ad-hoc approaches can run the risk of sub-optimal pedagogical action selection, because they do not take into the account the fact that there is uncertainty about the effect of the available tutorial actions, and they do not explicitly represent how this uncertainty, together with the uncertainty on the student state, impact the decision-making process. An alternative for generating individualized instruction is to rely on a decision-theoretic approach, where an ITS’s decision with respect to tutorial action selection depends on both what it believes (represented using probability theory) and what it wants (represented using utility theory). The advantage of this approach is that the selected tutorial action is guaranteed to be optimal, given the available information about the student and the preferences for the tutor’s behavior1.
In this chapter, we describe a decision-theoretic ITS, namely the Example Analogy (EA)- Coach. The tutor targets meta-cognitive skills needed to learn from a specific type of instructional activity: solving problems in the presence of worked-out examples, also referred to as Analogical Problem Solving (APS). We chose this activity since examples play a key role in cognitive skill acquisition: students rely heavily on examples when learning a new skill (Pirolli and Anderson, 1985; Reed, Dempster et al., 1985; Novick, 1995; VanLehn, 1998) and examples are more effective aids to problem solving than general procedures alone (Reed and Bolstad, 1991) or hints on the instructional material (Ringenberg and VanLehn, 2006). However, many students do not use examples effectively during APS (Reed, Dempster et al., 1985; VanLehn, 1998) and so support is needed to maximize learning and problem solving for these students.