Student Modeling in an Intelligent Tutoring System

Introduction

In the United States there are concerns about poor student performance on new high-stakes standards based tests required by the No Child Left Behind Act (NCLB, 2002) legislation. To address this issue, educational technologies, like intelligent tutoring systems (ITS) have been developed and proven to be useful helping students learn. For instance, the Cognitive Tutors leads to large learning gains (Koedinger et al, 1997). Recently, President Obama made a commitment to increase investment on educational software, saying that, “[W]e will devote more than 3 percent of our GDP to research and development. … Just think what this will allow us to accomplish: solar cells as cheap as paint, …, learning software as effective as a personal tutor….” ¹ So, how do we build ITS that are as effective as a personal tutor? To create an effective piece of learning software, we need a good model of student learning. Assessment of an examinee’s ability is the first step of student modeling in an ITS because student state is a prerequisite for creating a pedagogical strategy. The student model provides valuable information for the tutor to help build tutoring strategy (e.g., when to interrupt and what to say when interrupt), problem sequencing (e.g., what’s the next appropriate task to give to a student so his learning gain is maximized), performance prediction (how a student will respond to a step associated with certain rules), etc.

Recently, in an interview with U.S. News & World Report (Ramírez & Clark, 2009), U.S. Secretary of Education Arne Duncan weighed in on the NCLB Act and called for continuous assessment. He said that he is concerned about over-testing, and feels that fewer, better tests would be more effective. He wants to develop better data management systems that will help teachers track individual student’s progress in real-time, so teachers and parents can assess and monitor student strengths and weaknesses. To reflect on the “continuous assessment” idea, we think, one way that ITS research distinguishes itself from other educational software development is that it is concerned with modeling the knowledge of the learner in some computationally useful and inspectable way (McCalla & Greer, 1994). The modeling phase should involve understanding learner behavior in the rich context of the environment in which learning occurs, thus, obtaining a better understanding of each student’s pre-existing, or current knowledge status and how that knowledge is changing over time.

Student models in ITS are constructive, especially cognitive models. Cognitive modeling involves a great deal of detailed protocol collection and task analysis. The models are not easy to construct and are difficult to verify. Yet, cognitive modeling is very important in an ITS as it is the basis of cognitively diagnostic assessment and teacher reporting. Wiliam (2006) describes an assessment as formative only if information about what is being assessed results in change that would otherwise not occur. Here the definition of formative assessment becomes more detailed to the utility for teachers to understand if and how students are learning. By giving teachers cognitively diagnostic data in a timely fashion, teachers can change their teaching in response to the data they collect about student understanding. The US Department of Education (2003) stated, “Research shows that teachers who use student test performance to guide and improve teaching are more effective than teachers who do not use such information” (p. 2). Therefore, if assessments are to move from assessments of learning to assessments for learning (c.f., Stiggins 2005) then we must continue to focus on the box that encompasses diagnostic modeling data and teachers as the end-users.