Cognitive Diagnosis of Students' Test Performance Based on Probability Inference

Cognitive Diagnosis of Students' Test Performance Based on Probability Inference

Junjie Xu (School of Information Science and Technology, Dalian Maritime University, Liaoning, China) and Rong Chen (School of Information Science and Technology, Dalian Maritime University, Liaoning, China)
DOI: 10.4018/ijisss.2014010103


Cognitive diagnosis is aimed at inferring the degree of cognitive state from observations. This paper considers cognitive diagnosis as an instance of model-based diagnosis, which has been studied in artificial intelligence for many years. The model-based cognitive diagnosis we present runs on a model of students' courses in terms of knowledge items that they may learn, tests them and helps them to understand their faults in cognition, and thus improves their learning performance in an E-learning environment. To do so, courses are formally defined as set of knowledge items with requirement constraints, and associated with a set of exam questions. Moreover, the authors introduce Bayesian net to build a model of cognitive diagnosis, using probabilistic inference on it to help a student understand what knowledge item he/she does not master, and the recommendations like what should be done next. Experimental results show that the group of students with such understanding can improve their testing performance greatly in an E-learning environment. Although the demo system has been integrated with a specific computerized adaptive testing system, the general technique could be applied to a broad class of intelligent tutoring systems.
Article Preview


Educational researchers have investigated Intelligent Tutoring Systems (ITS) as a means of providing cost effective yet personalized tuition for many years. ITS implementations have demonstrated student learning improvements comparable to the assistance of a human tutor with intermediate expertise (Graesser et al., 2000). In an ITS cognitive diagnosis is often used synonymously with 'student modeling' that builds a representation of the student’s knowledge from the evidence provided by student inputs to solve problems (Self, 1993). A widely accepted definition of cognitive diagnosis has been given by Ohlsson (1986) “cognitive diagnosis is the process of inferring a person’s cognitive state from his or her performance.” It is widely accepted that cognitive diagnosis can help in personalized tuition an ITS must know, in particular what the student knows, what misconceptions he/she has, and thus help him/her improve learning outcomes.

Cognitive diagnostic model has become an exciting new field of psychometric research. Cognitive diagnostic models aim to diagnose students’ status of a group of discretely defined skills, thereby providing them with detailed information regarding their specific strengths and weaknesses (Huebner, 2010). To date, at least fourteen distinct cognitive diagnostic models have appeared in the literature (Xu, Chang, & Douglas, 2003). The NIDA model and the Fusion model are good examples of these models (Xu, et al., 2003); they can be thought of as a Q matrix, each row of which is a list of the cognitive attributes that an examinee needs to have mastered in order to give a correct response to the item. Rule-based models are also used to depict both desired students’ knowledge and common mistakes (McLaren, 2004), thereby student performance is monitored by comparing actual student problem solving sequences against the known production rules.

Although a number of methods for cognitive diagnosis have been developed, these methods have not been placed on a more rigorous rooting until John Self placed this problem in a general theoretical framework in 1993. He considered the problem of cognitive diagnosis as an instance of Model-Based Diagnosis (MBD), where many issues in cognitive diagnosis, previously discussed informally, are mapped onto formal diagnosis in Artificial Intelligence (AI) (Self, 1993). To illustrate the problem of diagnosing students’ attempts to solve problems, a three-column subtraction is modeled a subtraction problem-solving 'circuit' and a form of heuristic search is used in Ohlsson (1986) to find plausible sequences of actions that lead to a known solution. However, as an infinite number of ineffective behavior sequences are possible, such a cognitive diagnostic model is time consuming to develop and rarely complete.

As the precise cognitive modeling of a student is likely to be intractable, adaptive remediation methods have been used successfully by applying instructional techniques appropriate to the current student context rather than requiring complex cognitive student and domain modeling. This paper considers the problem of computerized cognitive diagnosis by combining model-based reasoning with a traditional testing system in order to provide the Computerized Adaptive Testing (CAT). Like Self, we put this problem in a Bayesian network framework to better define the nature of the cognitive diagnosis problem in CAT; we recast an adaptive course testing as a diagnosis system in Bayesian network; a course description is mapped onto the system description, knowledge items onto components, and observations are students’ inputs that indicate their test performance. As distinguished from Self’s work, we seek to use model-based cognitive diagnosis with hierarchical course modeling and employ adaptive remediation based upon the student’s current state and learning difficulties encountered revealed by his/her test behavior.

Complete Article List

Search this Journal:
Open Access Articles: Forthcoming
Volume 11: 4 Issues (2019): 1 Released, 3 Forthcoming
Volume 10: 4 Issues (2018)
Volume 9: 4 Issues (2017)
Volume 8: 4 Issues (2016)
Volume 7: 4 Issues (2015)
Volume 6: 4 Issues (2014)
Volume 5: 4 Issues (2013)
Volume 4: 4 Issues (2012)
Volume 3: 4 Issues (2011)
Volume 2: 4 Issues (2010)
Volume 1: 4 Issues (2009)
View Complete Journal Contents Listing