There is little room for doubt about that cognitive diagnosis has received much attention recently. Computerized adaptive testing (CAT) is adaptive, fair, and efficient, which is suitable to large-scale examination. Traditional cognitive diagnostic test needs quite large number of items, the efficient and tailored CAT could be a remedy for it, so the CAT with cognitive diagnosis (CD-CAT) is prospective. It is more beneficial to the students who live in the developing area without rich source of teaching, and distance education is adopted there. CD is still in its infancy (Leighton at el.2007), and some flaws exist, one of which is that the rows/columns could form a Boolean lattice in Tatsuoka’s Q-matrix theory. Formal Concept Analysis (FCA) is proved to be a useful tool for cognitive science. Based on Rule Space Model (RSM) and the Attribute Hierarchy Method (AHM), FCA is applied into CD-CAT and concept lattices are served as the models of CD. The algorithms of constructing Qr matrice and concept lattices for CAT, and the theory and methods of diagnosing examinees and offering the best remedial measure to examinees are discussed in detail. The technology of item bank construction, item selection strategies in CD-CAT and estimation method are considered to design a systemic CD-CAT, which diagnoses examinees on-line and offers remedial measure for examinees in time. The result of Monte Carlo study shows that examinees’ knowledge states are well diagnosed and the precision in examinees’ abilities estimation is satisfied.
TopIntroduction
Teachers, parents, and students look forward to know students’ knowledge deficiencies because it benefits teaching quality and reduces the burden on students, but it is not easy to achieve. Amplified by the current No Child Left Behind, Act of 2001 (No Child Left Behind Act, 2001), contemporary assessments are expected to provide more informative diagnostic reports to students, parents, teachers and principals that enable successful instructional intervention. In 2009, The Obama administration announced “Race to the Top” Assessment Program would provide $350 million in competitive grants to support the development of a new generation of multi-state assessment systems including cognitive diagnosis assessment. A paper in “SCIENCE” discussed the potential of technology to launch a new era of integrated, learning-centered assessment systems, especially for cognitive diagnosis (Edys S, 2009). That is, teachers should know students’ knowledge states, and offer corresponding remedial measures.
CD is treated as the core of new generation of psychology and education measurement theory, and it is the production of the combination of cognitive psychology and statistical measurement. It involves some concepts, such as cognitive attribute and state of knowledge. An attribute refers to a cognitive component or sub-process. To be precise, an attribute is a task, subtask, cognitive process, or skill involved in answering an item (Cheng & Chang, 2007). If the number of attributes involved in an exam is
, then a 
-dimension vector, whose element is 0 or 1, could describe the knowledge state of an examinee. Provided that the 
-th element is 1, it shows the examinee grasps the 
-th attribute, and 0, otherwise. Some researchers had thought different examinees grasped the same knowledge points but to different degree, and the difference derived from their different abilities. So DiBello et al. (DiBello, Stout, & Rousso, 1995) pointed out that the foundation of CD is that the performance of examinees is described by abilities and knowledge states. And CD is considered important not only in teaching but also in intelligent tutoring system.
For large-scale testing/examination, it is impossible that feedback for CD is given immediately. Even in class, teachers hardly can make a diagnosis for students on line and present timely response. But diagnosing on line and responding in time is one of the effective ways to improve teaching and learning. At present, many researchers focus on CAT which can estimate examinee’ ability in time but can not make cognitive diagnosis. This kind of CAT is called a traditional CAT. Item response theory (IRT) (Hambleton & Swaminathan, 1985) supports CAT and CAT is a successful application of IRT. For every examinee, an optimal exam would be constructed by CAT which is efficient, rapid, fair and personalized and it is popularized in Occident and Australia. In China, CAT has been launched now. For example, a CAT developed by The Fourth Military Medical University has been applied in the recruitment, and the nice effect is reported. In order to diagnose examinee’s knowledge state, the length of traditional cognitive diagnostic test must be long. The reason is that if an examinee has not grasped a knowledge point, no matter how easy items are, the examinee could not answer them correctly. But if traditional CAT enlarges its function and can make cognitive diagnosis, then CD-CAT can overcome this drawback since CD-CAT would not select the items including the attribute which has been judged by CD-CAT and examinees have not grasped.