Detection of Misconceptions and Misleading Questions by Using Quantitative Diagnostic Assessment

Detection of Misconceptions and Misleading Questions by Using Quantitative Diagnostic Assessment

Martin M. Weng, Wen-Chih Chang, Neil Y. Yen, Timothy K. Shih, Hui-Huang Hsu
Copyright: © 2014 |Pages: 25
DOI: 10.4018/ijdet.2014040102
(Individual Articles)
No Current Special Offers


Researches into E-Learning pay emphasis on how the technology is applied for comprehensive learning support in various perspectives. Lots of advantages are involved while performing the E-Learning. For example, timely contents are applicable to be delivered via the Internet, and in a similar manner, instant feedback which identifies the learning performance would be obtained through an online assessment system. In this article, a diagnostic approach for learning performance evaluation of student is proposed. Two primary issues are concentrated based on the analysis of quantitative assessment results. First the authors identify the misconceptions on specific subject(s) that students have learned and provide remedial suggestions about their learning status to students. Second, because questions may have several incorrect information to students, hence the authors approach provide suggested revisions to instructors. The proposed approach employs the Rule-Space model and the Student-Problem chart and is implemented in an online assessment system to obtain corresponding data (e.g., learning behavior) and performance (e.g., learning status). The feasibility is demonstrated through an empirical study (i.e., a series of pre-post tests) which was conducted in a domain-specific course (i.e., Fundamental of Information Technology) with around 450 students in a calendar year.
Article Preview


Owing to popularity of E-Learning, materials and channels for group communications are applicable to be retrieved and accessed over the Internet. From perspective of education, instruction and assessment are considered as primary elements in the lifecycle of learning in a traditional learning environment or an E-Learning environment. The instruction briefly refers to the delivery of concepts in specific course(s), and a well-designed assessment focuses more on the obtainment of learning performance, or learning feedback after learning process. In general, assessment can be primarily categorized into following four items: formative assessment, readiness assessment, summative assessment and diagnostic assessment. The formative assessment makes continuous judge on students’ progress. The readiness assessment determines whether students have essential knowledge, skill and ability to learn. The diagnostic assessment checks prerequisites (e.g., knowledge and concept) acquired by students who are willing to continue the learning activity. The summative assessment provides the review of assessment results regarding to pre-post test.

The pre-post test is widely applied to determine the learning performance after learning activity and provide students corresponding remediation in accordance with the learning analysis results. However, in general cases, the results in a posttest, comparing with a pretest, cannot reveal the concrete performance since some key concepts might be missed by instructors while designing the assessment.

As a solution, the diagnostic assessment is often applied to identify the learning knowledge (or learning concept) of a subject that required to be improved. Instructors and students benefit from the diagnostic results. For instance, instructors can adopt appropriate learning strategy to students and provide learning content to students adaptively, and students can improve the insufficiency in previous learning activities. In this situation, it becomes easier for instructors to provide remediation to students in accordance with the diagnostic results of assessment than traditional assessment process. That is, the diagnostic assessment has brought an open solution to improve the learning performance where assessment is often used.

As to cognitive development of students, it is more implicit to be measured from students and is typically determined through the scores of assessment. The scores reveal an unobvious indication of result composed of correct cognition and misconception which could be determined through approaches (Menucha, Anthony, & Tatsuoka 1992) based on Rule-Space model. After diagnose, the remediation would be provided. However, the cost of time might increase to apply the Rule-Space model into assessment analysis since complicated assessment items are required to be described in the beginning.

Researches (Drasgow, 1982; Harnisch & Linn., 1981) consider the correlations between student, course and assessment items (Student-Problem chart) to detect misconception caused by abnormal behavior of students while performing assessment. However, the Student-Problem chart can only find individual students instead of what problems may occur (Sato, 1985).

To achieve assessment support, the proposed approach allows instructors to revise the teaching strategies, redesign the courses and organize the learning material. In addition, the key conceptions of courses can easily be located by students. As a concrete contribution, the proposed approach enhance the diagnostic indices based on the integration of Student-Problem chart and Rule-Space model. The approach allows the discovery of students’ misconceptions and provides immediate remedial suggestions. Furthermore, the approach finds out the misleading questions and provides suggested revision to instructors.

The rest of this paper is organized as follows. Section 2 introduces related technique issues. Section 3 addresses the details of the proposed approach. Section 4 represents the empirical study, including an experiment and an example, based on implemented approach. Section 5 compares the proposed approach with related approaches. We conclude this work in section 6.

Complete Article List

Search this Journal:
Volume 22: 1 Issue (2024): Forthcoming, Available for Pre-Order
Volume 21: 2 Issues (2023)
Volume 20: 4 Issues (2022): 1 Released, 3 Forthcoming
Volume 19: 4 Issues (2021)
Volume 18: 4 Issues (2020)
Volume 17: 4 Issues (2019)
Volume 16: 4 Issues (2018)
Volume 15: 4 Issues (2017)
Volume 14: 4 Issues (2016)
Volume 13: 4 Issues (2015)
Volume 12: 4 Issues (2014)
Volume 11: 4 Issues (2013)
Volume 10: 4 Issues (2012)
Volume 9: 4 Issues (2011)
Volume 8: 4 Issues (2010)
Volume 7: 4 Issues (2009)
Volume 6: 4 Issues (2008)
Volume 5: 4 Issues (2007)
Volume 4: 4 Issues (2006)
Volume 3: 4 Issues (2005)
Volume 2: 4 Issues (2004)
Volume 1: 4 Issues (2003)
View Complete Journal Contents Listing