Article Preview
TopIntroduction
To diagnose the learning results of a learner to reveal whether each basic concept is well learned or unfamiliar within the testing scope is important feedback for the instructor (Bilder, 2005; Hwang, Panjaburee, Triampo, & Shih, 2013). Based on these results, instructors can adjust their teaching strategies and pay more effort to helping learners to clarify the misconceptions. Moreover, the instructor can then design accurate remedial teaching for the learner. Traditional testing cannot achieve the above objectives, since they only reflect a level value to represent the learning results of a learner for the testing scope (Barrada, Olea, & Ponsoda, 2007; Huang, Li, Hou, & Bi, 2015; Hwang, 2003).
This paper focuses on how to design algorithms for developing web-based adaptive testing and diagnostic system to automatically inspect an examinee's learning results and provide a check list to report whether individual students have learned each of the basic concepts (or attributes) well in the testing scope. Generally, using fewer test items to achieve the above objective can conserve testing time and consequently enhance the reliability of the testing results, which is the main objective of our method design. Besides, controlling the exposure rate of the test items can conserve the expense of new test item development; thus, exposure rate control is also considered in this research.
This study aims to design adequate test item selecting strategies and then to apply them to be the core of an intelligent diagnostic system. The objectives and the contributions of this study are outlined as follows.
- 1.
To diagnose a learner's learning results for each basic concept (well learned/unfamiliar) within a testing scope.
- 2.
To diagnose a learner's learning results using as few test items as possible.
- 3.
To reduce the average test item exposure rate of an item bank.
- 4.
To implement an intelligent diagnostic system using the proposed strategies of this paper.
TopRelevant Research
With the growth of Internet users and the expansion of computer network bandwidth, Web Based Testing (WBT) has received much attention from researchers. Consequently, the testing systems have been much enhanced in many aspects; for instance, the testing contents are not just limited to text but can also include multimedia (audio and video) data, and the testing environment will not be limited to a certain geographic area, but instead the examinee can take the test from anywhere via the Internet. The test item selecting method in the system has become increasingly intelligent, as it can now automatically compose a test sheet for testing, diagnose the examinee's level, or automatically identify the examinees' unfamiliar basic concepts in the testing scope using fewer test items. Thus, these enhanced situations bring many benefits for testing. For example, the aid of intelligent algorithms designed for diagnosing the learning results of an examinee (that is, the mastery profile) with fewer test items used can relieve the situation of the examinee being too tired and so losing patience during the testing, thus enhancing the reliability of the testing results (Hartz, 2002).
Testing theories can be classified into two categories: the Classical Testing Theory (CTT) and the Modern Testing Theory (MTT), according to the evolution of testing theories (Weissman, 2007; Xu, Chang, & Douglas, 2003). The CTT uses a score value to represent the level of learning results with respect to an examinee in the testing scope. By way of testing the examinee with a predetermined test sheet, the score of the answered test sheet can be easily computed. However, the main drawback of the CTT is that two examinees with two equal scores do not necessarily have the same capability in general. The MTT is based on modern Item Response Theory (IRT) so adopts more complicated numerical operations to reflect the examinee's true capability when facing a test (Linden, 2005; Lord, 1980; Lord, & Novick, 1968).