A Test Sheet Optimization Approach to Supporting Web-based Learning Diagnosis Using Group Testing Methods

A Test Sheet Optimization Approach to Supporting Web-based Learning Diagnosis Using Group Testing Methods

Chu-Fu Wang (National Pingtung University, Department of Computer Science, Pingtung, Taiwan), Chih-Lung Lin (National Pingtung University, Department of Computer Science, Pingtung, Taiwan), Gwo-Jen Hwang (National Taiwan University of Science and Technology, Graduate Institute of Digital Learning and Education, Taipei City, Taiwan), Sheng-Pin Kung (National Pingtung University, Department of Computer Science, Pingtung, Taiwan) and Shin-Feng Chen (National Pingtung University, Department of Education, Pingtung, Taiwan)
Copyright: © 2017 |Pages: 23
DOI: 10.4018/IJOPCD.2017100101
OnDemand PDF Download:
$37.50

Abstract

Assessment can help teachers to examine the effectiveness of teaching and to diagnose the unfamiliar basic concepts (or attributes) of students within the testing scope. A web-based adaptive testing and diagnostic system can achieve the above objective efficiently and correctly. From a diagnostic point of view, the major concerns are to diagnose whether or not an examinee has learned each basic concept well in the testing scope, while also limiting the number of test items used (the testing length) to as few as possible, which will be directly related to the patience of the examinee. In this paper, we consider a test item selecting optimization diagnostic problem to reveal the mastery profile of an examinee (that is, to diagnose each basic concept's learning status (well learned/unfamiliar) in the testing scope) with a short testing length and a limited test item exposure rate. This paper uses the techniques of Group Testing theory for the design of our test item selecting algorithm. Two test item selecting strategies, the bisecting method and the doubling method, are proposed. The effectiveness of the proposed methods was evaluated by experimental simulations. The results show that both of the proposed algorithms use fewer test items and a limited test item exposure rate compared to the conventional methods.
Article Preview

Introduction

To diagnose the learning results of a learner to reveal whether each basic concept is well learned or unfamiliar within the testing scope is important feedback for the instructor (Bilder, 2005; Hwang, Panjaburee, Triampo, & Shih, 2013). Based on these results, instructors can adjust their teaching strategies and pay more effort to helping learners to clarify the misconceptions. Moreover, the instructor can then design accurate remedial teaching for the learner. Traditional testing cannot achieve the above objectives, since they only reflect a level value to represent the learning results of a learner for the testing scope (Barrada, Olea, & Ponsoda, 2007; Huang, Li, Hou, & Bi, 2015; Hwang, 2003).

This paper focuses on how to design algorithms for developing web-based adaptive testing and diagnostic system to automatically inspect an examinee's learning results and provide a check list to report whether individual students have learned each of the basic concepts (or attributes) well in the testing scope. Generally, using fewer test items to achieve the above objective can conserve testing time and consequently enhance the reliability of the testing results, which is the main objective of our method design. Besides, controlling the exposure rate of the test items can conserve the expense of new test item development; thus, exposure rate control is also considered in this research.

This study aims to design adequate test item selecting strategies and then to apply them to be the core of an intelligent diagnostic system. The objectives and the contributions of this study are outlined as follows.

  • 1.

    To diagnose a learner's learning results for each basic concept (well learned/unfamiliar) within a testing scope.

  • 2.

    To diagnose a learner's learning results using as few test items as possible.

  • 3.

    To reduce the average test item exposure rate of an item bank.

  • 4.

    To implement an intelligent diagnostic system using the proposed strategies of this paper.

Relevant Research

With the growth of Internet users and the expansion of computer network bandwidth, Web Based Testing (WBT) has received much attention from researchers. Consequently, the testing systems have been much enhanced in many aspects; for instance, the testing contents are not just limited to text but can also include multimedia (audio and video) data, and the testing environment will not be limited to a certain geographic area, but instead the examinee can take the test from anywhere via the Internet. The test item selecting method in the system has become increasingly intelligent, as it can now automatically compose a test sheet for testing, diagnose the examinee's level, or automatically identify the examinees' unfamiliar basic concepts in the testing scope using fewer test items. Thus, these enhanced situations bring many benefits for testing. For example, the aid of intelligent algorithms designed for diagnosing the learning results of an examinee (that is, the mastery profile) with fewer test items used can relieve the situation of the examinee being too tired and so losing patience during the testing, thus enhancing the reliability of the testing results (Hartz, 2002).

Testing theories can be classified into two categories: the Classical Testing Theory (CTT) and the Modern Testing Theory (MTT), according to the evolution of testing theories (Weissman, 2007; Xu, Chang, & Douglas, 2003). The CTT uses a score value to represent the level of learning results with respect to an examinee in the testing scope. By way of testing the examinee with a predetermined test sheet, the score of the answered test sheet can be easily computed. However, the main drawback of the CTT is that two examinees with two equal scores do not necessarily have the same capability in general. The MTT is based on modern Item Response Theory (IRT) so adopts more complicated numerical operations to reflect the examinee's true capability when facing a test (Linden, 2005; Lord, 1980; Lord, & Novick, 1968).

Complete Article List

Search this Journal:
Reset
Open Access Articles: Forthcoming
Volume 7: 4 Issues (2017)
Volume 6: 4 Issues (2016)
Volume 5: 4 Issues (2015)
Volume 4: 4 Issues (2014)
Volume 3: 4 Issues (2013)
Volume 2: 4 Issues (2012)
Volume 1: 4 Issues (2011)
View Complete Journal Contents Listing