Adaptive Computer Assisted Assessment

Introduction

E-learning can be defined as the use of Information and Communication Technologies (ICT) to improve the quality of learning by facilitating access to resources and services as well as remote exchanges and collaboration (Page, 2006). In the last decades, given the increasing acceptance and use of ICTs by the population (Hopkins, 1998), there has been a great deal of research devoted to E-learning, and in particular to web-based learning.

The main benefits of web-based learning are well known: students having more control over their learning process, being able to study from any computer connected to Internet at anytime; and, in some cases, even receiving the contents and navigation of the course adapted to their particular features such as with Intelligent Tutoring Systems (Shute & Torreano, 2002), or Adaptive Educational Hypermedia systems (Brusilovsky, 2004).

On the other hand, web-based learning has also presented some problems of sociological and legal nature. For instance, Chung & O’Neill (1997) discussed about the negative effects of losing the student-teacher relationship. Furthermore, Ford (2000) warned that scores achieved with electronic media instead of traditional exams might not be legally defensible.

Therefore, Blended Learning (Graham, 2006) or Hybrid Learning has recently appeared to combine the traditional teaching methods with the application of ICTs for education. That way, it is possible to take advantage of the benefits of e-learning without bearing its disadvantages.

A possible scenario of Hybrid Learning could be as follows, a teacher who has 200 university students enrolled in a course. The teacher would like to give personalized tuition to each student. Moreover, s/he would like to be able to solve collaboratively exercises in class, and set partial exams. However, the teacher is overwhelmed by the number of students and tasks s/he has to accomplish. Therefore, s/he decides to impart lectures, ask the students to use an automatic evaluation system to get more training, and set a final traditional exam.

Furthermore, the students can ask the teacher the doubts they have not been able to solve with the automatic system. That way, students can practise more and receive a personalized tuition. At the same time, the teacher-student relationship is not lost. On the contrary, students ask more doubts to the human teacher as they have been trying to solve the exercises on their own, or collaboratively, with the computer after class.

Computer Assisted Assessment (CAA) is the field that studies how computers can effectively be used to assess students’ learning progress (Knowles, 1999). This field has also received a long-lasting attention because assessment is essential to learn (Dewey, 1933; Berry, 2003). Originally, CAA was limited just to Multiple Choice Questions (MCQs) and fill-in-the-blank exercises (the so-called objective testing). The reason can be found in the easiness of implementation of objective testing. It is only necessary to provide the computer with the correct item answer, and the evaluation will consist of checking whether the student option matches the one previously stored.

The general CAA community opinion is that only evaluating the students’ learning progress with MCQs and fill-in-the-blank exercises is not enough to measure the higher cognitive skills (Birenbaum, Tatsuoka & Gutvirtz, 1992; Sigel, 1999; Mitchell, Aldridge, Williamson & Broomhead, 2003). Therefore, a shift from the “evaluation” culture to the really “assessment” culture was done and, new kinds of assessments were developed such as the assessment of free-text answers. The subfield of free-text CAA was created to focus on how to automatically assess free-text students’ answers. Free-text CAA has been able to progress by using several Natural Language Processing (NLP) resources, techniques and tools.