Adaptive Computer Assisted Assessment

Adaptive Computer Assisted Assessment

Diana Pérez-Marín (Universidad Autónoma de Madrid and Universidad Rey Juan Carlos, Spain), Ismael Pascual-Nieto (Universidad Autónoma de Madrid, Spain) and Pilar Rodríguez (Universidad Autónoma de Madrid, Spain)
DOI: 10.4018/978-1-60566-380-7.ch010
OnDemand PDF Download:
$30.00
List Price: $37.50

Abstract

This chapter introduces the reader in the fields of automatic assessment of free-text students’ answers, student modeling and adaptive educational hypermedia. Traditionally, these fields have been studied separately missing the benefits of their synergic combination (i.e., free-text scoring systems which do not keep any student model, and adaptive educational hypermedia systems which do not use any natural language processing technique). In particular, a procedure to automatically generate students’ conceptual models from their answers to a free-text adaptive computer assisted assessment system will be fully described, together with its implementation in the will tools. Furthermore, the authors will explore how useful this new possibility of hybrid learning is both for teachers and students in two case studies carried out during the 2006-2007 and 2007-2008 academic years, in which traditional lessons were combined with the use of the Will Tools both in technical and non-technical domains.
Chapter Preview
Top

Introduction

E-learning can be defined as the use of Information and Communication Technologies (ICT) to improve the quality of learning by facilitating access to resources and services as well as remote exchanges and collaboration (Page, 2006). In the last decades, given the increasing acceptance and use of ICTs by the population (Hopkins, 1998), there has been a great deal of research devoted to E-learning, and in particular to web-based learning.

The main benefits of web-based learning are well known: students having more control over their learning process, being able to study from any computer connected to Internet at anytime; and, in some cases, even receiving the contents and navigation of the course adapted to their particular features such as with Intelligent Tutoring Systems (Shute & Torreano, 2002), or Adaptive Educational Hypermedia systems (Brusilovsky, 2004).

On the other hand, web-based learning has also presented some problems of sociological and legal nature. For instance, Chung & O’Neill (1997) discussed about the negative effects of losing the student-teacher relationship. Furthermore, Ford (2000) warned that scores achieved with electronic media instead of traditional exams might not be legally defensible.

Therefore, Blended Learning (Graham, 2006) or Hybrid Learning has recently appeared to combine the traditional teaching methods with the application of ICTs for education. That way, it is possible to take advantage of the benefits of e-learning without bearing its disadvantages.

A possible scenario of Hybrid Learning could be as follows, a teacher who has 200 university students enrolled in a course. The teacher would like to give personalized tuition to each student. Moreover, s/he would like to be able to solve collaboratively exercises in class, and set partial exams. However, the teacher is overwhelmed by the number of students and tasks s/he has to accomplish. Therefore, s/he decides to impart lectures, ask the students to use an automatic evaluation system to get more training, and set a final traditional exam.

Furthermore, the students can ask the teacher the doubts they have not been able to solve with the automatic system. That way, students can practise more and receive a personalized tuition. At the same time, the teacher-student relationship is not lost. On the contrary, students ask more doubts to the human teacher as they have been trying to solve the exercises on their own, or collaboratively, with the computer after class.

Computer Assisted Assessment (CAA) is the field that studies how computers can effectively be used to assess students’ learning progress (Knowles, 1999). This field has also received a long-lasting attention because assessment is essential to learn (Dewey, 1933; Berry, 2003). Originally, CAA was limited just to Multiple Choice Questions (MCQs) and fill-in-the-blank exercises (the so-called objective testing). The reason can be found in the easiness of implementation of objective testing. It is only necessary to provide the computer with the correct item answer, and the evaluation will consist of checking whether the student option matches the one previously stored.

The general CAA community opinion is that only evaluating the students’ learning progress with MCQs and fill-in-the-blank exercises is not enough to measure the higher cognitive skills (Birenbaum, Tatsuoka & Gutvirtz, 1992; Sigel, 1999; Mitchell, Aldridge, Williamson & Broomhead, 2003). Therefore, a shift from the “evaluation” culture to the really “assessment” culture was done and, new kinds of assessments were developed such as the assessment of free-text answers. The subfield of free-text CAA was created to focus on how to automatically assess free-text students’ answers. Free-text CAA has been able to progress by using several Natural Language Processing (NLP) resources, techniques and tools.

Key Terms in this Chapter

Summative Assessment: A type of assessment in which the goal is to evaluate the students by giving them a score, and finding out whether they have enough level to pass the subject or not. For example, by testing the students with an exam.

Conceptual Diagram: A knowledge presentation format to visualize conceptual knowledge as hierarchical diagrams in which each cell is a concept of a certain area-of-knowledge.

Free-Text Adaptive Computer Assisted Assessment: It is a subfield of CAA in which the assessment is automatic and adaptive. That is, the computer system asks open-ended questions to the answers, automatically evaluates the students’ free-text answers, and provides adaptive feedback to the student. Furthermore, the free-text ACAA system keeps a student model to adapt the type and order of the questions to each particular student.

Formative Assessment: A type of assessment in which the goal is not to score the students but to support them. For instance, by giving them detailed feedback. That way, students could progressively improve their understanding of the lesson.

Adaptive Educational Hypermedia (AEH): It is a subfield of Adaptive Hypermedia (i.e. the combination of hypermedia-based techniques with adaptive and user-model-based interfaces) whose application is on the educational field.

Concept Map: One powerful knowledge presentation format, devised by Novak, to visualize conceptual knowledge as graphs in which the nodes represent the concepts, and the links between the nodes are the relationships between these concepts.

E-Assessment or Computer Assisted Assessment (CAA): It is the research field that studies how computers can effectively be used to assess students’ knowledge.

Open Learner Modeling: Show the model kept by an educational system such as an AEH or a free-text ACAA system to the students so that they can reflect on it.

Natural Language Processing (NLP): It is a subfield of Computational Linguistics (i.e. the field that researches linguistics phenomena that occur in digital data), whose focus is on how to build automatic systems able to interpret/generate information in natural language.

Student’s Conceptual Model: Simplified representation of the concepts and relationships among them that each student keeps in his or her mind about an area-of-knowledge at a certain instant.

Complete Chapter List

Search this Book:
Reset