Support Interoperability and Reusability of Emerging Forms of Assessment Using IMS LD and IMS QTI

Support Interoperability and Reusability of Emerging Forms of Assessment Using IMS LD and IMS QTI

Yongwu Miao (Open University of The Netherlands, The Netherlands), Jo Boon (Open University of The Netherlands, The Netherlands), Marcel van der Klink (Open University of The Netherlands, The Netherlands), Peter Sloep (Open University of the Netherlands, The Netherlands) and Rob Koper (Open University of the Netherlands, The Netherlands)
DOI: 10.4018/978-1-61692-789-9.ch001
OnDemand PDF Download:
$30.00
List Price: $37.50

Abstract

Emerging forms of assessment (e.g., self-/peer assessment and 360 degree assessment) involve multiple phases and multiple roles/persons, which are process-oriented assessment. IMS Question and Test Interoperability (QTI) is an open technical specification for task-oriented assessment, which has insufficient expressiveness to specify emerging forms of assessment. Meanwhile, existing software tools supporting emerging forms of assessment lack interoperability and reusability. In this chapter, the authors claim that a combined use of QTI and IMS Learning Design (LD) is able to support interoperability and reusability of emerging forms of assessment. In order to support this claim, they analyze the characteristics of four emerging forms of assessment from the perspective of process technologies and present the method to specify emerging assessment forms using QTI and LD. Furthermore, the authors present the difficulties and problems that they encountered when modeling emerging assessment forms and propose possible solutions to solve the problems.
Chapter Preview
Top

Introduction

Competence is defined as ‘effective overall performance within an occupation, which may range from the basic level of proficiency through to the highest level of excellence’ (Cheetham & Chivers, 2005). A competence is the ability to handle a complex professional task by integrating the relevant cognitive, psychomotor and affective skills. Information gathering for the assessment of competences is increasingly based on qualitative, descriptive and narrative information, in addition to quantitative, numerical data. Such qualitative information cannot be judged against a simple, pre-defined standard (Vleuten & Schuwirth, 2005). Some emerging forms of assessment have gained in acceptance and popularity in education. Examples of such forms of assessment are self- and peer assessment, accreditation of prior learning, and 360 degree assessment. These forms of assessments address complex traits of students, foster deep learning and the development of competences (Topping, 1998; Boud, Cohen et al., 1999; Gipps, 1999).

Assessment consists of making judgments (task aspect) and carrying out administrative activities (process aspect). In comparison with traditional assessment, both of these aspects of assessment are much more problematic in emerging forms of assessment. In particular, emerging forms of assessment usually involve multiple phases and multiple roles/persons. The difficulties and the potential for errors and omissions increase in a non-linear fashion as the number of candidates and assessors involved grows (Rosbottom, 1994). As Bartram pointed out, 360 degree assessment by its very nature is an administrative nightmare to manage. People involved in the process tend to be geographically dispersed but also need close supervision in order to ensure that the ratings are carried out to schedule and that sufficient raters are obtained for each focus of the assessment (Bartram, 2005).

In order to make emerging forms of assessments work effectively and efficiently, many software tools have been developed and are increasingly being used. For example, MUCH (Rada, Acquah et al., 1993; Rushton, Ramsey et al., 1993), Peers (Ngu, Shepherd et al., 1995), Peer Grader (Gehringer 2001), SPARK (Freeman and McKenzie 2002), and ESpace (Volder et al., 2007) are multi-user tools that support self- or/and peer-assessment. The eSPRAT system (Lockyer, 2003; Davies & Archer, 2005) and Appraisal360 (Appraisal360 home page) are example tools that support 360 degree assessment. In self- and peer assessment, with the help of software tools, the tutor, freed from administrative chores, is able to provide a useful, added-value service to students by acting as a problem solver. Student-assessors can concentrate on the clarity, correctness and completeness of each individual exercise without worrying about the relationship with other exercises (Rosbottom, 1994). Similarly, for supporting 360 degree assessment, the software tools manage the workflow associated with the 360 degree assessment process, from initial set-up and preparation of the people involved, through the management of the rating process (including delivery and scoring of questionnaires), to the production of reports and their delivery to feedback providers (Bartram, 2005).

Key Terms in this Chapter

360 Degree Assessment: An assessment method where performance appraisals are collected from different sources.

Accreditation of Prior Learning: The assessment and recognition of one’s competences developed through work experiences.

Interoperability: The ability to take instructional components (e.g., an activity or a question) developed in one system and to use them in another system without special effort.

Reusability: The flexibility to incorporate instructional components (e.g., an activity or a question) in multiple applications and contexts.

IMS QTI: An open e-learning technical specification that describes a data model for representing assessment as a hierarchical structure and processing responses.

Self Assessment: A method where an individual assesses his or her own performance regarding a specific topic.

IMS LD: An open e-learning technical specification that provides a pedagogy-neutral modeling language for specifying a teaching/learning process and associated learning resources.

Peer Assessment: The assessment of a student’s performance by his/her fellow students.

Complete Chapter List

Search this Book:
Reset