Support Interoperability and Reusability of Emerging Forms of Assessment Using IMS LD and IMS QTI

Introduction

Competence is defined as ‘effective overall performance within an occupation, which may range from the basic level of proficiency through to the highest level of excellence’ (Cheetham & Chivers, 2005). A competence is the ability to handle a complex professional task by integrating the relevant cognitive, psychomotor and affective skills. Information gathering for the assessment of competences is increasingly based on qualitative, descriptive and narrative information, in addition to quantitative, numerical data. Such qualitative information cannot be judged against a simple, pre-defined standard (Vleuten & Schuwirth, 2005). Some emerging forms of assessment have gained in acceptance and popularity in education. Examples of such forms of assessment are self- and peer assessment, accreditation of prior learning, and 360 degree assessment. These forms of assessments address complex traits of students, foster deep learning and the development of competences (Topping, 1998; Boud, Cohen et al., 1999; Gipps, 1999).

Assessment consists of making judgments (task aspect) and carrying out administrative activities (process aspect). In comparison with traditional assessment, both of these aspects of assessment are much more problematic in emerging forms of assessment. In particular, emerging forms of assessment usually involve multiple phases and multiple roles/persons. The difficulties and the potential for errors and omissions increase in a non-linear fashion as the number of candidates and assessors involved grows (Rosbottom, 1994). As Bartram pointed out, 360 degree assessment by its very nature is an administrative nightmare to manage. People involved in the process tend to be geographically dispersed but also need close supervision in order to ensure that the ratings are carried out to schedule and that sufficient raters are obtained for each focus of the assessment (Bartram, 2005).

In order to make emerging forms of assessments work effectively and efficiently, many software tools have been developed and are increasingly being used. For example, MUCH (Rada, Acquah et al., 1993; Rushton, Ramsey et al., 1993), Peers (Ngu, Shepherd et al., 1995), Peer Grader (Gehringer 2001), SPARK (Freeman and McKenzie 2002), and ESpace (Volder et al., 2007) are multi-user tools that support self- or/and peer-assessment. The eSPRAT system (Lockyer, 2003; Davies & Archer, 2005) and Appraisal360 (Appraisal360 home page) are example tools that support 360 degree assessment. In self- and peer assessment, with the help of software tools, the tutor, freed from administrative chores, is able to provide a useful, added-value service to students by acting as a problem solver. Student-assessors can concentrate on the clarity, correctness and completeness of each individual exercise without worrying about the relationship with other exercises (Rosbottom, 1994). Similarly, for supporting 360 degree assessment, the software tools manage the workflow associated with the 360 degree assessment process, from initial set-up and preparation of the people involved, through the management of the rating process (including delivery and scoring of questionnaires), to the production of reports and their delivery to feedback providers (Bartram, 2005).