Enhancing the IMS QTI to Better Support Computer Assisted Marking

Enhancing the IMS QTI to Better Support Computer Assisted Marking

Damien Clark (Central Queensland University, Australia) and Penny Baillie-de Byl (University of Southern Queensland, Australia)
DOI: 10.4018/978-1-60566-342-5.ch014
OnDemand PDF Download:


Computer aided assessment is a common approach used by educational institutions. The benefits range into the design of teaching, learning, and instructional materials. While some such systems implement fully automated marking for multiple choice questions and fill-in-the-blanks, they are insufficient when human critiquing is required. Current systems developed in isolation have little regard to scalability and interoperability between courses, computer platforms, and learning management systems. The IMS Global Learning Consortium’s open specifications for interoperable learning technology lack functionality to make it useful for computer assisted marking. This article presents an enhanced set of these standards to address the issue.
Chapter Preview


Computer aided assessment (CAA), one of the recent trends in education technology, has become common-place in educational institutions as part of delivering course materials, particularly for large classes. This has been driven by many factors, such as:

  • The need to reduce educational staff workloads (Dalziel, 2000; Jacobsen & Kremer, 2000; Jefferies, Constable et al., 2000; Pain & Heron, 2003; Peat, Franklin et al., 2001);

  • A push for more timely feedback to students (Dalziel, 2001; Jefferies, Constable et al., 2000; Merat & Chung, 1997; Sheard & Carbone, 2000; Woit & Mason, 2000);

  • Reduction in educational material development and delivery costs (Jefferies, Constable et al., 2000; Muldner & Currie, 1999); and,

  • The proliferation of online education (White, 2000).

Internet-based technologies in CAA can be broadly categorised into the following system types: online quiz systems, fully automated marking, and semiautomated/computer assisted marking systems. The most common form of CAA, online quizzes, typically consist of multiple choice questions (MCQ) (IMS, 2000), as they can be automatically marked. Yet, there is much conjecture on the effectiveness of MCQs, particularly in the assessment of Bloom’s higher learning outcomes (1956) such as analysis, synthesis, and evaluation (Davies, 2001). This limits the scope by which a student’s abilities can be assessed. Short response and essay type questions are commonly used to assess the higher order skills of Bloom’s taxonomy. Still, these types of assessments are time consuming to mark manually (Davies, 2001; White, 2000).

A more ambitious approach to CAA involves the use of fully-automated marking systems. These can be defined as systems that can mark electronically submitted assignments such as essays (Palmer, Williams et al., 2002) via online assignment submission management (OASM) (Benford, Burke et al., 1994; Darbyshire, 2000; Gayo, Gil et al., 2003; Huizinga, 2001; Jones & Behrens, 2003; Jones & Jamieson, 1997; Mason & Woit, 1999; Roantree & Keyes, 1998; Thomas, 2000; Trivedi, Kar et al., 2003), and automatically generate a final grade for the assignment with little to no interaction with a human marker. The obvious benefit to this approach is the ability to assess some higher order thinking as per Bloom’s Taxonomy (1956) in a completely automated manner, thus improving marking turn-around times for large classes. Fully automated systems include MEAGER, which is designed to automatically mark Microsoft Excel spreadsheets (Hill, 2003), automatic essay marking systems, such as those evaluated by Palmer, Williams et al. (2002), and English and Siviter’s system (2000) designed to assess student hypertext mark-up language (HTML) Web pages, to name a few. Unfortunately, this approach is not suitable for all assessment types and can often require significant time to develop the model solution. In addition, most of the automated functionality examines students’ solutions against model solutions. This may lead to issues relating to marking quality when it is impossible for the assessment creator to identify all possible solutions.

Complete Chapter List

Search this Book:
Associate Editors
Table of Contents
Mahbubur Rahman Syed
Mahbubur Rahman Syed
Chapter 1
Hiroshi Takeda, Hisashi Yaginuma, Hajime Kiyohara, Akira Tokuyasu, Masami Iwatsuki, Norio Takeuchi, Hisato Kobayashi, Kazuo Yana
This article describes a new automatic digital content generation system we have developed. Recently some universities, including Hosei University... Sample PDF
Automatic Digital Content Generation System for Real-Time Distance Lectures
Chapter 2
Filomena Ferrucci, Giuseppe Scanniello, Genoveffa Tortora
In this chapter the authors present E-World, an e-learning platform able to manage and trace adaptive learning processes which are designed and... Sample PDF
E-World: A Platform for the Management of Adaptive E-Learning Processes
Chapter 3
Judy C.R. Tseng, Wen-Ling Tsai, Gwo-Jen Hwang, Po-Han Wu
In developing traditional learning materials, quality is the key issue to be considered. However, for high technical e-training courses, not only... Sample PDF
An Efficient and Effective Approach to Developing Engineering E-Training Courses
Chapter 4
Te-Hua Wang, Flora Chia-I Chang
The sharable content object reference model (SCORM) includes a representation of distance learning contents and a behavior definition of how users... Sample PDF
A SCORM Compliant Courseware Authoring Tool for Supporting Pervasive Learning
Chapter 5
WenYing Guo
Selecting appropriate learning services for a learner from a large number of heterogeneous knowledge sources is a complex and challenging task. This... Sample PDF
An Ontology-Based e-Learning Scenario
Chapter 6
Dan Phung, Giuseppe Valetto, Gail E. Kaiser, Tiecheng Liu, John R. Kender
The increasing popularity of online courses has highlighted the need for collaborative learning tools for student groups. In this article, we... Sample PDF
Adaptive Synchronization of Semantically Compressed Instructional Videos for Collaborative Distance Learning
Chapter 7
Jing Chen, Qing Li, Ling Feng
The abundance of knowledge-rich information on the World Wide Web makes compiling an online etextbook both possible and necessary. In our previous... Sample PDF
Refining the Results of Automatic e-Textbook Construction by Clustering
Chapter 8
Yueting Zhuang, Xiafen Zhang, Weiming Lu, Fei Wu
Chinese brush calligraphy is a valuable civilization legacy and a high art of scholarship. It is still popular in Chinese banners, newspaper... Sample PDF
Chinese Brush Calligraphy Character Retrieval and Learning
Chapter 9
William K. Cheung, Anders I. Mørch, Kelvin C. Wong, Cynthia Lee, Jiming Liu, Mason H. Lam
In this article we investigate the use of latent semantic analysis (LSA), critiquing systems, and knowledge building to support computer-based... Sample PDF
Grounding Collaborative Learning in Semantics-Based Critiquing
Chapter 10
Giuliana Dettori, Paola Forcheri, Maria Grazia Ierardi
Learning Objects (LOs) are increasingly considered potentially helpful to improve teachers’ work and to spread innovation in the school system.... Sample PDF
Improving the Usefulness of Learning Objects by Means of Pedagogy-Oriented Design
Chapter 11
Frederick W.B. Li, Rynson W.H. Lau, Taku Komura, Meng Wang, Becky Siu
Human motion animation has been one of the major research topics in the field of computer graphics for decades. Techniques developed in this area... Sample PDF
Adaptive Animation of Human Motion for E-Learning Applications
Chapter 12
Gennaro Costagliola, Vittorio Fuccella
On-Line Testing is that sector of e-learning aimed at assessing learner’s knowledge through e-learning means. In on-line testing, due to the... Sample PDF
eWorkbook: An On-Line Testing System with Test Visualization Functionalities
Chapter 13
Brian Stewart, Derek Briton, Mike Gismondi, Bob Heller, Dietmar Kennepohl, Rory McGreal, Christine Nelson
Athabasca University—Canada’s Open University evaluated learning management systems (LMS) for use by the university. Evaluative criteria were... Sample PDF
Choosing MOODLE: An Evaluation of Learning Management Systems at Athabasca
Chapter 14
Damien Clark, Penny Baillie-de Byl
Computer aided assessment is a common approach used by educational institutions. The benefits range into the design of teaching, learning, and... Sample PDF
Enhancing the IMS QTI to Better Support Computer Assisted Marking
Chapter 15
Ali Dashti, Maytham Safar
Distance education created new challenges regarding the delivery of large size isochronous continuous streaming media (SM) objects. In this paper... Sample PDF
Streaming of Continuous Media for Distance Education Systems
Chapter 16
Manjulika Srivastava, Venugopal Reddy
The question why some learners successfully study through distance mode and others do not is increasingly becoming important as open and distance... Sample PDF
How Did They Study at a Distance? Experiences of IGNOU Graduates
Chapter 17
Gwo-Jen Hwang, Ting-Ting Wu, Yen-Jung Chen
The prosperous development of wireless communication and sensor technologies has attracted the attention of researchers from both computer and... Sample PDF
Ubiquitous Computing Technologies in Education
Chapter 18
S. Grunwald, B. Hoover, G.L. Bruland
In this chapter the authors describe the implementation of an emerging virtual learning environment to teach GIS and spatial sciences to distance... Sample PDF
An eLearning Portal to Teach Geographic Information Sciences
Chapter 19
Maria Manuela Cunha, Goran D. Putnik
Individualised open and distance learning at the university continuing education and post-graduate education levels is a central issue of today. The... Sample PDF
A Changed Economy with Unchanged Universities? A Contribution to the University of the Future
Chapter 20
Richard Y.D. Xu, Jesse S. Jin
This article presents a schematic application of computer vision technologies to e-learning that is synchronous, peer-to-peer-based, and supports an... Sample PDF
Rationale, Design and Implementation of a Computer Vision-Based Interactive E-Learning System
Chapter 21
Dorothée Rasseneur-Coffinet, Georgia Smyrniou, Pierre Tchounikine
This article presents an approach and tools that can help learners appropriate a Web-based learning curriculum and become active participants in... Sample PDF
Supporting Learners' Appropriation of a Web-Based Learning Curriculum
Chapter 22
Gwo-Jen Hwang, Hsiang Cheng, Carol H.C. Chu, Judy C.R. Tseng, Gwo-Haur Hwang
In the past decades, English learning has received lots of attention all over the world, especially for those who are not native English speakers.... Sample PDF
Development of a Web-Based System for Diagnosing Student Learning Problems on English Tenses
Chapter 23
Chi-Syan Lin, C. Candace Chou, Ming-Shiou Kuo
The paper outlines a new paradigm and its underlying rationales for implementing networked learning environments that is emerging from new... Sample PDF
Inhabited Virtual Learning Worlds and Impacts on Learning Behaviors in Young School Learners
Chapter 24
Rory McGreal, Terry Anderson
Any view of e-learning in Canada must be informed by the uniquely Canadian feature of provincial jurisdiction over education. Therefore any... Sample PDF
Research and Practice of E-Learning in Canada 2008
About the Contributors