Analysis and Validation of Learning Technology Models, Standards and Specifications: The Reference Model Analysis Grid (RMAG)

Analysis and Validation of Learning Technology Models, Standards and Specifications: The Reference Model Analysis Grid (RMAG)

Jan M. Pawlowski (University of Jyväskylä, Finland) and Denis Kozlov (University of Jyväskylä, Finland)
DOI: 10.4018/978-1-4666-2160-2.ch013
OnDemand PDF Download:


The paper presents a model for the analysis, comparison and validation of standards, specifications and in particular reference models in the field of Technology Enhanced Learning (TEL). The Reference Model Analysis Grid (RMAG) establishes categories of reference models and standards. Based on those categories, a set of criteria for the analysis and validation of standards was elaborated as a part of the ICOPER project that aims at interoperable open content for competency-based TEL. The analysis of standards in this context is targeted at developing a set of validated approaches that lead to a new reference model. Four standards were investigated, taking into account a broad range of aspects like practical and semantic interoperability and integration issues. In the case study, the authors analyzed both, the standards and specifications and the usefulness of the RMAG. The results of this case study can be used for further analyses of TEL standards as well as for reference models targeted at interoperability.
Chapter Preview

1. Introduction

A variety of models, standards and specifications is used in the field of Technology-Enhanced Learning (TEL). Many of those follow the idea of reference modeling, i.e., providing a base model or specification for solving a design or development problem. Most reference models aim at achieving interoperability between different systems and platforms. However, these models have different underlying theoretical foundations, different scopes, methodologies, and implementations. Hence, users of the reference models cannot clearly decide whether it is suitable for their purpose and context. Therefore, the Reference Model Analysis Grid (RMAG) has addressed this issue.

The main research question addressed in this paper is: How can current learning technology reference models, standards and specifications be assessed and how can they be combined to provide meaningful solutions to content developers, educators and users? The main goal is to develop an analysis scheme aiming at assessing reference models as well as standards and specifications built on existing practice. Our approach is based on an extensive literature research in the field – for the construction of the reference model, we follow the Design Science Research methodology (Hevner et al., 2004), developing an artifact – the analysis grid – and evaluating those against the key objectives.

A reference model represents for example generic processes, systems, and data as well as actors of a specific domain aiming at supporting developers by adapting this model to a specific context. It is a conceptual framework that can be used as a blueprint for systems development. Reference models are also called universal models, generic models or model patterns (Fettke & Loos, 2003a; Fettke & Loos, 2003b; Fettke & Loos, 2006). Concrete examples of reference models are SAP’s reference model (Keller & Teufel, 1998), Hay’s data model patterns (Hay, 1998) or Scheer’s reference model for production planning and control systems (Scheer, 1994).

The application of the general methodology of the reference models to the field of Technology Enhanced Learning (TEL) is relatively new. Examples are the Open Lausanne Model (Madhour & Wentland Forte, 2007), the Course Validation Reference Model COVARM (Franklin et al., 2008), or the Framework Reference Model for Assessment FREMA (Barn et al., 2006). These examples, in particular the survey of Franklin at el. (2008), show that reference models of different complexity exist: from modeling the whole domain of TEL or providing a model for a specific component like assessment. Models like IMS Common Cartridge (IMS, 2008) or SCORM (Advanced Distributed Learning, 2009) can also be seen as reference models as they provide process and data models for the TEL domain. Generally, reference models shall support and ease the implementation of systems in this domain. Therefore, we summarize a variety of models in this domain (system development models, process models, data models) under the term of reference models. The main aim is to support actors to develop TEL systems by adapting the reference model.

One of the most important issues is to analyze how various existing reference models, standards and specifications can be used or adapted for the needs of systems design and development processes. Currently, many standards and specifications are used in the TEL field. However, their distribution and adoption has not reached a critical mass yet. This is in particular the case for the design and development of systems which follow the idea of Open Educational Resources (OERs) (UNESCO, 1998) and reusability. OERs can only be successful if they reach a critical mass in terms of available content, participating users and adoption rates. In particular, interoperability achieved by the use of widely accepted standards is a critical success factor.

In particular in the TEL domain, the use of common-sense specifications and standards as well as reference models is still very low in comparison to other vertical industries (cf. Tyrväinen, Warsta, & Seppänen, 2008). This leads to a variety of proprietary systems and to inefficient development and deployment processes in this domain. There are several reasons for the lack of adoption of TEL models, standards and specifications, such as:

Complete Chapter List

Search this Book: