Evaluating e-Learning Initiatives: A Literature Review on Methods and Research Frameworks

Evaluating e-Learning Initiatives: A Literature Review on Methods and Research Frameworks

Stelios Daskalakis (University of Patras, Greece) and Nikolaos Tselios (University of Patras, Greece)
DOI: 10.4018/jwltt.2011010104
OnDemand PDF Download:
No Current Special Offers


Evaluation aspects, in relation to e-learning initiatives, are gaining substantial attention. As technology continuously influences learning, technical as well as organizational requirements need to be thoroughly investigated across a variety of stakeholders. In this paper, an outline of those aspects is presented, which occurred from a literature review on methods and research frameworks utilized toward the evaluation of e-learning initiatives. The review identified a series of studies that take advantage of well-established theories in the area of users’ acceptance of technology combined with additional, e-learning context-specific factors. Results of the review are presented, according to the adopted research model, to ease the process of locating and retrieving e-learning evaluation paradigms per theoretical model. In addition, research findings are discussed and future implications for e-learning evaluation initiatives as well as potential stakeholders are highlighted.
Article Preview

1. Introduction

The continuous penetration of web technology characterises everyday aspect of life (Lee, Christy, Cheung, & Chen, 2005, p. 1095). Some researchers perceive the web as an active learning environment that supports creativity (Becker & Dwyer, 1994). According to Thuring, Mannemann, and Haake (1995), the web encourages exploration of knowledge and browsing behaviours that are strongly related to learning. Reasonably, the potentially use of e-learning is increasingly explored and it has been widely accepted that the hyper-medial structure of the web could promote learning (Lee, 2008, p. 34; U.S. Department of Education, Office of Planning, Evaluation, and Policy Development, 2010).

The above mentioned technological evolution created a variety of learning alternatives which go beyond the traditional classroom settings, and are identified as “e-learning” initiatives. Johnson, Hornik and Salas (2008) define those initiatives as “training or educational initiatives which provide learning material in online repositories, where course interaction and communication and course delivery are technology mediated” (p. 357). In this context, e-learning initiatives continue to grow as they expand into a wide range of educational needs, with a variety of teaching and learning modes, approaches and styles, underlining the broad range of both its use and “complexity” (Ozkan, Koseler, & Baykal, 2008, p. 1).

The evaluation of e-learning initiatives is not a straightforward process. Many factors contribute to the success of e-learning systems in general and Learning Content Management Systems (LCMS) in particular. Such a technological learning environment should present innovative learning opportunities, thus extending and not just replicating traditional learning approaches. Control and responsibility of the learning process should be gradually shifted from the educators to the learners. In this context, socio-cultural theories influence considerably the learning procedure and have strengthened the perceptions of the educational community towards adoption and effective integration of open and distance learning (ODL) systems in the educational process (Duffy & Kirkley, 2004).

As a result, a multifaceted approach is needed, beyond the “abstract” use of past theories that identify e-learning evaluation from a “high-level” perspective (Loukis, Georgiou, & Pazalos, 2007, p. 374). In line with this ascertainment, equal attention should be paid in the transition from “theoretical” underpinnings to practical implications “during the evaluation of “of e-learning environments’ effectiveness” (Ozkan, Koseler, & Baykal, 2008, p. 1). The encapsulation of several factors under consideration will assist in the formulation of a holistic view with regards to e-learning evaluation, similar to the “holistic approach” with regards to implementantion, as pointed by Sela and Sivan (2009, p. 164) with a major aim the “enhancement of student learning” (Mandinach, 2005, p. 1827). In this context, Keller (2005) formulates the proposition that the implementation of Virtual Learning Environments (VLE) is oriented in “three perspectives: implementation as technology acceptance, implementation as diffusion of innovations and implementation as a learning process” (Keller, 2005, p. 299).

In concordance with the above statements, it became evident from initial studies based on models such as Technology Acceptance Model (TAM, Davis, 1989) that students’ personal beliefs and attitudes towards web-based education constitute a critical factor to the successful incorporation of an Information System (IS) in the learning practices of an institution. Aspects such as the perceived usefulness and perceived ease-of-use were typically examined. Davis (1989) defined perceived usefulness as “the degree to which a person believes that using a particular system would enhance his/her job performance” (p. 320). Perceived ease of use is defined as “the degree to which a person believes that using a particular system would be free of physical and mental effort” (Davis, 1989, p. 320).

Complete Article List

Search this Journal:
Volume 18: 1 Issue (2023)
Volume 17: 8 Issues (2022)
Volume 16: 6 Issues (2021)
Volume 15: 4 Issues (2020)
Volume 14: 4 Issues (2019)
Volume 13: 4 Issues (2018)
Volume 12: 4 Issues (2017)
Volume 11: 4 Issues (2016)
Volume 10: 4 Issues (2015)
Volume 9: 4 Issues (2014)
Volume 8: 4 Issues (2013)
Volume 7: 4 Issues (2012)
Volume 6: 4 Issues (2011)
Volume 5: 4 Issues (2010)
Volume 4: 4 Issues (2009)
Volume 3: 4 Issues (2008)
Volume 2: 4 Issues (2007)
Volume 1: 4 Issues (2006)
View Complete Journal Contents Listing