Evaluation of E-Learning

1. Introduction

E-learning has evolved intensively over the past decade technologically as well as concerning its scope. E-learning means the acquisition and use of knowledge distributed and facilitated primarily by electronic means (Learnframe.com, 2005). Beginning with text based measures that were presented on media like CDs only 10 to 15 years ago, e-learning nowadays use all possible electronic media and hardware to serve the learner with multimedia, virtual and personalized contents. This enlargement – technologically and with regard to content – poses new challenges on evaluators. They relate mainly to the rapid development of e-learning and the associated changes in infrastructures (Ertl, Winkler, & Mandl, 2007) as well as in missing experiences in the applicability of the new technologies for beneficial learning. Both aspects emphasize the need for appropriate quality management that can be established by thorough evaluation.

Stockmann (2000) defines four possible results of an evaluation: to get insights into a project and receive data necessary for decisions, to get control over a project and to be able to make refinements, to establish a dialogue between different stakeholders, e.g. financiers, providers and the target group, and to legitimize costs and sustainability of a program. In sum, evaluation means to exactly define and measure a product’s or programme’s usefulness and worth (Reinmann-Rothmeier, Mandl, Erlach, & Neubauer, 2001).

In the field of e-learning, evaluation mainly focuses on the quality of the learning environment and on learners’ negotiation in and with the learning environment. Thereby, evaluation has two main purposes: To improve and to adapt the learning environment to learners’ needs (which combines the functions of insight and control) and to prove the quality of the learning environment and its values and benefits for financiers and participants (legitimization). Regarding the first aspect, Mandl and Hense (2007) emphasize the importance for evaluators to learn about the particular functions and effects of a learning environment to realize learners’ best benefits. We will elaborate on this in the further sections of this paper. However, also the second argument has special weight: given the costs of research and development as well as purchase of an e-learning environment or program, money plays a role for especially two stakeholders. The one is the company that is offering the environment or product on the education market and the other are companies and organisations that buy and apply it (Haben, 2002). As profit-organisations are interested in satisfactory cost-benefit-relations, the producer may be interested in knowing how well one performs with its product and will try to test it or to get evaluation data to confirm the product’s quality (Harhoff & Küpper, 2002). Furthermore, the purchaser might be interested in information about the usefulness of the implementation of the product or environment in terms of learning results (Harhoff & Küpper, 2002): Human resources divisions in companies are responsible for implementation of and reporting on the usefulness of methods and measures offered to the company’s employees and are obliged to choose measures that support the organisation’s overall success (Sonntag, 2002; Knyphausen-Aufseß, Smukalla, & Abt, 2009).

Another reason to broaden the efforts in evaluating e-learning is the growing market: not only the quantity of e-learning measures and products evolved strongly during recent years, but also the array of recipients increased intensively: young adults with academic qualification take master-programs at distance universities to qualify themselves during they are in job (Schnurer, 2005), undergraduate students have the choice to study at home without attending presence courses, adults without higher formal qualification try to educate further while taking e-learning-courses of private institutions (Erlach, Hausmann, Mandl, & Trillitzsch, 2002) and so on. Having this in mind, it might not only be scientific interest but mere practical need to evaluate e-learning further.