Factors Determining the Overall Effectiveness of E-Learning Systems used in Higher Education

Introduction

This chapter is very closely based on part of an unpublished monograph, prepared largely by Julie Coultas, for Eduserv under the title “How compelling is the evidence for the effectiveness of e-Learning in the post-16 sector? A review of the literature in higher education, the health sector and work-based learning and a post-review stakeholder consultation”. The chapter has a narrower scope than the monograph and concentrates, largely from a UK perspective, on higher education and outlines policy issues and further research.

The original monograph was informed by a number of seminars with experts who pointed us towards relevant literature. They emphasised that E-learning is a dynamic concept that has been both complex and ever-changing, so to measure how effective it has been in itself is a challenge. The review identified definitional elements, key factors in e-learning, how to measure effectiveness within the literature, but also raised the question of whether the research was examining the right issues, which was born out in the review process. Relevance to institutions, work-based learning and health were also reviewed in the original monograph but not included here.

Definitions were identified as containing three components; skills, computer technology and style of learning. Key factors in effective e-learning were identified as being learner confidence, prior knowledge (both operational and conceptual), the presence and involvement of the Teacher, communication (the dialogues between teachers and learners) and the cultural issues relating to managing change. This range of categories in itself identifies the complexity of the field under review, even so both the complexity of the definitions, and the number of key factors, were added to through the expert review process. In the end we adopted the following definition of e-learning:

E-Learning is a portmanteau term covering:

• •

  A style of learning with a particular focus on technology-mediated interactivity and collaboration.

• •

  The use of computer technology in leaning with a particular focus on internet technology.

• •

  The set of skills that enables learners to exploit technology in order to develop understanding or capability.

E-learning requires a different mode of learning, which was characterised as “independent learning”, and critically changes the combinations of Space, Time and Money necessary for learning to take place. Teachers are beginning to get insights in how to use “timely interventions” to deal with a learning mix of “theorising and socialising”. Institutions, however, are not yet equipped to modify the ways that they deal with these combinations not least because key performance indicators (KPI’s) map to traditional learning outcomes.

E-learning is no longer a subset of learning used in distance learning but has evolved into “learning in technology-rich environments” which occurs in multiple contexts both within and beyond the institution. With technology and learning both developing rapidly, but independently, we now have the capability for the co-creation of learning through the integration of the range of technologies available for learning. Institutions however need to move away from the use of simple KPI outcome measures and begin to focus on quality improvement. This has a number of repercussions for an ITS perspective on the field.

In the first section of the chapter, on Higher Education, the HEFCE e-Learning strategy is outlined and previous, partial reviews of e-Learning in higher education are examined. The evidence on the effectiveness of e-Learning in higher education is presented using Kirkpatrick’s (1998) levels of evaluation, as follows:

• (1)

  What are the learners’ reactions to e-Learning? What is the student experience of e-Learning?

• (2)

  What is the quality of the learning?

  • o

    Effective e-Learning and learning styles

  • o

    Comparing the quality of the learning

  • o

    Studies of students’ motivation and self-efficacy in e-Learning environments

• (3)

  Does e-Learning in higher education alter the behaviour of the learner?

• (4)

  What is the impact of e-Learning on the organisation