Technology, Curriculum, and Pedagogy in the Evaluation of an Online Content Program in Australasia

Technology, Curriculum, and Pedagogy in the Evaluation of an Online Content Program in Australasia

Peter Freebody (The University of Sydney, Australia), Sandy Muspratt (Griffith University, Australia) and David McRae (Educational Consultant, Melbourne, Australia)
DOI: 10.4018/978-1-59904-861-1.ch023
OnDemand PDF Download:


The question addressed in this chapter is: What is the evidence for the effects of online programs of learning objects on motivation and learning? Much of the research available on information and communication technologies (ICTs) generally yields short-term or ambiguous findings, with recommendations that centre on the need for more attention to theorizing and documenting: how ICTs can be located within sequences of curricular learning; the kinds of learning that new ICTs offer (factual, conceptual, application, and transfer); and the ways in which existing pedagogies and uses of ICTs both adapt to and transform one another. This chapter aims to advance discussion of these issues by summarizing ongoing evaluations of a large-scale national program of online learning objects across key curriculum areas, drawing on survey and interview data, and a field experiment in which the effects of exposure to learning objects on learning outcomes in mathematics are documented.
Chapter Preview

Introduction And Background

Investment in ICTs by educational systems has grown dramatically over the last decade, as has research evaluating the efficacy of those investments. By way of background to the data summarized later, this section presents a brief, selective summary of this research. (For more extensive coverage, see the British Educational Communications and Technology Agency (BECTA) site; Cox, Abbott, Webb, Blakeley, Beauchamp, & Rhodes, 2003; Freebody, 2005, 2006; Mitchell & Savill-Smith, 2004; Owen, Calnin, & Lambert, 2002; Parr, 2006). We follow this research summary with a brief description of the origins of the national program under examination in the middle sections of this chapter.

Overall, it is striking how many research reports and summaries have signalled their “disappointment” with regard to the ratio between, on the one hand, effort and expenditure, and, on the other, the dissemination, creative use, and efficacy of ICTs in educational settings. Nichol and Watson’s (2003, pp. 132-133) conclusion, following extensive examination of the educational uses of ICTs in the UK, gives the flavour:

the role and nature of ICT in schools is problematic, with minimal involvement of ICT across the curriculum in the everyday teaching of pupils … Rarely in the history of education has so much been spent by so many for so long, with so little to show for the blood, sweat and tears expended.

Similarly, Jamieson-Proctor, Burnett, Finger, and Watson (2006, p. 511) concluded their extensive survey of ICT usage in classrooms in Queensland Australia with this:

there is evidence of significant resistance to using ICT to align curriculum with new times and new technologies … current initiatives with ICT are having uneven and less than the desired results system wide.

It is clear that the introduction of materials based on ICTs into classrooms has not, of itself, been shown to have brought about changes that commentators claim are needed for new forms of economic and civil life (e.g., CEO Forum, 2000).

Some studies have focussed on rates of ICT usage in schools. Pittard and Bannister (2005), for example, drew together studies that reported simple rates of uptake of ICTs across a range of curriculum domains in UK. They showed that reported usage is increasing, but that this increase has not been consistent across curriculum domains: There has been increased usage in mathematics and science, but substantially less in English and other arts/humanities-based subjects.

On the question of the outcomes of ICT usage, Pittard and Bannister (2005) sounded three cautionary notes. First, they noted that some promising outcomes may appear only after the passage of some considerable time, but also that, contrariwise, some other effects may in fact disappear. The research is not yet at a point of offering guidance on which kinds of outcomes will be visible over what timeframes. Second, Pittard and Bannister (2005) noted that we might expect differential consequences for learners with varying linguistic, cultural, and socio-economic characteristics. Finally, they posed the puzzle of how we may determine how much of an effect is due to the use of the technology itself, how much to the work of the teacher, and how much to other factors.

Three technology-related features that Pittard and Banister (2005) used to account for the minor gains observed in some studies were:

  • 1.

    The enhanced presentational capabilities available for lesson and assignment work (see Abidin & Hartley, 1998; Dori & Barak, 2001);

  • 2.

    The enhanced range of resources available to support students’ and teachers’ research; and

  • 3.

    The immediacy of feedback for self-evaluation (see Marzano, Gaddy, & Dean, 2000).

Key Terms in this Chapter

Motivation: Taken here to comprise persistence, engagement, and enjoyment.

Pedagogy: Taken here to involve curriculum materials, teacher-student interaction, and assessment activities.

Learning Object: Files or modules of learning material that represent interactive learning activities that may include texts, and/or graphic, audio or animated materials, and that are reusable in multiple settings and for multiple purposes.

Learning: Taken here to involve the understanding and management of factual content, the understanding and use of concepts, and the application of content and concepts to new settings.

Field Experiment: A trialling of an intervention that is designed to keep most variables constant except the intervention in question within the constraints of the allocation of schools and classrooms into conditions, rather than the random allocation of students, and the reliance on ‘standard practice’ in a contrast group, rather than a strict ‘control group’.

Multilevel Modelling: A set of procedures for partitioning variance at different levels of unit formation (e.g., students within classrooms within schools).

Online Content: Referring here to digital materials accessible from digital repositories, as referenced, located, and accessed by metadata descriptors.

Complete Chapter List

Search this Book:
Table of Contents
Tom Carey
Lori Lockyer, Sue Bennett, Shirley Agostinho, Barry Harper
Lori Lockyer, Sue Bennett, Shirley Agostinho, Barry Harper
Chapter 1
Shirley Agostinho
The term “learning design” is gaining momentum in the e-learning literature as a concept for supporting academics to model and share teaching... Sample PDF
Learning Design Representations to Document, Model, and Share Teaching Practice
Chapter 2
Isobel Falconer, Allison Littlejohn
Practice models are generic approaches to the structuring and orchestration of learning activities for pedagogic purposes, intended to promote... Sample PDF
Representing Models of Practice
Chapter 3
Rob Koper, Yongwu Miao
IMS learning design (IMSLD) is an open standard that can be used to specify a wide range of pedagogical strategies in computer-interpretable models.... Sample PDF
Using the IMS LD Standard to Describe Learning Designs
Chapter 4
David Griffiths, Oleg Liber
The IMS LD specification is internally complex and has been used in a number of different ways. As a result users who have a basic understanding of... Sample PDF
Opportunities, Achievements, and Prospects for Use of IMS LD
Chapter 5
Franca Garzotto, Symeon Retalis
“A design pattern describes a problem which occurs over and over again in our environment, and then describes the core of the solution to that... Sample PDF
A Critical Perspective on Design Patterns for E-Learning
Chapter 6
Sherri S. Frizell, Roland Hübscher
Design patterns have received considerable attention for their potential as a means of capturing and sharing design knowledge. This chapter provides... Sample PDF
Using Design Patterns to Support E-Learning Design
Chapter 7
Peter Goodyear, Dai Fei Yang
This chapter provides an overview of recent research and development (R&D) activity in the area of educational design patterns and pattern... Sample PDF
Patterns and Pattern Languages in Educational Design
Chapter 8
Gráinne Conole
The chapter provides a theoretical framework for understanding learning activities, centering on two key aspects: (1) the capture and representation... Sample PDF
The Role of Mediating Artefacts in Learning Design
Chapter 9
Elizabeth Masterman
This chapter uses activity theory to construct a framework for the design and deployment of pedagogic planning tools. It starts by noting the impact... Sample PDF
Activity Theory and the Design of Pedagogic Planning Tools
Chapter 10
Barry Harper, Ron Oliver
This chapter describes the development of a taxonomy of learning designs based on a survey of 52 innovative ICT-using projects that formed the basis... Sample PDF
Developing a Taxonomy for Learning Designs
Chapter 11
Carmel McNaught, Paul Lam, Kin-Fai Cheng
The chapter will describe an expert review process used at The Chinese University of Hong Kong. The mechanism used involves a carefully developed... Sample PDF
Using Expert Reviews to Enhance Learning Designs
Chapter 12
Matthew Kearney, Anne Prescott, Kirsty Young
This chapter reports on findings from a recent project situated in the area of preservice teacher education. The project investigated prospective... Sample PDF
Investigating Prospective Teachers as Learning Design Authors
Chapter 13
Paul Hazlewood, Amanda Oddie, Mark Barrett-Baxendale
IMS Learning Design (IMS LD) is a specification for describing a range of pedagogic approaches. It allows the linking of pedagogical structure... Sample PDF
Using IMS Learning Design in Educational Situations
Chapter 14
Robert McLaughlan, Denise Kirkpatrick
Decision-making processes in relation to complex natural resources require recognition and accommodation of diverse and competing perspectives in a... Sample PDF
Online Role-Based Learning Designs for Teaching Complex Decision Making
Chapter 15
Garry Hoban
Digital animations are complex to create and are usually made by experts for novices to download from Web sites or copy from DVDs and CDs to use as... Sample PDF
Facilitating Learner-Generated Animations with Slowmation
Chapter 16
Yongwu Miao, Daniel Burgos, David Griffiths, Rob Koper
Group interaction has to be meticulously designed to foster effective and efficient collaborative learning. The IMS Learning Design specification... Sample PDF
Representation of Coordination Mechanisms in IMS LD
Chapter 17
Johannes Strobel, Gretchen Lowerison, Roger Côté, Philip C. Abrami, Edward C. Bethel
In this chapter, we describe the process of modeling different theory-, research-, and best-practicebased learning designs into IMS-LD, a... Sample PDF
Modeling Learning Units by Capturing Context with IMS LD
Chapter 18
Daniel Burgos, Hans G.K. Hummel, Colin Tattersall, Francis Brouns, Rob Koper
This chapter presents some design guidelines for collaboration and participation in blended learning networks. As an exemplary network, we describe... Sample PDF
Design Guidelines for Collaboration and Participation with Examples from the LN4LD (Learning Network for Learning Design)
Chapter 19
Tom Boyle
This chapter argues that good design has to be at the heart of developing effective learning objects. It briefly outlines the “knowledge... Sample PDF
The Design of Learning Objects for Pedagogical Impact
Chapter 20
Margaret Turner
This chapter introduces an approach to writing content for online learning over networked media. It argues that few resources currently utilise the... Sample PDF
Visual Meaning Management for Networked Learning
Chapter 21
Christina Gitsaki
Due to the increasingly diverse student population in multicultural nations such as Australia, the U.S., Canada, and the UK, educators are faced... Sample PDF
Modification of Learning Objects for NESB Students
Chapter 22
Daniel Churchill, John Gordon Hedberg
The main idea behind learning objects is that they are to exist as digital resources separated from the learning task in which they are used. This... Sample PDF
Learning Objects, Learning Tasks, and Handhelds
Chapter 23
Peter Freebody, Sandy Muspratt, David McRae
The question addressed in this chapter is: What is the evidence for the effects of online programs of learning objects on motivation and learning?... Sample PDF
Technology, Curriculum, and Pedagogy in the Evaluation of an Online Content Program in Australasia
Chapter 24
David Lake, Kate Lowe, Rob Phillips, Rick Cummings, Renato Schibeci
This chapter provides a model to analyse the effectiveness and efficiency of Learning Objects being used in primary and secondary schools by... Sample PDF
Effective Use of Learning Objects in Class Environments
Chapter 25
Robert McCormick, Tomi Jaakkola, Sami Nurmi
Most studies on reusable digital learning materials, Learning Objects (LOs), relate to their use in universities. Few empirical studies exist to... Sample PDF
A European Evaluation of the Promises of LOs
Chapter 26
Tomi Jaakkola, Sami Nurmi
There has been a clear lack of rigorous empirical evidence on the effectiveness of learning objects (LOs) in education. This chapter reports the... Sample PDF
Instructional Effectiveness of Learning Objects
Chapter 27
Robert McCormick
This chapter will examine the approach taken in the evaluation of a large-scale feasibility trial of the production, distribution, and use of... Sample PDF
Evaluating Large-Scale European LO Production, Distribution, and Use
Chapter 28
John C Nesbit, Tracey L. Leacock
The Learning Object Review Instrument (LORI) is an evaluation framework designed to support collaborative critique of multimedia learning resources.... Sample PDF
Collaborative Argumentation in Learning Resource Evaluation
Chapter 29
Philippe Martin, Michel Eboueya
This chapter first argues that current approaches for sharing and retrieving learning objects or any other kinds of information are not efficient or... Sample PDF
For the Ultimate Accessibility and Reusability
Chapter 30
Sue Bennett, Dominique Parrish, Geraldine Lefoe, Meg O’Reilly, Mike Keppell, Robyn Philip
As the notion of learning objects has grown in popularity, so too has interest in how they should be stored to promote access and reusability. A key... Sample PDF
A Needs Analysis Framework for the Design of Digital Repositories in Higher Education
Chapter 31
William Bramble, Mariya Pachman
Reusable learning objects (LOs) constitute a promising approach to the development of easily accessible, technologically sound, and curriculum... Sample PDF
Costs and Sustainability of Learning Object Repositories
Chapter 32
Kristine Elliott, Kevin Sweeney, Helen Irving
This chapter reports the authors’ experiences of developing a learning design to teach scientific inquiry, of integrating the learning design with... Sample PDF
A Learning Design to Teach Scientific Inquiry
Chapter 33
Lisa Lobry de Bruyn
This chapter explores through a case study approach of a tertiary-level unit on Land Assessment for Sustainable Use, the connections between three... Sample PDF
Adapting Problem-Based Learning to an Online Learning Environment
Chapter 34
Tan Wee Chuen, Baharuddin Aris, Mohd Salleh Abu
This chapter aims to guide the readers through the design and development of a prototype Web-based learning system based on the integration of... Sample PDF
Learning Objects and Generative Learning for Higher Order Thinking
Chapter 35
Sebastian Foti
The author describes the work of Dr. Mary Budd Rowe and the establishment of an early learning object databases. Extensive training with K-12... Sample PDF
Applying Learning Object Libraries in K-12 Settings
Chapter 36
L. K. Curda, Melissa A. Kelly
We present guidelines for designing and developing a repository for the storage and exchange of instructional resources, as well as considerations... Sample PDF
Guidelines for Developing Learning Object Repositories
Chapter 37
Sandra Wills, Anne McDougall
This study tracks the uptake of online role play in Australia from 1990 to 2006 and the affordances to its uptake. It examines reusability, as one... Sample PDF
Reusability of Online Role Play as Learning Objects or Learning Designs
Chapter 38
Lori Lockyer, Lisa Kosta, Sue Bennett
Health professional education is changing to meet the demands of a limited workforce and a focus on community-based clinical training. The change... Sample PDF
An Analysis of Learning Designs that Integrate Patient Cases in Health Professions Education
Chapter 39
Mohan Chinnappan
The shift in the way we visualise the nature of mathematics and mathematics learning has presented educational technologists with new challenges in... Sample PDF
Reconceptualisation of Learning Objects as Meta-Schemas
Chapter 40
Henk Huijser
This chapter provides an in depth discussion of the issues involved in integrating learning design and learning objects into generic Web sites. It... Sample PDF
Designing Learning Objects for Generic Web Sites
Chapter 41
Morag Munro, Claire Kenny
E-learning standards are a contentious topic amongst educators, designers, and researchers engaged in the development of learning objects and... Sample PDF
Standards for Learning Objects and Learning Designs
Chapter 42
Eddy Boot, Luca Botturi, Andrew S. Gibbons, Todd Stubbs
In developing modern instructional software, learning designs are used to formalize descriptions of roles, activities, constraints, and several... Sample PDF
Supporting Decision Making in Using Design Languages for Learning Designs and Learning Objects
Chapter 43
Gilbert Paquette, Olga Mariño, Karin Lundgren-Cayrol, Michel Léonard
This chapter summarizes the work on instructional engineering and educational modeling accomplished since 1992 at the LICEF Research Center of... Sample PDF
Principled Construction and Reuse of Learning Designs
About the Contributors