Assessing 3D Virtual World Learning Environments with the CIMPLe System: A Multidisciplinary Evaluation Rubric

Introduction

Although the idea of virtual environments in education might seem radical, it is not new; rather, programs such as Quest Atlantis (Indiana University), River City (Harvard University) and SciCentr (Cornell Theory Center) have been in use for more than ten years. Research from these programs suggests that students exhibit gains in engagement, efficacy and achievement (Barab, et al, 2005; Ketelhut, et al, 2006). Additionally, a recent study (Hansen et al, 2004) noted that students actively involved in three-dimensional construction of computational models had a more sophisticated understanding of dynamic spatial relationships than students in a traditional classroom environment. Other studies (e.g., Kim, 2006) suggest a statistically significant effect of 3-D virtual environments on both achievement and on developing a positive attitude toward science.

Finally, Jones (2004) proposed that multi-user, 3-D, online learning environments demonstrate numerous important educational benefits such as engaged immersion, situated learning, multi-modal communications, breakdown of socio-cultural barriers, bridging the digital divide, problem solving, and the ability to create empathy and understanding for complex systems. Other advantages of virtual environments for learning include the ability to provide experiences that may not be available in real life, the ability to analyze phenomena from different points of view to gain deeper understanding, and the ability to work with virtual companions distributed over different geographical locations (Chittaro and Ranon, 2007).

Why do learners respond so well to virtual environments? In the late 1990s, researchers began hypothesizing, for example, that the level of presence in a virtual world—the feeling of being somewhere else—as well as the level of immersion—the feeling of interacting directly with the environment—account for the success of instructional virtual worlds (Witmer & Singer 1998). More recent studies investigate other aspects of successful virtual worlds for instructional contexts, including the role of social facilitation, or the degree to which having others “present” impacts performance (Park & Catrambone, 2007); (Bronack, Cheney, Riedl, & Tashner, 2008); the role of place metaphors in guiding action (Prasolova-Forland, 2008) and the complementary concerns of cognitive load and system adaptivity (Scheiter & Gerjets, 2007); (Kalyuga, 2007).

While these and many other studies analyze 3D virtual worlds from the perspective of one discipline or another and offer recommendations about building these worlds from those perspectives, none of these studies have proposed a multidisciplinary method of evaluating the success of a virtual world learning environment (VWLE) that considers the complex interactions of context, interactivity, motivation, presence and cognitive load.Virtual worlds require simultaneous attention to a number of factors to ensure that they are successful and when we add the complications of instructional purposes, the range of considerations expands even further. Consequently, many of the approaches that focus on a single aspect of a virtual world, such as presence or interactivity, gloss over the complexity that these environments require. To begin moving instructional designers, trainers, and researchers toward a more complex understanding of assessing these environments, this chapter introduces an assessment rubric for virtual world learning environments built from proven principles of user experiencedesign, instructional design, interface design, learning theory, technical communication, instructional systems design (ISD), and VIE motivation theory. We have titled this rubric the “CIMPLe System” since it captures the ways that context, interactivity, motivation, presence, and cognitive load weave together to form a successful virtual world learning environment.