Virtual Performance Assessment in Immersive Virtual Environments

Introduction

The design and validation of learner interactions in virtual environments is critical for advancing assessment and interactive media in education. Although there has been much attention and speculation as to the effectiveness of the use of immersive virtual environments (IVE) in education, little research has been conducted that validates and provides concrete evidence that links the design of in-world interactions to learning outcomes. Starting with the learning objectives, a framework for evaluating and linking student in-world interactions to learning outcomes is essential. Validating those interactions throughout their development enables a clear connection to how and why students perform certain interactions in a given context. Selected interactions can then be captured in databases and scored for the purposes of establishing a cognitive model of the learner for a given domain.

Sophisticated interactive media, such as IVEs and serious games, enable the automated and invisible collection of rich and detailed event-logs on individual learners, in real-time, during the act of playing and learning (Clarke-Midura, 2009). Such event-logs provide time-stamped records and details of learners’ interactions while they make choices and perform actions within the environment. Since the data captured is explicitly linked to learning outcomes and research questions, extraneous fuzzy data such as key strokes that do not provide information about the learning objectives become irrelevant. This data plays a key role in establishing the validity of in-world interactions for the purposes of assessing learning processes, such as science inquiry. Data can then be used to model students’ thought processes and knowledge development providing a base for understanding the cognitive implications of those actions.

A cognitive model is required to make specific inferences about student learning. Cognitive models in educational measurement relative to serious games and IVEs are simplified descriptions of problem solving on in-world tasks. These models help to characterize the knowledge and skills that students have acquired and facilitate the explanation and prediction of students’ in-world performance. Since in-world interactions have been captured and stored, we propose that problem solving or inquiry in serious games can be described in terms of a cognitive model that can be used to characterize the knowledge, skills, and abilities of students at different stages in their development. As serious games and virtual worlds provide a medium in which to situate students to exercise inquiry practices, there are far reaching implications for serious games and virtual worlds as an assessment tool.

The purpose of this chapter is to offer a framework for the design and validation of interactions in IVEs as they are linked to learning outcomes. In order to illustrate this framework, we provide an overview of the Virtual Performance Assessment (VPA) project at Harvard University (IES# R305A080141). The VPA project is developing immersive technology-based performance assessments that measure middle school students’ scientific inquiry knowledge, skills, and abilities (KSAs) aligned with national standards (College Board, 2009; National Research Council (NRC), 1996). In this chapter, we propose that validating interactions for the purposes of assessment is a multi-step process. First, to frame the discussion we provide a brief overview of the theoretical background of the Virtual Performance Assessment project. Next, we will discuss the process of designing and validating interactions back to the original learning objectives, using the VPA project as an example. Validating interactions back to the original learning objectives ensures the efficiency and content validity of the design. Third, we provide a case study of the VPA project and one empirical step in the process we are using to validate these interactions as outcomes of learning. Through our design framework and case study, this chapter will provide educators, designers, and researchers with a model for how to effectively design virtual and game-based learning environments for the purpose of assessing student problem solving and inquiry learning.