Abstract
In the last five years, the analytical techniques for identifying the processes of online learning have developed to the point where applications for the assessment of learning can be discussed. This would be most appropriate for twenty-first century skills—such as collaboration, decision-making, and teamwork skills—which are the core learning outcomes in immersive learning environments. The state of the art in this field is still at the stage of discovering patterns of the processes of learning, identifying stages, and suggesting their meaning. However, already it is important to consider what technologies can offer and what information teachers need in order to evaluate students' situated performance and to provide useful feedback. This chapter describes an imagined virtual world, one that affords the range of twenty-first century skills, in order to illustrate types of analyes that could be conducted on learning process data. Such analytical methods could provide both descriptive information about the performance of learners and depict structures and patterns of their learning processes. The future assessment of learning in immersive virtual worlds may draw on data about deep embodied processes and multiple senses that usually underpin professional skills, such as affect, visual perception, and movement. This type of assessment could also provide deeper insights into many psychological processes in collaborative learning, decision-making, and problem-solving in virtual worlds, such as motivation, self-efficacy, and engagement. Overall, the view of the assessment presented in this chapter extends beyond the formal learning outcomes that are usually required by tertiary education quality and standards agencies and assessed in traditional courses in higher education to include a range of new capacities that may not be required but are essential for successful performance in contemporary workplaces.
TopIntroduction
The design of assessment for virtual worlds is a complex task. In order to structure this design task, perhaps the first aspect to consider is the end user/s of the information collected. Assessment should provide both teachers and students with information that helps them progress with their roles. Teachers need data that both informs the design of subsequent tasks and updates them on the progress of their students. Students need information about their progress, to guide their choices for future learning pathways. Learning in virtual worlds usually focuses on providing students with authentic experiences of “being” and skills for “acting” in a rich situated environment, and thus many of the learning outcomes are embodied in interactions and embedded in processes, rather than depicted by discrete knowledge states or decontextualized skills. For example, a pilot’s ability to scan the environment, control an aircraft and coordinate actions with other crew members could hardly be enhanced without actually scanning a rich, dynamically changing environment, physically operating the plane and coordinating one’s actions.
Open-ended learning experiences and continuous streams of less structured data, such as those generated during interactions within and with virtual worlds, do not easily lend themselves to the objectification of assessment measures that are considered to be essential in standardized tests, certification exams and other high stakes summative assessments; nor do they offer easy-to-interpret information that could inform learning choices or instructional decisions. Embedding summative and formative assessment into immersive authentic tasks requires new assessment approaches and new methods for data analysis.
Researchers in the domains of learning analytics, educational data mining, decision-making and related domains have been increasingly adapting and developing assessment techniques to gain insights into learning processes. They draw on assessment approaches and data analysis methods from such fields as serious gaming, data mining, predictive modelling, model tracing, computational linguistics, social network analysis and information visualisation (Romero Ventura, Pechenizkiy, & Baker, 2010; Shum, 2012). The application of these techniques in virtual worlds on the large datasets that result from recording learner behaviour provides possibilities for gaining new insight into rather different aspects of students’ performance. In this chapter, we discuss emerging process data analysis techniques that could be deployed for assessment in immersive virtual learning environments. Our main focus is analytic techniques for formative assessment of collaborative students’ performance. First, we will clarify what we mean by assessment, and distinguish between summative and formative assessment; outcome and process measures; individual and collaborative performance; and cognitive and non-cognitive skills.