Design, Technology, and Measurement Considerations in Virtual-Reality Assessment

Design, Technology, and Measurement Considerations in Virtual-Reality Assessment

Sean P. Gyll, Karen K. Shader, Paul Zikas, George Papagiannakis
Copyright: © 2023 |Pages: 29
DOI: 10.4018/978-1-6684-7644-4.ch008
(Individual Chapters)
No Current Special Offers


Virtual reality (VR) simulations as an assessment tool represent a much-needed effort to move beyond the shortcomings of today's forms-based measures. Within VR, we assess for competency and problem-solving skills versus the content memorization typically supported by multiple-choice assessments. This chapter reviews the development process for the behavioral healthcare coordination VR assessment deployed at Western Governors University. It follows three patients undergoing behavioral health care treatment and highlights essential design, technology, and measurement considerations in developing a VR assessment. For any assessment program, construct validity is the chief validity component. This means that standards-based principles must be maintained to support the inferences drawn from test scores. However, without a framework for developing and maintaining those standards, assessment developers are left to their own devices to determine which practices are most likely to be effective. This chapter provides practical examples to aid assessment professionals in maintaining those standards.
Chapter Preview


In online competency-based higher education (OCBHE), students move through their degree programs one course at a time by demonstrating mastery of various content and professional domains. As students work within an individual domain of knowledge, they are assessed on one or more topics, each consisting of a series of competencies with associated test objectives and performance tasks (Gyll & Ragland, 2018). As the popularity of OCBHE programs continues to grow, their integrity will be scrutinized by students and employers alike, and their credibility will largely depend on the quality of the assessments used (McClarty & Gaertner, 2015).

To meet this demand, institutions of higher learning are beginning to realize the importance of real-world, performance-based measures like those found in Virtual Reality (VR) environments, where demonstrations of competence are crucial to success. Central to the development of VR assessments is the need for high-fidelity demonstrations of learner aptitudes and competencies that include a significant emphasis on cognitive abilities (i.e., knowing what) and the performance of those abilities (i.e., demonstrating how). Assessment fidelity describes the reality of function and how closely the environment imitates the real-life counterpart. The most significant advantage of this testing mode is the real-world relevance that can be incorporated into the assessment, and technological innovations continue to expand these opportunities.

Competency-based skills education follows a system of assessing needs, designing processes, developing materials, and evaluating them against specific learning outcomes. Within this context, learning is no longer viewed as transmitting knowledge from instructor to student (e.g., sitting passively, listening to a lecture, taking notes, and applying concepts) but as an active process acquired through various instructional and media types. Compared to traditional instructor-led methods, research shows that competency-based skills are improved when technology integrates learning into the educational experience rather than being delivered in a compartmentalized fashion (Ford & Gopher, 2015; Hoogveld, 2003; Van Merrienboer & Kirschner, 2013; Merrill, 2002). As a result, students' capacity to develop a particular domain's competency and then transfer that information to learning outcomes improves (Gyll & Hayes, in press).

Some have suggested that skills development as a first step to acquiring competency can be better taught with high-fidelity VR simulation because VR allows for a more active and immersive learning experience (Kyaw et al., 2019; Birrenbach et al., 2021). For example, VR is often used in medical education for developing skills of varying complexity, ranging from simple nursing skills to laparoscopic/endoscopic/endovascular skills (Pantelidis et al., 2017; Khan et al., 2019) or complex surgical skills (Hooper et al., 2019; Grantcharov, 2008). Additionally, VR simulation using head-mounted devices (HMDs) offers a multisensory, 3-D, fully immersive experience. Through immersion, sense of presence, and interaction with the virtual environment in a real-time and realistic manner, VR simulation can create emotional experiences that facilitate experiential learning, exceeding other 2-D learning modalities (Babini et al., 2020). However, while these studies have demonstrated the utility of VR simulation as a teaching tool, scant literature exists on its utility as an assessment instrument. Furthermore, fewer studies exist on meaningful development guidelines to aid assessment professionals in the development process.

Complete Chapter List

Search this Book: