Rich-Media Interactive Simulations: Lessons Learned

Rich-Media Interactive Simulations: Lessons Learned

Suzanne Tsacoumis (HumRRO, USA)
DOI: 10.4018/978-1-4666-9441-5.ch010
OnDemand PDF Download:
$30.00
List Price: $37.50

Abstract

High fidelity measures have proven to be powerful tools for measuring a broad range of competencies and their validity is well documented. However, their high-touch nature is often a deterrent to their use due to the cost and time required to develop and implement them. In addition, given the increased reliance on technology to screen and evaluate job candidates, organizations are continuing to search for more efficient ways to gather the information they need about one's capabilities. This chapter describes how innovative, interactive rich-media simulations that incorporate branching technology have been used in several real-world applications. The main focus is on describing the nature of these assessments and highlighting potential solutions to the unique measurement challenges associated with these types of assessments.
Chapter Preview
Top

Introduction

For over half a century, high-fidelity assessments have proven to be powerful tools for measuring a broad range of knowledge, skills, abilities, and competencies in both the workplace and educational settings. High-fidelity tools are measures that mirror or closely simulate a particular activity or group of activities. For example, these types of assessments include work samples such as driving a bus, running statistical analyses to answer a question, or taking photographs. They also include measures that do not necessarily replicate the exact activity but simulate it. If an important job activity is to analyze information about a project and then make recommendations on how to proceed, the assessment could create a fictitious project similar to one that would be completed on the job and the test taker could be asked to review the materials and make suggestions on next steps. As another example, a student may be asked to plan an approach to working with classmates to complete an assignment.

The validity of high-fidelity assessments is well documented (e.g., Tsacoumis, 2007; Arthur, Day, McNelly, & Edens, 2003; Schmidt & Hunter, 1998; Klimoski & Brickner, 1987; Moses, 1977; Bray & Grant, 1966), and they tend to be well received given their perceived relevance and face validity. However, they typically are resource and time intensive to implement since they involve live role players and because they require the evaluators to observe each test taker as he or she participates in the assessment. Given this, organizations often reserve the use of high-fidelity assessments only to evaluate candidates for their most senior or critical positions. In addition, businesses are continuing to search for more efficient ways to gather the information they need about one’s capabilities. Organizations have become increasingly reliant on technology-based solutions to evaluate students, teachers, and job candidates. In fact, the use of computers to administer traditional multiple-choice tests is now commonplace, and there is a growing trend to create and implement tests with multimedia components that use sound, video, animation, or some combination, along with text. The computer-based counterpart to live high-fidelity simulations are “rich media” assessments, which involve animation or video and allow the test taker to “interact” with the simulation and dictate how the assessment proceeds or unfolds. High-end interactive simulations have been used for training, such as pilot simulators, but in order for computer-based versions of high-fidelity simulations to gain traction in more traditional processes to assess job candidates and students, the technology must be easily accessible and the measurement challenges need to be addressed. In truth, the technological tools are available to develop and implement interactive computer-based simulations; however, the actual use of these tools for high-stakes processes, such as personnel selection, is still in its infancy.

The objective of this chapter is to describe the lessons learned from developing and implementing several rich media interactive simulations for promotional and developmental purposes in an organizational context, rather than an educational one. That said, the results generalize to any arena focused on creating accurate measures of a variety of skills and abilities. The ultimate goal is to figure out how to use the benefits offered by technology to help master the complexities associated with effectively measuring one’s competencies with enough precision and confidence to make personnel decisions—without the use of live evaluators.

Key Terms in this Chapter

Simulation: An assessment that mirrors the activities of interest, although they are not exact replicas. These measures are used in circumstances when it is impossible to have the individual complete an exact slice of the job.

Stimulus Material: Test material that provides information about the context of the scenario for the test taker, such as a description of the organization, the employees in the office, or details about the projects, issues, or concerns.

High-Fidelity Assessment: A test that mirrors or closely simulates a real-world situation in which test takers takes action based on the scenario and available information just as they would in real life.

High-Stakes Test: An assessment used to make important decisions for the test taker, such as one used to determine who to accept into a college or one used to hire or promote within an organization.

Virtual Role Play: A rich media simulation that presents a realistic, multifaceted scenario that an individual might encounter on the job and that is designed to evoke targeted competencies. Test takers select who they meet with, what they do next, and what type of information to review, and they are scored on their responses to a series of closed-ended questions throughout the simulation (e.g., “Rate the effectiveness of the following sets of responses,” “How soon would you do each of the following?”).

Distance Score: The difference between the test taker’s answer and the keyed response.

Rich Media Simulation: An assessment that uses animation or live video along with branching technology to present the test material in a manner that simulates how the scenario may unfold in real life by allowing the test taker to dictate how the assessment proceeds or unfolds.

Complete Chapter List

Search this Book:
Reset