Designing training tools for complex jobs within military domains can be challenging. Training for aircraft maintainers, in particular, requires detailed instruction and guidance from job guides, instructors and aircraft equipment. These approaches are often inefficient, outdated and resource-intensive. Gaming and simulation can provide many of the necessary support capabilities to facilitate more cost-efficient, timely, engaging and effective training. Because current maintenance training procedures follow a very hands-on approach in terms of instructor and student involvement, it is essential that development of game-based training technologies is undertaken with these end users in mind. This chapter describes how iterative usability testing was employed throughout the development process for two software applications created for aircraft maintenance training. It also describes how a user-centered design approach supported the development of these game-based programs for superior training methods and experiences for all end users, including instructors, students and administrators.
TopIntroduction
Technological advances have revolutionized training capabilities for individual, team, and team-of-teams environments. Computer games and simulations have received much attention for delivering realistic, interactive, engaging, and in many cases, more effective instruction for stronger learning outcomes, compared to traditional classroom instruction (Connolly, Boyle, MacArthur, Hainey, & Boyle, 2012; Vogel, Vogel, Cannon-Bowers, Bowers, Muse, & Wright, 2006; Wilson et al., 2009). The efficacy of including games in instruction has not been found to be universal, however subject matter with very specific content (such as specific procedures, tools, and techniques for maintenance training) can be more effectively learned when gaming is used with instruction (Randel, Morris, Wetzel, & Whitehill, 1992). Gaming can also be used together with simulation to enhance instruction. Our software applications leveraged simulation, or operating models of the existing aircraft, as the foundation for two respective training programs. Gaming features, including rules, strategies, challenges, and consequences for failure contained in the environment (Garris, Ahlers, & Driskell, 2002), were also incorporated into the simulation. These elements help provide immersive, high-fidelity training experiences that can be more motivating instructional environments for students (Alexander, Brunyé, Sidman, & Weil, 2005; Cannon-Bowers & Bowers, 2009; Driskell & Dwyer, 1984; Rieber, 1996). However, the primary goal of leveraging simulation and gaming in our programs was to facilitate better instruction and more enduring learning, by “[bridging] the gap between the classroom and the real world” in a complex, simulated environment (Gredler, 2004).
When innovative educational technologies are custom-designed for a particular educational or training audience, the end users must be a focal consideration in the development process since the user is primarily in control of manipulating the application. The term user-centered design refers to this notion of recognizing the user’s needs and interests and allowing the user to influence the design process to enhance the design’s usability and effectiveness (Abras, Maloney-Krichmar, & Preece, 2004). This includes tailoring the design to both the needs as well as the capabilities of the end-user or operator (Endsley & Jones, 2016). Several recommendations for conducting user-centered design are well-documented in the literature (Norman & Draper, 1986; Norman, 1988; Preece, Rogers, & Sharp, 2002). Incorporating some of these techniques into the design process can not only enhance usability, but can offer several long-term benefits such as a higher likelihood of implementation, utilization, and sustainment of the system.
Usability testing and evaluation are instrumental for effectively integrating user-centered design into the development of educational technologies. Usability evaluations can be conducted using a variety of methods, including more straightforward questionnaires such as the widely-used System Usability Scale (Brooke, 1996), or alternatively, more interactive processes like cognitive walkthroughs (Lewis, Polson, Wharton, & Rieman, 1990; Mahatody, Sagar, & Kolski, 2010) and heuristic evaluations (Nielsen, 1992). Keeping users in the loop throughout development not only positions the user as a top consideration in design decision-making (Dumas & Redish, 1993), but it also helps build user acceptance and a sense of ownership with the new system. This is especially important with new military technology. Military personnel are often adopting new systems, and as a result it may be difficult to get buy-in and support from these user groups for executing these new systems. Iterative usability testing can help foster user acceptance, system favorability and ultimately adoption among user groups. We incorporated usability testing as a central element of our user-centered design approach. Beyond that, we also used a more comprehensive ecological interface design approach to guide our development.