A Protocol for Reviewing Off-the-Shelf Games to Inform the Development of New Educational Games

Introduction

A number of design process models specify methods for developing educational games, highlighting unique design concerns in this space (Chamberlin et al, 2012; De Freitas and Neumann, 2009; De Freitas and Oliver, 2006; De Freitas and Routledge, 2013; Dormann and Biddle, 2008; Groff, J. et al, 2015; Gunter, G.A. et al, 2008; Kiili, K. et al, 2012; Mislevy and Haertel, 2006). In some of these models, a design step is to review existing games, not only to understand the market space, but also to identify design principles and good pedagogy (Chamberlin et al, 2012; De Freitas and Routledge, 2013; Dormann and Biddle, 2008). Existing game review models (Petri, G. et al, 2016; Lucero, A. et al, 2014; Sweetser and Wyeth, 2005) evaluate games based on the player’s experience, or analyze them from the perspective of a learning theory, such as flow (Csikszentmihalyi, 1996). While it is important to review games for design purposes, little information is given regarding the structure of the process and the ways this process differs from reviewing games for other reasons.

Game review is often a way to assess games for use in a classroom, curate a list of educational games, or identify what games are available in a content area for a certain age group. While it is important to review games to identify those that are of value for a specific type of player, it is also critical to review games as a way of identifying gaps in content and pedagogy. This type of game review can identify the need for new games, dictate what types of new games are needed, and might suggest ways in which games can present content in novel ways.

As part of a design process, the performance of game review serves another purpose when done collaboratively: it engages the design team in a deeper level of pedagogical understanding about the content and design of the games. Effective educational game design teams should represent diverse backgrounds and disciplines (Chamberlin et al, 2012), and it is important that all team members be well played (Being “well played” refers to game literacy, like being “well read”: having a diverse knowledge and experience of playing games) (Davidson, 2009). Further, the process brings the team together in articulating a shared expertise vision for addressing game content, identifying valuable game mechanics, and deciding the best approaches for how a game should work. When interdisciplinary design teams review games with a variety of design lens, they: a) look at the overall quality of a game; b) identify gaps in content or pedagogical approach; c) continually reframe the problem; d) constantly question the underlying assumptions during the design process; and e) establish their own framework to guide the design of new games.

In this chapter, the authors contribute the description of a protocol for game review used to evaluate existing educational games to inform the development process of new mathematics games. In the present research, a design team at a non-profit, educational game studio is developing new games for early algebra. As part of their design process, the team reviewed games with related content (e.g., pre-algebra, algebra, patterns, ratios, properties, functions, expressions, equations), games for similar audiences (i.e., students in grades 4–6), and games with similar pedagogical approaches (i.e., inquiry-oriented problem solving). In the remainder of this chapter, the steps of the protocol are presented in the context of the team’s learning approach and the game design model implemented in the research. Next, the outcomes from the group’s review of the algebra games are tied as a discussion piece. The final section focuses on future work that the research team will work on. The authors expect this process to be valuable to other development teams as a structured way to review games with the purpose of informing their own design process.