A Protocol for Reviewing Off-the-Shelf Games to Inform the Development of New Educational Games

A Protocol for Reviewing Off-the-Shelf Games to Inform the Development of New Educational Games

Ruth Torres Castillo (New Mexico State University, USA) and Sara Morales (New Mexico State University, USA)
DOI: 10.4018/978-1-5225-9069-9.ch003
OnDemand PDF Download:
No Current Special Offers


This chapter shares a protocol for reviewing games and documents the process in which it was used by an educational game design team for evaluating existing games to inform the design and development of new games for early algebra. While the design team has used their own learning games design model to develop several games—all of which included some kind of immersive learning and review activity—there has been no documentation provided on the specific processes used to review games as part of that immersion. Observations offer structured means for assessing existing games in a particular space and are thus valuable to identify how best to pursue the alignment of learning objectives with teaching content and game mechanics in the development of educational games.
Chapter Preview


A number of design process models specify methods for developing educational games, highlighting unique design concerns in this space (Chamberlin et al, 2012; De Freitas and Neumann, 2009; De Freitas and Oliver, 2006; De Freitas and Routledge, 2013; Dormann and Biddle, 2008; Groff, J. et al, 2015; Gunter, G.A. et al, 2008; Kiili, K. et al, 2012; Mislevy and Haertel, 2006). In some of these models, a design step is to review existing games, not only to understand the market space, but also to identify design principles and good pedagogy (Chamberlin et al, 2012; De Freitas and Routledge, 2013; Dormann and Biddle, 2008). Existing game review models (Petri, G. et al, 2016; Lucero, A. et al, 2014; Sweetser and Wyeth, 2005) evaluate games based on the player’s experience, or analyze them from the perspective of a learning theory, such as flow (Csikszentmihalyi, 1996). While it is important to review games for design purposes, little information is given regarding the structure of the process and the ways this process differs from reviewing games for other reasons.

Game review is often a way to assess games for use in a classroom, curate a list of educational games, or identify what games are available in a content area for a certain age group. While it is important to review games to identify those that are of value for a specific type of player, it is also critical to review games as a way of identifying gaps in content and pedagogy. This type of game review can identify the need for new games, dictate what types of new games are needed, and might suggest ways in which games can present content in novel ways.

As part of a design process, the performance of game review serves another purpose when done collaboratively: it engages the design team in a deeper level of pedagogical understanding about the content and design of the games. Effective educational game design teams should represent diverse backgrounds and disciplines (Chamberlin et al, 2012), and it is important that all team members be well played (Being “well played” refers to game literacy, like being “well read”: having a diverse knowledge and experience of playing games) (Davidson, 2009). Further, the process brings the team together in articulating a shared expertise vision for addressing game content, identifying valuable game mechanics, and deciding the best approaches for how a game should work. When interdisciplinary design teams review games with a variety of design lens, they: a) look at the overall quality of a game; b) identify gaps in content or pedagogical approach; c) continually reframe the problem; d) constantly question the underlying assumptions during the design process; and e) establish their own framework to guide the design of new games.

In this chapter, the authors contribute the description of a protocol for game review used to evaluate existing educational games to inform the development process of new mathematics games. In the present research, a design team at a non-profit, educational game studio is developing new games for early algebra. As part of their design process, the team reviewed games with related content (e.g., pre-algebra, algebra, patterns, ratios, properties, functions, expressions, equations), games for similar audiences (i.e., students in grades 4–6), and games with similar pedagogical approaches (i.e., inquiry-oriented problem solving). In the remainder of this chapter, the steps of the protocol are presented in the context of the team’s learning approach and the game design model implemented in the research. Next, the outcomes from the group’s review of the algebra games are tied as a discussion piece. The final section focuses on future work that the research team will work on. The authors expect this process to be valuable to other development teams as a structured way to review games with the purpose of informing their own design process.

Key Terms in this Chapter

Affordances: The qualities or properties of a game/tool that define its use or purpose as they are perceived on the way users see and experience the game/tool.

Researcher: A person, who performs systematic investigation, oversees internal research and often serves as lead author on reports and articles. This person will also direct development of instruments and lead validation of research.

Game Designer: A person who oversees the production of tools, guides formative product testing and leads the design process. Often a game designer is also its developer (as is the case for this study), so may be referred to as a game developer.

Game Developer: A person who fleshes out the details of a game’s design, oversees its testing, and revises the game in response to player feedback.

Game Artist: A person with art training who creates art for games.

Content Expert: A person with special knowledge or skills in a particular subject area or learning topic who informs content and pedagogical decisions during research and development.

Embedded Data: Any gameplay information that would likely be recorded through logging systems.

Complete Chapter List

Search this Book: