This chapter will discuss the growing importance of applying considered rationales to which games are chosen for study, whether it be for ethnography, classroom use, or anything else. A brief overview of how games are currently being chosen for study is presented through a meta analysis of studies with games that were published between 2003 and 2006 in order to demonstrate that most published games studies do not include a supported rationale for the games chosen. The chapter will then present various ways that game choices can be justified, and propose and explain a data fusion technique that can be applied to game reviews and other lists in order to facilitate representative and defensible game choices.
Why is it important to justify the choice of games being used as an example in a scholarly article or for the purposes of study? In the early days of games studies, there seemed little call for careful scrutiny of one’s game choices. We studied what we had handy and wrote about the games we were already playing. However, if we want to make the case that the game in question is good by some measure (however we decide to define “good”), then we really should have some evidence to back this up. When a single game or a small number of games are chosen as the subject(s) of study, they form part of the bounded system that is the case being examined, and also form part of what makes the case of special interest (Stake, 1995). If we are proposing the use of a game in the classroom or the study of some specific game to learn something applicable to our agenda, whether that agenda is to examine the educational potential of the game or to learn something else about the game that may inform other instruction, then as academics we have a responsibility to explain why that game is suitable for our purpose.
One reason for putting thoughtful effort into justifying the choice of a game used in a study is that it helps to make the study itself more credible. This has implications for the increased acceptance of games studies academically as well as for helping to improve relations between academia and the games industry. In a recent article offering suggestions for how the Academy of Interactive Arts and Science could build stronger ties with the games industry, Hopson (2006) argues that we should:
…(u)se examples from bestsellers. A good example from a popular game is more effective than a great example from something they’ve never heard of. Industry people often suffer from an ‘if-they’re-so-smart-, why-ain’t-they-rich’ attitude towards smaller titles. Even if the small title is a perfect example of how the theory works, they’re going to be less likely to listen if they haven’t heard of the game ahead of time. Commercial success is one way of making sure that the audience will respect your examples, but you can also use titles that are well known or critically acclaimed but which weren’t necessarily huge blockbusters. It’s also important to keep your examples as current as possible, because many industry folks will see a three-year-old example as ancient history.
Critical and commercial success are key recognizable and accepted (albeit subjective) measures of a game’s popularity, and that popularity in turn gives some indication of that game’s perceived quality as judged by players, developers, and game critics. When it comes to resources that are primarily creative or artistic in nature, subjective measures are often the only ones we have. In sports for example, such as sprinting, determining who the fastest sprinter is can be done quite objectively—it is a matter of comparing competition times, and the runner with the fastest time wins; no such objective measure exists for most creative endeavors, and since games are creative designs, we can only produce subjective measures. To further compound the problem, lists of ‘top games’ tend to be quite unstable and change not only from year to year as new titles gain recognition, but sometimes from day to day as in review sites where players can contribute. One consequence of this is that no single list can reasonably be used to support claims about a particular game’s qualities. One solution is to combine multiple lists into one comprehensive one. By combining multiple lists, we can increase our confidence in the qualifications of games that end up on top. However, the challenge in combining measures from these various sources is that the criteria used to produce lists of ‘good’ games are often so divergent that they cannot be compared or combined directly. Categories and scores vary, the methodology used to rate and rank the games varies, even the contributors vary—in some cases they are paid professional critics; in other cases association members or even the public at large contributes votes and reviews. The data fusion technique described in this chapter offers a solution to this problem that is both verifiable and repeatable. Combining a number of different measures to come up with a single measure ensures that games that end up at the top of the final list qualify as successful by more than one measure and have been assessed by more than one source. Using a systematic approach to ranking games results in a list with which most (industry, gamers, and critics) could agree.
Key Terms in this Chapter
Voting Strategy: A mathematical system for merging decisions about choices among several alternatives that come from multiple sources.
T (Teen): Content that may be suitable for ages 13 and older.
E (Everyone): Content that may be suitable for ages 6 and older.
IDGA (International Game Developer’s Association): The premier association for people involved in the game development industry. See more at http://www.igda.org/.
First-Person Shooter: A game played from the first-person perspective where the game space is seen from a position slightly behind and over the shoulder of the character being played. The player takes on the role of one of the game characters, and the primary mode of game-play involves the use of weapons that are used to shoot opponents.
EC (Early Childhood): Suitable for ages 3 and older.
Qualitative Meta-Analysis: An analysis of the methods used in a collection of studies rather than the results found by those studies.
ESRB (Entertainment Software Rating Board): A non-profit, voluntary regulatory body that assigns ratings and enforces advertising policies in interactive entertainment software. This is the body responsible for defining the ratings found on most commercial video games. The ratings are affected by the level of violence in the game as well as the subject matter. A summary of the ratings is included here, but for a detailed description, see the ESRB Web site at http://www.esrb.org/ratings/index.jsp:
E10+ (Everyone 10 and older): Content that may be suitable for ages 10 and older.
Game-Play: The experience of playing a game.
Backyard Research: A term used to describe research conducted in an environment in which the researcher already holds another role. An example would be a classroom teacher conducting research within her own classroom.
Longitudinal Study: A research study that involves repeated observations over long periods of time, usually including the same items, which are often correlated.
Normalized Count: The count in a list divided by the total number of observations. In the method described in this chapter, the normalized count is the score associated with a game that relates to its position in that list. The number is normalized so that the first-place game of any list will have the same score, thereby contributing the same weight towards its total. In other words, it makes the first-place game in each list worth the same regardless of the actual length of the list.
Borda Count: A well-known methodology for assigning scores to multiple ranked lists that can then be combined to produce a single ranked list that incorporates the results of the other lists.
RP (Rating Pending): Submitted to the ESRB and awaiting final rating. (This symbol appears only in advertising prior to a game’s release.)
Role-playing Game: A game usually played from the first-person perspective where the player pretends to be one of the characters in an unfolding story. Roles may be assigned with little flexibility, for example playing James Bond in Goldeneye 007, or with a great deal of player input, such as in World of Warcraft where players may choose the gender, race, and profession of their characters as well as many other variables.
Data Fusion: The process of combining data from multiple sources into some form of coherent data set.
AO (Adults Only): Content that should only be played by persons 18 years and older.
M (Mature): Content that may be suitable for persons ages 17 and older.