The literature on using serious games for learning has been growing exponentially during the last decade. It is time to examine some methodological issues associated with this line of research. There is evidence that research on serious games, if designed with methodological rigor and executed properly, such as the serial studies of prisoners’ dilemma, modality effect in individual interactive learning, and changes in attitude toward mathematics in a computer-based simulation game, can be fruitful and have a profound, positive impact on learning and training. Since adopting serious games as an educational technology tool is by no means cheap, we should ensure that methodological issues are carefully considered before conducting a study on educational games. Whereas there are excellent studies in the existing literature of simulations and games, it is not uncommon for some studies to adopt convenience samples or own-control designs. Studies on serious games tend to be conclusive if they have used true experimentation, well-controlled quasi-experimental design, surveys with representative samples and validated instruments, comprehensive design research, or training programs having a pretest–posttest design with group comparisons. The potential values and informative contributions of using different methodologies for serious game research should be recognized because of the ecological relevance. Future research should pay more attention to randomized sampling, controlled but feasible research design, validity of instruments, appropriate analytical methods, and interdisciplinary or cross-disciplinary research to enhance the internal/external validity of various approaches. In regard to analytical methods, both quantitative approaches and qualitative evaluations, if applied appropriately, are considered as valuable, indispensable, and complementary to each other. It is hoped that this chapter can be helpful not only for future researchers in this field to design and execute rigorous projects but also for wider readership to understand and evaluate research outcomes in the discipline of serious games.
TopIntroduction
Although game playing has long been used for studies on conflict resolution, business training, group psychotherapy and children’s experience (e.g., Charness, Fréchette, & Qin, 2007; Graetz, 1995; Kay, 1997; Kirova, 2006; Pratto, Pearson, Lee, & Saguy, 2008;Pruitt & Kimmel, 1977; Vinacke, 1969; Wiener, 1999; Zhong, Loewenstein, & Murnighan, 2007; Zizzo & Tan, 2007), educational games equipped with advanced informational technology for the learning of a wide range of academic knowledge and skills have attracted researchers’ attention only during the last two decades or so. In line with the proliferation of personal computers and internet usage across nations and cultures, a number of conceptual models as well as guidelines for the design and execution of educational computer games have been proposed (e.g., Dempsey, Haynes, Lucassen, & Casey, 2002; Dipietro, Ferdig, Boyer, & Black, 2007; Garris, Ahlers, & Driskell, 2002; Jones, 2007; Karakus, Inal, & Cagiltay, 2008; Kebritchi & Hirumi, 2008; Raybourn, 2007; Reese, 2007; Spencer-Oatey, 2007; Tahiroglu, Celik, Uzel, Ozcan, & Avci, 2008). In line with Moore’s Law predicting periodically remarkable increases in the number of transistors economically available on an integrated circuit, the advancement of software designed for computer games is partially due to the rapid expansion of processing capabilities of CPUs. Consequently, research activities on both games for purely entertainment purposes and serious games for educational/training purposes have been growing exponentially. While educational and technological experts are still debating the theoretical bases of gaming and its pedagogical implications, methodological concerns in this diversified field have also been raised. According to de Freitas and Jarvis (2007), many studies on digital game-based learning (DGBL) lack firm empirical evidence and thus are not conclusive. In his assessment of serious games as a field and the associated challenges in forthcoming years, Van Eck (2007) calls for more rigorous research of both a grounded-theory and empirically oriented nature. As pointed out by Dipietro and colleagues (2007) in their recent effort to promote a framework for understanding electronic educational gaming: “We need more research, but this research must be structured and rigorous” (p. 241).