In this chapter, the topic of evaluating learner experience in serious games is discussed with respect to four different dimensions: gaming, learning, using and context with a special focus on using multimodal data. After reviewing relevant research fields, the steps involved in a serious games evaluation process is investigated and relevant evaluation studies are reviewed with emphasis on the use of different modalities for recording and assessing in-game interactions. Finally, a theoretical framework (LeGUC) is proposed defining parameters related to the four dimensions discussed which can be observed during evaluation studies of serious games and how they relate to logged in-game interactions. The framework is based on relevant literature as well as a conducted observational user study.
TopIntroduction
As game-based learning is becoming a more and more acknowledged learning field and with the rapid advancement in digital technologies, the use of digital educational games is on the rise. In the commercial games market, some learning games like the mathematics game Dragon Box on the Apple App Store even outperform games offering pure entertainment revenue-wise (Gilbert, 2015). Such a phenomenon, although still not common, shows the potential of educational games when designed properly, be it with the need of a practical definition of the word “properly” to make this success reproduceable. With assessment being integral to any learning process, it plays an important role in digital game-based learning, especially where the time-consuming and expensive design process needs extra justification. The features which make educational games more appealing and promising than non-gaming learning environments, for instance “intrinsic motivation” and “pleasurable frustration”, are the same aspects which make their design and, more so, evaluation, a challenging task (Mitgutsch & Alvarado, 2013). The lack of consensus regarding the best approach to combining gaming and educational theories and on methods to fully capture and analyze the rich and deep interaction experience with this interdisciplinary software makes it challenging to link certain product elements to its success or failure to improve further development cycles (Khine, 2011). Looking into established evaluation techniques of non-gaming learning environments or non-educational games adds to the confusion as they do not provide tailored methods which can be directly applied to evaluate the aspects which distinguish learning games from both product categories (Loh, Sheng & Ifenthaler, 2015). The need for more insight into the serious gameplay experience is further intensified with the emergence of novel interaction paradigms and modern educational theories and landscapes accompanying the digital revolution increasingly rendering classical evaluation methods insufficient (Mirza-Babaei et al., 2014; Shoukry, Göbel & Steinmetz, 2014b; Bosch, Hall & Dame, 2015).This need for understanding naturalistic serious play experience has led to more and more research being conducted to use multimodal methods in the evaluation of serious games. A main focus of this chapter is to investigate the ways in which the field of Serious Games evaluation can benefit from multimodal methods by reviewing current trends as well as future potential. Although there are numerous reviews on Serious Games evaluation frameworks and techniques, they lack the focus on multimodality despite its many promises and challenges (Mitgutsch & Alvarado, 2013; Gee et al., 2014; Kiili et al., 2014). A main problem in providing such an overview is the variation in terminology, research background and objectives among different researchers when discussing similar topics. This can be facilitated by starting with locating this interdisciplinary research area and related terminology in the research landscape which is done in the first section of this chapter. Serious Games apply concepts of game technology in a broad application spectrum ranging from educational, training and simulation to persuasive and social impact games or games for health. In this chapter the term Serious Games will be used to refer to digital educational games, although this is one application area of Serious Games. Because of its interdisciplinarity, the evaluation of Serious Games lies in the intersection of and benefits from metrics and strategies from several different fields. Hence, it is important to review some important fields which are considerably related to this topic.