Integrating Assessment, Feedback, and Learning Analytics in Educational Games: Literature Review and Design of an Assessment Engine

Introduction

Games-based learning (GBL) is increasingly used as a supplementary tool for education. Various institutions use GBL for learning and training, ranging from schools (Kiili & Ketamo, 2017; Meluso, Zheng, Spires, & Lester, 2012) and higher education institutions (Cózar-Gutiérrez & Sáez-López, 2016) to healthcare (Lv, Esteve, Chirivella, & Gagliardo, 2017; Sliney & Murphy, 2008) and the army (W. L. Johnson, 2007; Zyda, 2005). A number of studies about the use of games-based learning (GBL) in education show empirical evidence of increased motivation and engagement (Annetta, Minogue, Holmes, & Cheng, 2009; Kiili & Ketamo, 2017; Rosas et al., 2003) as well as learning and retention (Gander & Parkway, 2000; Girard, Ecalle, & Magnan, 2013; Squire, Barnett, Grant, & Higginbotham, 2004). Using educational games to assist the learning process offers a wide range of possibilities that can be difficult to attain in a traditional classroom; for example, GBL gives players the possibility of going at their own pace and learning through trial and error in a controlled and safe environment.

Assessment and feedback are fundamental in a teaching and learning process. Learners rely on them to measure their progress and improve, and educators need assessment to determine whether their learning goals have been achieved. GBL also offers many options for assessment and feedback; for example, using real-time assessment data, an educational game can be adapted to a learner’s needs (Conlan, Hampson, Peirce, & Kickmeier-Rust, 2009; Kickmeier-Rust, Hockemeyer, Albert, & Augustin, 2008; V. Shute, Ke, & Wang, 2017). GBL also offers the possibility of formative assessment and feedback according to a player’s actions (Jarvis & de Freitas, 2009). In addition, an educational game can have assessment logic embedded into its core mechanics and offer non-invasive assessment (Kickmeier-Rust et al., 2008; V. Shute et al., 2017).

However, while many teachers agree that GBL increases motivation towards learning (Sandford, Ulicsak, Facer, & Rudd, 2006) and despite the evidence that games are valid assessment tools (Harteveld & Sutherland, 2015), there seems to be a lack of trust in an educational game’s assessment (Sandford et al., 2006; Serrano-Laguna, Torrente, Moreno-Ger, & Fernández-Manjón, 2012). Teachers need to feel in control before introducing a new tool in the classroom and there is a need for ownership over the game (Ketelhut & Schifter, 2011); without control, educators might feel threatened by a game rather than supported by it.

The main limitation of GBL is that educational games are too often distributed as “black-boxes”; they are closed and self-contained systems, making it difficult to modify or retrieve data from (Serrano-Laguna et al., 2017). This can mean that the potential of the game is reduced for two main reasons. First, teachers cannot improvise or adapt a game to their students’ needs as they would do in traditional teaching (Hunt, 1976). Second, they cannot retrieve data about the gameplays to appreciate whether their teaching goals have been met. Educators and researchers have very little insight about what the students learn through a computer game and how they interact with it. Learning Analytics (LA) is an emerging field based on data mining processes (Siemens & Gasevic, 2012) that can provide such detailed reports about the gameplays; data from the gameplays of several educational games are collected and data mining algorithms allow conclusions to be drawn about the games and the players. However, due to the novelty of the field, presently very few papers exist on LA and its application in GBL and LA is still beyond the reach of most teachers (L. Johnson et al., 2013).