Selective sustained attention, or the ability to allocate perceptual and mental resources to a single object or event, is an important cognitive ability widely assumed to be required for learning. Assessing young children's selective sustained attention is challenging due to the limited number of sensitive and developmentally appropriate performance-based measures. Furthermore, administration of existing assessments is difficult, as children's engagement with such tasks wanes quickly. One potential solution is to provide assessments within an engaging environment, such as a video game. This chapter reports the design and psychometric validation of a video game (Monster Mischief) designed to assess selective sustained attention in preschool children. In a randomized controlled trial, the authors demonstrate that Monster Mischief is significantly correlated with an existing measure of selective sustained attention (rs ≥ 0.52), and more motivating for young children as almost three times more children preferred Monster Mischief to the existing measure.
TopIntroduction
Cognitive tests can provide a wealth of data. This data is beneficial not only for research scientists but also for practitioners as performance on cognitive tests can be used for diagnostic purposes. However, if participants do not complete the assessment batteries, the utility of such tests are limited. This issue is especially pertinent for young children as their enthusiasm and engagement with cognitive assessments can wane rapidly. The gamification (Deterding Deterding, Dixon, Khaled, & Nacke, 2011) of cognitive tests is one promising strategy for increasing the utility of cognitive assessments. While the simple addition of game-elements to assessments may have some value, games offer the potential to provide a richer space of assessment tasks (Holmgård, Togelius, & Henriksen, 2016). Thus, creating valid game-based assessments has considerable potential value for researchers, practitioners, and the educational system at large.
While games have been used extensively to provide intrinsic motivation for learning (Malone, 1981; for review and meta-analysis see Clark et al., 2013), there have been comparatively few studies demonstrating both the validity of games as assessments and empirical validations demonstrating that game-based assessments improve children’s intrinsic motivation. MacPherson and Burns (2008) explicitly compared game-like versions of assessments of working memory and processing speed to traditional assessments; but they did not assess whether the games improved player affect or motivation. There have also been several ambitious efforts to design batteries of game-based assessments for a range of neuropsychological constructs (including Aalbers et al., 2013; Delgado et al., 2014; Méndez et al., 2015; Rosetti et al., 2017). In many cases, these research efforts have demonstrated that games can serve as reliable assessments that correlate well with traditional neuropsychological assessments (or even predict psychiatric diagnoses like ADHD). But across these studies, no comparison was made to determine if the game-based assessments enhanced player affect or motivation. However, work by Attali and Arieli-Attali (2015) found that the addition of game-like points to an online math assessment somewhat improved task likability for middle school students while having no effect on the accuracy of responses. In a recent review of the literature on cognitive assessment and training games (Lumsden, Edwards, Lawrence, Coyle, & Munafò, 2016), only one study (Prins et al., 2011) reported behavioral measures (more voluntary completion and fewer absences) to validate the enjoyability of their game; however this was a working memory training game that was compared to a working memory training exercise. A related line of research has investigated how game data can be used to generate evidence of ability, as in Shute’s (2011) notion of “stealth assessments” or other investigations of the psychometrics of games (GlassLab, 2014). Despite the limited evidence, game-based assessments (Mislevy et al., 2012) may have the potential to support higher levels of user enjoyment while preserving assessment validity. Accordingly, we sought to redesign an existing measure of attention as an engaging video game that would maintain sufficient validity as an assessment while also increasing children’s motivation.