Cognitive Modeling of Learning Using Big Data From a Science-Based Game Development Environment

Cognitive Modeling of Learning Using Big Data From a Science-Based Game Development Environment

Leonard Annetta (East Carolina University, Greenville, USA), Richard Lamb (East Carolina University, USA), Denise M. Bressler (East Carolina University, USA) and David B. Vallett (Quebec Quantitative Qualitative Research and Evaluation, Canada)
Copyright: © 2020 |Pages: 18
DOI: 10.4018/IJGBL.2020100102
OnDemand PDF Download:
No Current Special Offers


The purpose of this study was to identify the underlying cognitive attributes used during the design and development of science-based serious educational games. Study methods rely on a modification of cognitive diagnostics, item response theory, and Bayesian estimation with traditional statistical techniques such as factor analysis and model fit analysis to examine the data and model structure. A computational model of the cognitive processing using an artificial neural network (ANN) allowed for examination of underlying mechanisms of cognition from a server-side data set and a 21st century skills assessment. ANN results indicate that the model correctly predicts successful completion of science-based serious educational game (SEG) design tasks related to 21st century skills 86% of the time and correctly predicts failure to complete SEG design tasks related to 21st century skills 78% of the time. The model also reveals the relative importance of each particular cognitive attribute within the 21st century skills framework.
Article Preview


The skills, knowledge, and expertise necessary for today’s work-life include critical thinking, communication, and collaboration (Battelle for Kids, 2019). Research shows that the workforce is drastically ill-prepared in such expertise; companies need to train their workers in these 21st century skills (Casner-Lotto, Rosenblum, & Wright, 2009). In order to properly prepare students for today’s workforce, there is a considerable pressure on educators to teach and for students to learn 21st century skills. In response to these calls, many educational institutions P through 20 have incorporated into their mission statements a call to action to teach 21st century skills in their classrooms.

One educational practice that holds promise for promoting 21st century skills is game-based learning. Qian and Clark (2016) reviewed the literature to date and determined that game-based learning has the potential to be an effective way for students to develop 21st century skills. Serious educational games (SEGs) are specifically designed for teaching and learning purposes (Annetta, 2008)—they are showing potential to foster specific 21st century skills such as critical thinking (Cicchino, 2015), collaborative problem-solving (Li & Tsai, 2013), and social/cultural skills along with technology literacy skills (Romero, Usart, & Ott, 2015). Some researchers argue that—even more than playing SEGs—game construction and authorship of SEGs can foster 21st century skills (Navarrete, 2014; Thomas, Ge, & Greene, 2011; Yang & Chang, 2013). Ultimately though, according to Qian and Clark (2016), the field lacks high-quality empirical evidence for how games can support 21st century skill development.

There is also a dearth of research on assessment of learning in SEGs (Hummel, Brinke, Nadolski, & Baartman, 2017); however, a few labs have started research in this area. In order to understand students' implicit understanding of physics and identify common errors, Rowe et al. (2017) mined data from gameplay and analyzed it. Shute, Wang, Greiff, Zhao, and Moore (2016) even designed in-game stealth assessments to measure students’ problem-solving skills. Experts posit that SEGs offer an opportunity to collect multi-dimensional data that will help us assess 21st century skills which are generally difficult to assess (Cassie, 2018; Groff, 2018).

With increased pressure from business and world competition, the science education community has entered a new era in assessment. Now, it is far more important for students to apply their knowledge of science content in the coming years rather than just know it (Bertram & Loughran, 2011); in other words, students need to demonstrate 21st century skills. This new era of assessment arises from increases in the use of “big data” (data with greater than 100,000 data points from multiple sources) derived from computer-based informatics and analytics arising from the play and design of science-based SEGs (Annetta, 2008). The use of networked educational technology in the classroom is uniquely suited to take advantage of the revolution in educational information through novel application of data mining, measurement, and cognitive diagnostic techniques. The new era of assessment provides a means to conceptualize the roll of cognition in science learning.

Complete Article List

Search this Journal:
Open Access Articles
Volume 12: 4 Issues (2022): 1 Released, 3 Forthcoming
Volume 11: 4 Issues (2021)
Volume 10: 4 Issues (2020)
Volume 9: 4 Issues (2019)
Volume 8: 4 Issues (2018)
Volume 7: 4 Issues (2017)
Volume 6: 4 Issues (2016)
Volume 5: 4 Issues (2015)
Volume 4: 4 Issues (2014)
Volume 3: 4 Issues (2013)
Volume 2: 4 Issues (2012)
Volume 1: 4 Issues (2011)
View Complete Journal Contents Listing