Applying Learning Analytics Approaches to Detect and Track Students' Cognitive States During Virtual Problem-Solving Activities

Introduction

Problem-based learning (PBL) is a learner-centered pedagogy that exposes students to an ill-structured problem-solving scenario (Permatasari et al, 2019). In PBL, students are granted full autonomy to apply their skills and knowledge in developing viable solutions (Savery, 2006). Meanwhile, teachers adopt the role of facilitators, assisting students in problem-solving (Horak & Galluzzo, 2017). This new dynamic between students and teachers creates distinct expectations for teachers, who are not meant to provide direct instructions but rather scaffoldings to support problem-solving (Kim & Hannafin, 2011). However, extensive research on PBL has indicated that teachers often encounter challenges in delivering real-time and individualized interventions (Dolmans & Gijbels, 2013; Kim et al., 2019; Nariman & Chrispeels, 2016). Specifically, when facilitating PBL activities, they struggle with determining whom to assist and when (Hmelo-Silver, 2004; Liu et al., 2020; Spronken-Smith & Harland, 2009).

The utilization of learner activity data collected from virtual PBL environments offers potential solutions to address this issue (Baresh et al., 2019). When students engage in problem-solving activities within virtual environments, a substantial amount of usage logs or trace data (Winne, 2013) is generated. This data serves as valuable evidence for teachers to comprehend students' progress in problem-solving and determine when and how to provide appropriate support (Lang et al., 2017; Winne, 2011). For instance, in a study by Saqr and López-Pernas (2023), temporal learning analytics were applied to analyze the sequential and temporal aspects of students' interactions at both the group and individual levels when they were engaging in an online PBL activity. This study shed light on providing scaffoldings to sustain cognitive interaction in an online setting. In another study, researchers explored the associations between students’ behavioral patterns and self-efficacy (Liu et al., 2023), the results revealed that students with varying levels of self-efficacy had correlated success rates in problem-solving. These findings hold practical implications for teachers aiming to maintain self-efficacy during problem-solving, such as dedicating more time to guide students at the beginning of the PBL process. While the application of LA techniques has revealed potential for providing teachers with valuable information for facilitating PBL, teachers may encounter challenges when attempting to access and interpret the vast and complex log data generated directly by the system. To enhance teachers' ability to make sense of this data, learning analytics dashboards (LADs) have gained significant attention in recent years. LADs are defined as “a single display that consolidates various indicators about learners, learning processes, and learning contexts into one or multiple visualizations” (Schwendimann et al., 2016, p. 37). According to Ruipérez et al. (2015), LADs have the capacity to automatically process large amounts of log data and present interpretable outcomes directly to instructors and learners. In addition, -previous studies have highlighted the need for further investigation into processing real-time log data on a large scale (Molenaar & Campen, 2018; Rienties et al., 2018) and tracing students' cognitive processes based on log activities (Rogers et al., 2016). Previous studies investigated providing feedback or scaffolding in learning contexts, however, more studies about how to apply log data in supporting teachers in the PBL contexts, and specifically, using LAD, is needed.