A main opportunity provided by digital learning environments is the ability to not only examine the final products of learning activities (e.g., essays, test scores, final answers to problems), but also the detailed logs of how learners interact with the environment itself. Those logs of the learners' actions serve as breadcrumbs marking the path they take as they engage with the environment, providing fine-grained information about when and how they interact with specific components of its user interface. The emerging fields of learning analytics and educational data mining have taken a particular interest in studying how we can make sense of those fine-grained interactions to better inform us of digital learners' experiences and how we can provide new opportunities to better support learners as they engage with digital learning environments. This chapter discusses how those fine-grained logs can be analyzed to identify high-level behaviors, investigate their relationships with learning, and provide us with insights about how to adapt learning environments to learners' needs.
TopIntroduction
One of the main opportunities provided by digital learning is the capability not only to examine the final products of learning activities (e.g., essays, final answers to problems and test scores), but also to collect detailed logs of how learners participate in learning activities. These logs can be especially valuable when digital learning takes place outside of a traditional classroom, where it would be difficult or impossible for an instructor to monitor learners as they engage with their learning activities—for example, when completing after class homework or when participating in asynchronous online courses.
Action logs consist of records of all the actions that the learners execute within a learning environment. These action logs are comparable to invisible breadcrumbs left behind by learners, marking the path they took as they engaged with the environment and providing fine-grained information about when and how they interacted with specific components of the environments’ user interface. Studying and examining these breadcrumbs can be a valuable source of information, allowing us to follow the learners through their learning experiences in order to better understand and support them.
This chapter discusses how logs of learners’ actions within digital learning environments can provide us with information about their learning experiences. It starts by describing in more detail what action logs are and what kind of information they contain. Then, it discusses two research communities, Educational Data Mining (EDM) and Learning Analytics (LA), who are particularly interested in the usage of action logs to study learning and digital learning environments. This is followed by selected examples of how action logs can be analyzed and how the result of those analyses can be used to support learning.
Action Logs
A unique aspect of digital learning environments is their capability to collect detailed logs of the learners’ actions. Generally, such logs will contain a list of the meaningful actions that each learner executed within the digital learning environments, accompanied by the exact times at which each action occurred and any other important variables that are needed to describe the action that occurred. As each digital environment allows for different sets of meaningful action, the format and content of action logs can greatly vary between the different environments. To better illustrate what type of information interaction logs can contain, three examples, from three different types of digital learning environments, are presented and an explanation of what their action logs might contain is provided: Cognitive Tutor Algebra (Koedinger & Corbett, 2006), an intelligent tutoring system in which learners solve mathematical problems; Physics Playground (Shute et al. 2013), an educational game in which learners construct simple machines and use principles of Newtonian physics in order to guide a ball to a balloon; and massive open online course (MOOC) platforms such as Coursera1 and EdX2.
Example #1: Cognitive Tutor Algebra
Cognitive Tutor Algebra (Koedinger & Corbette, 2006) is an intelligent tutoring system, in which learners practice solving algebra problem. Solving a problem within Cognitive Tutor Algebra requires learners to complete multiple steps towards computing the final answer. The tutor evaluates the correctness of each step as soon as the learner completes them, identifying whether a step is correct, contains a known error (called a bug), or is otherwise incorrect, and provides immediate feedback to the learner. In addition, at any moment learners have the option of requesting hints about how to execute the next step of the problem-solving process.
As the number of types of actions that can be executed in Cognitive Tutor Algebra is very limited—an action can either be a step or a hint request—the associated action logs can be kept simple. Each entry in the log will provide information about one action, its type (step attempt or hint request), the exact time at which the action happened and, if the action is a step attempt, the name of the step that was attempted and how the tutor evaluated it (correct, bug, or incorrect). This simple format allows for the reconstruction of a learner’s attempt at solving a problem within Cognitive Tutor Algebra.