Integrating ACT-R Cognitive Models with the Unity Game Engine

Introduction

Cognitive architectures are computational frameworks that can be used to develop computational models of human cognitive processes (Langley et al., 2009; Taatgen & Anderson, 2010; Thagard, 2012). Cognitive architectures have been useful in terms of advancing our understanding of human cognition in specific task environments, and they have also been used to support the development of a variety of intelligent systems and agents (e.g., cognitive robots). Although a variety of cognitive architectures are available, such as SOAR (Laird, 2012; Laird et al., 1987), ACT-R (Anderson, 2007; Anderson et al., 2004) and CLARION (Sun, 2006a; Sun, 2007), the focus of the current chapter is on ACT-R. ACT-R is a rule-based system that has been widely used by cognitive scientists to model aspects of human cognitive performance. It is also one of the few cognitive architectures that has an explicit link to research in the neurocognitive domain: the structural elements of the core ACT-R architecture (i.e., its modules and buffers) map onto different regions of the human brain (Anderson, 2007), and this enables cognitive modelers to make predictions about the activity of different brain regions at specific junctures in a cognitive task (see Anderson et al., 2007)¹.

Given their role in the computational modeling of cognitive processes, it is perhaps unsurprising that cognitive architectures have been used in the design of intelligent virtual characters. The SOAR architecture, for example, has been used to control a humanoid character that co-exists in a virtual 3D environment alongside a human-controlled avatar (Rickel & Johnson, 2000). The aim, in this case, is to provide a training environment in which the SOAR-controlled character possesses expertise in a particular domain of interest and then mentors human subjects as they progress through the stages of skill acquisition. This is an excellent example of the productive merger of cognitive architectures with virtual environments. As Rickel and Johnson (2000) point out, virtual tutors that cohabit a virtual environment with human players benefit from the ability to communicate nonverbally using gestures, gaze, facial expression and locomotion. In addition, the virtual agents can closely monitor the behavior of human subjects in a way that is not typically possible outside of a virtual environment; for example, a user’s actions and field of view can be carefully monitored to determine their likely focus of attention. Rickel and Johnson’s (2000) work is also a clear example where a multidisciplinary focus is required to engineer the intelligent virtual agent: the development of an intelligent virtual tutor draws on technical and scientific advances in the fields of knowledge elicitation (Shadbolt & Smart, 2015), knowledge modeling (Schreiber et al., 2000), human expertise development (Chi et al., 1988), and educational psychology.

Another example of cognitive architectures being used in virtual character design is provided by Best and Lebiere (2006). They used ACT-R to control the behavior of synthetic team-mates in a virtual environment as part of military training simulations². As is the case with virtual tutors, the use of a virtual environment is important here because it enables human actions to be closely monitored in a way that is difficult (if not impossible) with real-world environments. As a result of such monitoring, the behavior of ACT-R-controlled virtual agents can be adjusted in ways that respect the norms and conventions of team-based behavior. This is a topic of particular interest in the context of military behavior simulations, where issues of team coordination and the synchronization of team-member responses (often in alignment with doctrinal specifications) are all-important.