Integrating ACT-R Cognitive Models With the Unity Game Engine

Integrating ACT-R Cognitive Models With the Unity Game Engine

Paul Richard Smart (University of Southampton, UK), Tom Scutt (Mudlark, UK), Katia Sycara (Carnegie Mellon University, USA) and Nigel R. Shadbolt (University of Oxford, UK)
Copyright: © 2019 |Pages: 24
DOI: 10.4018/978-1-5225-8060-7.ch024
OnDemand PDF Download:
No Current Special Offers


The main aim of the chapter is to describe how cognitive models, developed using the ACT-R cognitive architecture, can be integrated with the Unity game engine in order to support the intelligent control of virtual characters in both 2D and 3D virtual environments. ACT-R is a cognitive architecture that has been widely used to model various aspects of human cognition, such as learning, memory, problem-solving, reasoning and so on. Unity, on the other hand, is a very popular game engine that can be used to develop 2D and 3D environments for both game and non-game purposes. The ability to integrate ACT-R cognitive models with the Unity game engine thus supports the effort to create virtual characters that incorporate at least some of the capabilities and constraints of the human cognitive system.
Chapter Preview


Cognitive architectures are computational frameworks that can be used to develop computational models of human cognitive processes (Langley et al., 2009; Taatgen & Anderson, 2010; Thagard, 2012). Cognitive architectures have been useful in terms of advancing our understanding of human cognition in specific task environments, and they have also been used to support the development of a variety of intelligent systems and agents (e.g., cognitive robots). Although a variety of cognitive architectures are available, such as SOAR (Laird, 2012; Laird et al., 1987), ACT-R (Anderson, 2007; Anderson et al., 2004) and CLARION (Sun, 2006a; Sun, 2007), the focus of the current chapter is on ACT-R. ACT-R is a rule-based system that has been widely used by cognitive scientists to model aspects of human cognitive performance. It is also one of the few cognitive architectures that has an explicit link to research in the neurocognitive domain: the structural elements of the core ACT-R architecture (i.e., its modules and buffers) map onto different regions of the human brain (Anderson, 2007), and this enables cognitive modelers to make predictions about the activity of different brain regions at specific junctures in a cognitive task (see Anderson et al., 2007)1.

Given their role in the computational modeling of cognitive processes, it is perhaps unsurprising that cognitive architectures have been used in the design of intelligent virtual characters. The SOAR architecture, for example, has been used to control a humanoid character that co-exists in a virtual 3D environment alongside a human-controlled avatar (Rickel & Johnson, 2000). The aim, in this case, is to provide a training environment in which the SOAR-controlled character possesses expertise in a particular domain of interest and then mentors human subjects as they progress through the stages of skill acquisition. This is an excellent example of the productive merger of cognitive architectures with virtual environments. As Rickel and Johnson (2000) point out, virtual tutors that cohabit a virtual environment with human players benefit from the ability to communicate nonverbally using gestures, gaze, facial expression and locomotion. In addition, the virtual agents can closely monitor the behavior of human subjects in a way that is not typically possible outside of a virtual environment; for example, a user’s actions and field of view can be carefully monitored to determine their likely focus of attention. Rickel and Johnson’s (2000) work is also a clear example where a multidisciplinary focus is required to engineer the intelligent virtual agent: the development of an intelligent virtual tutor draws on technical and scientific advances in the fields of knowledge elicitation (Shadbolt & Smart, 2015), knowledge modeling (Schreiber et al., 2000), human expertise development (Chi et al., 1988), and educational psychology.

Another example of cognitive architectures being used in virtual character design is provided by Best and Lebiere (2006). They used ACT-R to control the behavior of synthetic team-mates in a virtual environment as part of military training simulations2. As is the case with virtual tutors, the use of a virtual environment is important here because it enables human actions to be closely monitored in a way that is difficult (if not impossible) with real-world environments. As a result of such monitoring, the behavior of ACT-R-controlled virtual agents can be adjusted in ways that respect the norms and conventions of team-based behavior. This is a topic of particular interest in the context of military behavior simulations, where issues of team coordination and the synchronization of team-member responses (often in alignment with doctrinal specifications) are all-important.

Complete Chapter List

Search this Book: