Abstract
Believable agents designed for long-term interaction with human users need to adapt to them in a way which appears emotionally plausible while maintaining a consistent personality. For short-term interactions in restricted environments, scripting and state machine techniques can create agents with emotion and personality, but these methods are labor intensive, hard to extend, and brittle in new environments. Fortunately, research in memory, emotion and personality in humans and animals points to a solution to this problem. Emotions focus an animal’s attention on things it needs to care about, and strong emotions trigger enhanced formation of memory, enabling the animal to adapt its emotional response to the objects and situations in its environment. In humans this process becomes reflective: emotional stress or frustration can trigger re-evaluating past behavior with respect to personal standards, which in turn can lead to setting new strategies or goals. To aid the authoring of adaptive agents, we present an artificial intelligence model inspired by these psychological results in which an emotion model triggers case-based emotional preference learning and behavioral adaptation guided by personality models. Our tests of this model on robot pets and embodied characters show that emotional adaptation can extend the range and increase the behavioral sophistication of an agent without the need for authoring additional hand-crafted behaviors.
TopIntroduction
When we see a pet we’ve met before, we recall not just its name and temperament but how our interactions with it made us feel. We feel happy when we see the dog we had fun playing with, and feel sour about the cat that shocked us with its hiss. And just as we learn from them, they learn from us; the dog, remembering its happiness upon playing with us, may seek us out when we are down; and the cat, remembering our shocked reaction when it hissed, may avoid us, or be more cautious with its anger in the future. Pets don’t need to be ‘configured’ to live with us, and neither do we: all we need is the ability to react emotionally to our situations, a memory for our past emotional states, and the ability to let those recalled emotions color our current emotional state and guide our behaviors appropriately. We argue that robots and synthetic characters should have the same ability to interpret their interactions with us, to remember these interactions, and to recall them appropriately as a guide for future behaviors, and we present a working model of how this can be achieved.
Of course, humans are more complicated than pets; we have not just emotions but also ideals for our behavior, and can modify our reactions and plans when they violate our ideals. We may snarl back at the hissing cat, but that outburst of emotion can make us reconsider when we should show anger. Even if we do not reconsider at first, if we see the same cat multiple times we may eventually be prompted to figure out why it continues to try to enter our new home, to realize it was probably abandoned, and to change our routines to leave food for it – turning a hissing cat into a new companion. It may seem a tall order make robots have this kind of flexibility – but we argue it is possible by using emotion to trigger behavior revision guided by a personality model, and we present a working model of how it can be achieved.
In this chapter, we review efforts to build agents with believable personalities, point out problems particular to making these personalities convincing over long-term interactions with human users, and discuss research in cognitive science into the mechanisms of memory, emotion, and personality. Based on these psychological results, we present a method for building believable agents that uses emotion and memory to adapt an agent’s personality over time. We then present two case studies illustrating this idea, the first demonstrating emotional long term memory in a robot, and the second demonstrating emotion-driven behavioral updates in an embodied character. Finally, we conclude with lessons learned.
Key Terms in this Chapter
Blame Assignment: In learning and adaptation, blame assignment is the process of identifying the causes of a failure of a computational system to deliver the behaviors desired of it.
ABL (A Behavior Language): ABL is a programming language explicitly designed to support programming idioms for the creation of reactive, believable agents (Mateas and Stern, 2004). ABL has been successfully used to author the central characters Trip and Grace for the interactive drama Facade (Mateas and Stern, 2003). The ABL compiler is written in Java and targets Java; the generated Java code is supported by the ABL runtime system.
Appraisal: In the OCC (Ortony et al. 1988) and Frijda (1993) models of emotion, appraisal matches the experience of an agent against its goals, standards, preferences and other concerns. The results of this matching give emotional events their positive or negative feeling or weight, called affect, and can also place this affective response in context.
SEU (Subjective Expected Utility Theory): Subjective expected utility theory (Simon 1983) holds that a rational agent should attempt to maximize its reward by choosing the action with the highest expected utility — effectively, the sum of the rewards of the outcomes discounted by the probabilities of their occurrence.
Concern: In Frijda’s (1986) model of emotion, concerns correspond to the needs, preferences and drives of an agent – things that “matter” and can trigger changes to the emotional state of the agent.
OCC Model of Emotion: Ortony, Clore and Collins’s (Ortony et al. 1988) model of emotion is a widely used model of emotion that states that the strength of a given emotion primarily depends on the events, agents, or objects in the environment of the agent exhibiting the emotion. A large number of researchers have employed the OCC model to generate emotions for their embodied characters. The model specifies about 22 emotion categories and consists of five processes that define the complete system that characters follow from the initial categorization of an event to the resulting behavior of the character. These processes are namely a) classifying the event, action or object encountered, b) quantifying the intensity of affected emotions, c) interaction of the newly generated emotion with existing emotions, d) mapping the emotional state to an emotional expression and e) expressing the emotional state.
Case-Based Reasoning: Case-based reasoning (Kolodner 1993) is a reasoning architecture that stores experiences with lessons learned as cases in a case library and solves problems by retrieving the case most similar to the current situation, adapting it for reuse, and retaining new solutions once they have been applied. Case-based reasoning is also a pervasive behavior in everyday human problem solving.
MDP (Markov Decision Processes): Markov decision processes provide a mathematical framework for modeling decision-making characterized by a set of states where in each state there are several actions from which the decision maker must choose and transitions to a new state at time t + 1 from time t are only dependent on the current state and independent of all previous states. MDPs are useful for studying a wide range of optimization problems solved via dynamic programming and reinforcement learning.