In this chapter we present our experience with automatic dialogue act recognition using empirical methods for exploiting lexical semantics in an unsupervised framework. Moreover, we show how automatic dialogue act annotation of human-ECA (Embodied Conversational Agent) interactions may be used as a preliminary step in conversational analysis for modeling the users’ attitudes. Experiments are presented, by exploiting corpora of English and Italian natural dialogues. In both cases the approaches employed have been conceived as general and domain-independent and may be relevant to a wide range of both human-computer and human-human interaction application domains.
Top1. Introduction
In recent years, the research on intelligent interfaces has focused with great enthusiasm on developing intelligent embodied conversationalists, which are better known as Embodied Conversational Agents (ECAs). An ECA is a ‘virtual agent that interacts with a User or another Agent through multimodal communicative behavior’ (Poggi et al., 2005). It represents the system as a person and the information is conveyed to human users through multimodal behavior, using speech and hand gestures; the internal representation is modality independent and both propositional and nonpropositional (Cassell, 2001). ECAs offer to people the possibility to relate with computer media at a social level (Reeves & Nass, 2003) and, therefore, to make the interaction more natural and enjoyable.
To be successful an ECA has to be believable and should be perceived as intelligent. To this aim, it has to engage humans in face-to-face natural language interaction by properly adopting the conversation behavior governing human-human dialogues. In natural conversations people can ask for information, agree with their partner, state some facts and express opinions. They proceed in their conversations through a series of dialogue acts to yield some particular communicative intentions. Moreover, humans proved to coordinate themselves in conversations by matching their nonverbal behavior and word use (Niederhoffer & Pennebaker, 2002), and they demonstrate the ability of properly using interactional skills, i.e. knowing when to interrupt, when to give/wait for feedback, which conversational style to adopt, introducing small talk in the dialogue, understanding who is holding the floor in the conversation, etc. (Bickmore & Cassell, 2005). A natural language intelligent interface should be able to emulate these abilities and should give the user the feeling of cooperating with a virtual companion rather than just using a system.
ECAs are one of the forms in which this intelligent kind of interaction promises to be effective. According to Cassell (2001), two kinds of intelligence should be integrated into the reasoning ability of a virtual conversational character: propositional intelligence, regulating the propositional functions (domain-oriented intelligence, ability to retrieve the information that the user needs, internal representation of the knowledge in the domain etc.) and interactional intelligence, regulating interactional functions.
In addition, ongoing research on intelligent interfaces is now focusing on the role played by affective factors during the interaction and aims at developing adaptive conversational systems that can both adjust to individual differences among users and track and adapt to changes in key features of affective states that users experience during the conversation (see Section 2.2 for a review). The more a virtual character succeeds in its goal of appearing intelligent, the more the users are expected to attach anthropomorphic features to the agent and to react also affectively to their virtual interlocutor. ECAs definitely represent a great potential in this sense and should be equipped to show some forms of socio-emotional intelligence in their turn (Mazzotta et al., 2007; De Carolis et al., 2010).
In this perspective, developing a virtual conversationalist able to believably interact with humans means to develop a character endowed with the ability of:
- •
providing domain and task-oriented support to the user, by managing appropriately its domain knowledge,
- •
managing the interaction in a way that successfully matches the user expectation and,
- •
adapting the interaction style to both the affective and cognitive factors of the user state of mind.
Designing an architecture for such an ECA requires, first of all, to deal with the understanding of the conversational structure and dynamic evolution of the interaction: at every step of the interaction the agent should be able to understand who is telling what to whom (i.e. understanding the illocutionary force (Austin, 1962) of the communicative actions of the user). While not constituting per se a deep understanding of the dialogue (Cohen & Levesque, 1990), automatic dialogue act tagging is a task that certainly plays a fundamental role in this sense.