This chapter describes a collection of experiences and recommendations related with the design and evaluation of interactive applications integrating Embodied Conversational Agents (ECA) technology in real environments of use with children in Special Education. Benefits and challenges of using ECAs in this context are presented. These benefits and challenges have guided the creation of Special Education reinforcement applications incorporating ECAs, which have been used for extended periods of time at Infanta Elena Special Education School in Madrid. Co-design principles were applied in the development of two of the applications discussed here, with the participation of the school’s teaching staff and children with severe motor and mental disabilities (mainly with cerebral palsy). From the design experience a set of recommendations and observations were extracted, which the authors hope may serve as guidance for the scientific and educational communities when undertaking further research. For example, in an application to reinforce the learning of emotions it believe it beneficial to include ECAs that display a number of exaggerated facial expressions together with a combination of auditory and gestural reinforcements. The ECA should show its eyes and mouth clearly, in order to help the children focus their attention. These and other ECA strategies have been analysed to provide reinforcement in learning and also to attract the children’s attention when interacting with the application.
TopIntroduction
Special Education addresses the specific needs of children with disabilities so that they may achieve the maximum possible degree of personal development. Computer-based interactive multimedia technologies can be used to reinforce learning in children with special educational needs (Vazquez & Rota, 2002) and to improve their social integration (Kiung et al., 2008), their affective communication skills and their ability to express emotions (Baron-Cohen et al., 2009), (Picard, 2009).
One interaction tool that is particularly promising is the Embodied Conversational Agent, or ECA. An ECA is an animated human-like avatar capable, to some degree, of engaging in conversation with real human users. This involves the ability to understand and generate speech, hand movements and facial expressions (Cassell, 2000). It is possible to give ECAs expressiveness and social capabilities that make them helpful in educational contexts (see, e.g., (Wik & Hjalmarsson, 2009)).
Noteworthy educational benefits of ECAs have been identified in the literature. For instance, a virtual agent can be designed to play different roles vis-à-vis the user. This is useful for mimicking peer tutoring, with an ECA in the role of peer, which takes advantage of on the observation that children pay more attention when another child explains something to them than when the teacher does so (Bolich, 2001). It has also been shown that the rapport between children and ECAs contributes to improving communication efficiency, increasing motivation and interest and obtaining improved learning results compared to systems featuring only text or voice outputs (Atkinson, 2002), (Gratch et al. 2007). ECAs can be designed to perform a great variety of gestures and facial expressions with different levels of intensity. Expressions can be exaggerated, and the evidence suggests that this can be used to improve learning by imitation. For example, Massaro (Massaro et al., 2000) gave the ECA Baldi lip, tongue and tooth movements to help deaf children learn to speak.
These are but a few examples to illustrate the fact that the analysis of the potential of ECAs in educational contexts, to support specific aspects of learning, is emerging as an important field of research embracing the scientific and educational communities. The relative novelty of this line of research requires that the very place and the possibilities of ECAs need to be explored. Moreover, incorporating this technology in interactive systems entails general interaction design challenges, and specific ones concerning evaluation in real user contexts, to which special considerations must be added for the particular context of learning reinforcement for children with special educational needs (Sánchez and García, 2003). Typical usability tests cannot be applied directly since metrics for effectiveness, efficiency and satisfaction cannot easily be obtained from the children. Furthermore, there are large differences in ability between children.
We propose to approach the specific complications in designing interactive special education support applications incorporating ECAs, through the adoption of design methods that allow, and take into account, collaboration between the child and the educator, as well as the research team itself. With this collaborative approach we have sought to explore the capabilities of ECAs in the classroom, in especially designed educational applications that are easy to use and that may be adapted dynamically to the needs of the children and the educators. In this chapter we illustrate the process through the description of our design experiences with two software applications for Special Education support featuring ECA technology. We took inspiration from the co-design methodology proposed by Sanders (Sanders and Stappers, 2008), which provided researchers and engineers, as well as children and teachers – for whom the applications were designed –, the opportunity to bring their creativity together.