A Methodology to Design and Evaluate Agents: The Study Language Agent Case

Introduction

Education is key in any society to improve both the individual and social welfare (Spanish Education Law, 2006). In particular, in Primary Education children start to develop their personality, and to acquire the basic oral communication, reading, writing and calculus skills. Teachers at this stage are encouraged to foster the individual and team work of the children with effort and responsibility according to each child learning rhythm.

Essential to any other learning, children are required to improve their learning of reading. This is because children who are unable to read are prone to fail in the learning of any other knowledge (Montero-Vivo, 2001). Some of the factors that seem to be related in the ability of correctly learning are the following (Fellbaum, 2005; González-Trujillo, 2006): speed and efficiency decoding words, vocabulary development, understanding the text structure, making inferences, self-assessment, morphology, syntax and prosody.

Moreover, in the last decades, it has been studied how children enjoy interacting with the computer and, 71% of the children have stated that they would like to talk to their computers (Narayanan, 2002). It is our hypothesis that this is possible by devising Pedagogic Conversational Agents (PCAs) defined by Johnson et al. (2000) as “lifelike autonomous character that cohabite the learning environment creating a rich face-to-face interface with the student”. Several benefits of using PCAs have already been reported in the literature such as the Persona effect, the Proteus effect and, the Protégé effect.

Lester et al. (1997) discovered the Persona effect, according to which, just the presence of an interactive agent in an educational computer environment has a positive influence in the students’ perception of the learning experience. Similarly, Yee and Bailenson (2007) discovered the Proteus effect, according to which students are motivated to achieve the features of the agents to become more like them; and, Chase et al. (2009) discovered the Protégé effect, according to which child students can make greater efforts to teach their agents than to study on their own.

In a previous paper, we presented Lingu, a Pedagogic Conversational Agent that implements a procedure to generate Spanish or English sentences to be morphologically and syntactically analyzed by children in an interactive way (Pérez-Marín & Gallo, 2015). In this way, children can work on their own, at their rhythm and with an infinite number of sentences.

In this chapter, the focus is on the methodology design, as Lingu was developed without methodology and it is too focused on the indicated task, but sometimes teachers need to modify the agent, or extend it to more study tasks. Without a methodology, it is not possible. This is the reason why a literature review was performed to find out whether such kind of methodology was available, and when none was found, a new Design and Evaluating Methodology for Agents called MEDIE has been proposed (Tamayo-Moreno, 2017).

Moreover, it is proposed, to use in first place, MEDIE to design and adapt the agent to each school and teachers/students needs, and later a Blended Learning methodology (Graham, 2005) to combine the use of the agent with traditional face-to-face lessons, as proposed by Dethare and Perez-Marin (2010), to keep the schedule of the school, and provide the students with on-line educational systems so that the students can work on their own. That way, the exercises can be adapted to the level of difficulty that can be tackled by the students. Moreover, if procedures to automatically generate the exercises are implemented, teachers will not be overloaded by having to design exercises for F2F and non F2F teaching hours.