This chapter presents the Stanza Logo-Motoria, an Interactive Multimodal Environment (IME) for learning, which the authors have been developing and experimenting with since 2009 in several educational institutions. The aim of this chapter is a) to describe the activities carried out by the Resonant Memory, the first application for the Stanza Logo-Motoria, b) to illustrate the validation protocol of the system used as a listening tool for learning English as a Second Language (ESL), and finally, c) to document the positive partial results that demonstrate the improvement in ESL oral comprehension in pupils using the Stanza Logo-Motoria. The authors have also found that this environment can offer pupils: a) a truly interactive multimodal learning experience, b) a social opportunity for learning among children, and c) an intrinsically motivating experience.
TopIntroduction
Although the verbal mode of instruction has long dominated education, our research has focused on Interactive Multimodal Environments (IMEs), especially those that combine auditory and pictorial representations of knowledge, because pupils’ understanding can be enhanced by the addition of non-verbal knowledge representations to verbal explanations (Moreno & Mayer, 2007). The following are just a few examples of IMEs which, in recent years, have been developed in order to implement new modalities of learning in physical spaces.
SOUND=SPACE (Almeida, et al., 2008) was one of the first examples of using an IME with children. It is an interactive multi-user musical environment in which users (in particular groups with special needs) influence the sound production by moving in an empty space surveyed by an ultrasonic echolocation system.
In the area of education and entertainment, Krueger explored the full-body interaction in a room-sized space (Krueger, Gionfriddo & Hinrichsen, 1985). His applications (specifically Videoplace) have been used in the educational field as shared workspaces, scientific exploration for children, and physical therapy.
The interactive physical spaces were further explored by Davenport and Friedlander (Davenport & Friedlander, 1995) who developed the Wheel of Life which was an interactive world situated in a real space; the space contained a narrative which might be actualized by the actions of the visitor moving through it. The room used light, sound, video, and computer displays.
Another important perceptually-based and interactive story environment was the KidsRoom (Bobick, et al., 1999) which re-created a special bedroom for children. Two of the bedroom walls resembled the real walls; the other two were large video projection screens where images were back-projected. Six computers controlled the room, coloured lights on the ceiling, and four loudspeakers. Four video cameras and one microphone were installed. The goal of this system was to stimulate a child’s imagination by using images and sound in order to transform the space into a fantasy world.
Finally, it is very interesting to mention the Hazard Room Game (Fails, et al., 2005) a game implemented both in desktop and physical environments, that focused on teaching children about environmental health hazards. The main goal of this project was to investigate, in direct comparison, the desktop and physical educational environments in the domain of young children’s learning. The final results (obtained by means of qualitative and quantitative measurements) proved the advantages for physical environments over desktop environments.
In this chapter we will describe the Stanza Logo-Motoria (Camurri, et al., 2010) an IME for learning, which is used - as an alternative/additional tool for teaching - in several educational institutions, where we are performing a lot of interactive activities for schoolchildren and conducting an experimentation (which we will fully explain in the paragraph entitled “Validation Protocol of the Stanza Logo-Motoria”). With the Stanza Logo-Motoria it is possible to actualize an IME in which, whatever happens depends on the learner’s actions (Moreno & Mayer, 2007). This system, which we will describe in the paragraph entitled “The Stanza Logo-Motoria”, takes a broad empty space and, using video tracking techniques, fills the environment with sound events and images; the user’s presence inside the “interactive space” triggers the playback of the audio/visual content.
Unlike the IMEs that we have briefly illustrated above, the Stanza Logo-Motoria uses standard hardware and input/output conventional peripheral devices such as a webcam, two loudspeakers and a video projector. The easy implementation of the system and the low cost of both the hardware and the software allow the use of the system on the part of public schools with real classes of schoolchildren. The modular software architecture enables users to adapt the environment to different educational contexts, for example, learning a second language, studying History, or improving their spatial ability. In the following paragraphs we will explain why, when, and how we use the Stanza Logo-Motoria in an educational environment such as the primary school (Figure 1).
Figure 1. Learning in the Stanza Logo-Motoria