Kinesthetic Communication for Learning in Immersive Worlds

Introduction

The question of whether video games can facilitate real learning dates to the very beginning of interactive computing technology in the late 1960s. Early experiments in Artificial Intelligence focused on the learning potential of the computer by developing game-playing agents for classic board games such as checkers. Researchers soon observed, however, that such interactions often led to demonstrable learning on the part of the human user, inspiring the hypothesis that interactive computer interfaces could foster rich learning environments for people.

Almost half a century later the promise still appears unfulfilled, despite significant technological advances including immersive three-dimensional worlds and sophisticated input devices, and despite the maturation of the medium to include a variety of game styles and to appeal to a wide spectrum of game players. An underlying theory of how learning can take place in such environments remains in its infancy, with most self-proclaimed learning games resembling either a multimedia, screen-based version of a printed textbook, or an awkward hybrid of game-play glued on top of rote instruction or vice-versa. Such games do little to encourage larger abstract problem-solving skills, nor do they accommodate likely variations in learning styles on the part of the players/students.

In this chapter we posit a game-design architecture that fully exploits the pedagogical potential of a rich graphical environment, coupled with an intuitive and sophisticated kinesthetic interface. While many educational games are structured around their particular content — the information to be imparted to the user — our approach is grounded in the game’s context, with the assumption that genuine learning takes place in context, and that the richer and more compelling that context, the more thoroughly concept and skill mastery will occur. In this way, our work complements that of Janet Murray (1998) and other theorists who point toward the need for richer, contextualized environments to improve the educational applications of interactivity. Building on the Constructionist approach of Seymour Papert, we endeavor to move beyond games that demand the single “correct” answer to a multiple choice question in favor of broader processes of discovery and problem-solving. By creating a Spanish-learning game in which students are able to exercise considerable control over their environment, and fulfill learning goals by means of the innovative and intuitive physical interface, the students’ journey through the story is intimately tied to their learning process.

The development of kinesthetic interfaces such as the Nintendo Wii controller opens up countless new possibilities for computerized foreign-language teaching. Using a motion-sensitive controller has allowed us, for instance, to assimilate well-tested pedagogical principles that link sensory-motor experience and second-language learning, such as James Asher’s Total Physical Response (TPR), first introduced in the 1960s. In Asher’s movement-based method, teachers give students verbal commands to act out without having to speak. During the first few hours of classroom time, students perform simple commands such as jumping, sitting, and standing, though instructions quickly become more complex (Asher, 2003). Studies by Asher suggest that in beginning foreign language classrooms, students taught using Total Physical Response outperform those schooled using more traditional methods (Asher, 1965, 1966, 1969, 1977; Asher, Kusudo, & De la Torre, 1974). Other researchers have supported the idea that bodily movement aids in vocabulary retention (Swaffar & Woodruff, 1978; Thiele & Schneiber-Herzig, 1983; Wolfe & Jones, 1982). Most recently, Seth Lindstromberg and Frank Boers (2005) suggest that TPR may aid in more complex forms of language processing.