Augmented Reality in Language and STEM Education: Implications and Potentials for ELLs

Introduction

Augmented reality (AR) is an interactive simulation of a real-world environment where the objects that reside in the real world are enhanced and manipulated by computer-generated perceptual information (Azuma, 1997). The applications created by AR enable the usage of three-dimensional models (i.e., 3D objects, images, videos, and animations) both separately and simultaneously (Wang et al., 2013). The merging between computer and the reality, therefore, can involve multiple sensory modalities, including visual, auditory, haptic, somatosensory, and olfactory. AR technology can also be easily launched and effectively combined with global positioning system (GPS)-enabled smart devices or quick response (QR) codes to create a unique, compelling, and meaningful learning experience (Lara-Prieto, Bravo-Quirino, Rivera-Campa, & Gutiérrez-Arredondo, 2015). Students can access AR through their smart phones, laptops or tablets by scanning AR-embedded codes or images to reach the three-dimensional (3D) demonstrations in front of their eyes. Hence, AR technology, as a medium for immersive collaborative simulation, technology-mediated narrative, and situated problem solving affordance, can transform the scenario of a traditional English language classroom (Godwin-Jones, 2016).

As an example of AR, The Seed Shooting Game, created on the platform of Zooburst, is not a printed book where students flip the pages to read the story. In this AR 3D pop-up version, the students can observe the story happening right in front of their eyes (as illustrated in Figure 1). Its vivid images in a 3D look give students the feeling like they are standing right next to the story characters in a garden. In this AR book, students can also participate in different language games to fulfill different objectives. The objectives include: 1) understanding how to use quotation marks, 2) demonstrating the understanding of the story plot, 3) placing the story events in chronological order, 4) matching the vocabulary with pictures, and 5) using prepositions to describe the objects in the story.

Figure 1.

A screenshot of 3D pop-up book called The Seed Shooting Game

Source: Vate-U-Lan, 2011

This AR 3D pop-up book integrates the content with auditory and visual entertainment through storytelling techniques and colorful animations, so students are positively engaged with the two main characters in a racing game to throw the seeds and explore the garden along the journey. For each object, animal, and tree they encounter in the garden, students are invited to examine it and learn new words. They can also put the AR book on the table or take it to a real garden outside their class and enjoy the story. After interacting with a safe and entertaining augmented digital environment provided by the story, students go out to the playground to search for those objects’ representations to learn how to spell the words, pronounce them, and learn other information such as their habitats and/or characteristics. They also can make the decision to plant the seeds and complete the gardening tasks on their laptop devices. The students then work in pairs, dragging all the characters they have found to create a storyline they imagine and transforming the garden in the way they want. The post-test for students’ learning outcomes showed that students not only maintained their engagement throughout the learning session but also better retained their vocabulary and thus increased their reading comprehension (Vate-U-Lan, 2012). Isn’t it fascinating to have these ELLs interact with both real and synthetic environments as in this scenario?

There have been several studies which investigated the effectiveness of AR as multimedia in optimizing students’ performance and motivation in English language classrooms (Brinson, 2017; Cadavieco, Goulão & Costales, 2012; Chang, Morreale, & Medicherla, 2010; Ji & Shin, 2019; Liu & Tsai, 2013; Santos, Lübke, Taketomi, Yamamoto, Rodrigo, Sandor, & Kato, 2016; Singhal, Bagga, Goyal, & Saxena, 2012). Moreover, AR is also a pedagogical tool for facilitating complexly structured learning academic content, such as STEM subjects (Science, Technology, English, and Mathematics) in an English language learning environment (Chen, Kao & Sheu, 2003; Hockley, 2019; Rau, Zheng, Guo, & Li, 2018; Oliveira, Behnagh, Ni, Mohsinah, Burgess, & Guo, 2019; Yoon, Elinich, Wang, Steinmeier, & Tucker, 2012).