New possibilities of representation through the use of multiple modes has challenged language's power as a dominant means to express meanings. Multimodal communication is now becoming the norm in the modern era of communication and Internet. Diverse forms of media exist and are combined in different ways to create new meanings though what constitutes media (media language or visual grammar), and the motivations behind design often remain transparent to users. The increase and diversity in different forms of representation other than written or spoken language also bring along changes in the field of literacy. In this chapter, the main focus is on Multimodal Social Semiotics —the theory of communication formed by Kress and his colleagues. The new language of multimodality and design builds on what it means to be media literate and has significant implications for media literacy education. This chapter's focus is on the basics of reading multimodal texts and the connection between new literacy and media literacy studies based on Multimodal Social Semiotics Theory.
TopIntroduction
There has never been a technology more influential than Internet that transformed reading more extensively since the invention of the printing press (Lang, 2012). In human history, Internet is the only technology that has been adapted for literacy by so many people, in very different contexts, with such profound consequences (Coiro, Knobel, Lankshear, & Leu, 2008). Today, more people have access to Internet than ever in the history. If he exponential growth rate continues, it is likely that Internet use will increase more in the future. The widespread adaptation of Internet also causes literacy practices to change and evolve continuously.
It is not just that a single technology of literacy has changed with the appearance of Internet but that the Internet, as a technology, permits immediate, global, and continuous change to literacy technologies themselves. (Coiro et al., 2008, pp. 4-5)
The widespread use of the Internet is transforming the global society into a digital society. This shift to a digital society, especially the ever-changing nature of technologies, has led literacy to become “deictic”. According to Leu et al. (2004), deixis is a special term that linguists use “to define words whose meanings change rapidly as their context changes” (p. 1150). Leu et al. (2013) applied this terminology to explain fluid and everchanging nature of literacy: Literacy became deictic as “we live in an age of rapidly changing information and communication technologies, each of which requires new literacies” (p. 1150). In other words, as the online technologies and social practices influencing literacy change, the definition of literacy changes as well (Baker, 2010; Lankshear & Knobel, 2006). As a result, it is not easy to define literacy simply as “acquiring the ability to take advantage of the literacy potential inherent in any single, static, technology of literacy” (Coiro et al., 2008, p. 5). Instead, literacy is fluid as people around the world constantly engage in literacy activities such as reading, writing, and communicating through embedded technologies powered by Internet (such as videos, instant messaging, social media platforms). Even the most recent technologies rapidly evolve and less popular as new versions of technological communication on Internet replace or modify the older ones. With profound changes in technologies, new literacies change in areas such as gaming, video technologies, Internet communities, search engines, webpages, and many more yet to emerge (Leu, Kinzer, Coiro, & Cammack, 2004).
Online texts daily change in structure, form, and content (Afflerbach & Cho, 2009; Coiro, 2011). As a result of the changing literacies, reading format and ways to engage with reading also change accordingly. “Electronic texts introduce new supports as well as new challenges that can have a great impact on an individual's ability to comprehend what he or she reads” (Coiro, 2003, p. 458). One of the main components of such supports is multimodality. Electronic reading is much more different from print reading due to the additional features of the Web (Sutherland-Smith, 2002). The digital coding of the textual elements is significantly different in online texts. Text does not only constitute of print letters, but they include multimodal features (Kress & Van Leeuwen, 2001; Kress, 2003). Different modes, the inherent meanings embedded within modes (i.e., the meaning of colors, arrangements, perspective etc. along with embedded meanings that the creators of these modes add to them) and the interaction of modes within a non-sequential reading platform (i.e., readers’ flexibility of reading multimodal texts in a non-linear order on Web) constitute complex meanings, which led to an increasing need for media literacy