Literacy researchers and educators assert that expanding the ELA curriculum to include visual and multimodal literacy is necessary to meet the demands of the twenty-first century. Researchers suggest that developing visual and multimodal literacies can be accomplished by teaching students to use a metalanguage of visual grammar. This gives students the tools they need to discuss and analyze visual and multimodal texts. This chapter considers how a metalanguage of visual grammar is developed in one high school English classroom during World War II literature unit that featured a number of multimodal texts. The authors explore 1) how wordless picturebooks can be used to scaffold students reading of multimodal texts, 2) how wordless picturebooks can be used to develop students' metalanguage of visual design, and 3) how students utilize their developing metalanguage when analyzing and discussing multimodal texts. Insights for classroom instruction are provided.
TopBackground
Multimodal texts utilize more than one mode, where modes are defined as socially and culturally shaped resources for representing and communicating meanings (Kress, 2010). One particular mode is not inherently better than another mode; the semiotic resources associated with various modes simply do different things in different ways (Kress, 2004). Each mode has particular affordances and limitations as to how it represents and communicates ideas and events (Duncum, 2004; Stockl, 2007). Multimodal texts require students to navigate, apply, and interpret different semiotic resources and create meanings from the configuration of these varied resources including image, gesture, gaze, intonation, sound, body posture, and design (Jewitt, 2006; Jewitt & Kress, 2003). It is important to understand how different modes and semiotic resources are utilized and how students can develop their facilities for applying various semiotic resources in both representing and communicating ideas (Serafini, 2014a).
The semiotic resources used for communication and representation cannot be understood in isolation from social and cultural forces (Iedema, 2003). A shift from the dominance of written language and the printed book to the increasing significance of the visual image and multimodal text is also a revolution in the social and cultural institutions in which these resources function (Serafini, 2010). Traditional theories and instructional practices of literacy education focusing on written language delivered through print-based texts have become inadequate as the sole systems of representation and communication in today’s culture (Serafini, 2014b).
The New London Group (1996) argued for an accessible functional grammar, or metalanguage, for describing and discussing how texts and images work. Unsworth (2006) asserted that teachers need to explicitly develop students’ knowledge of visual grammar in order for students to interpret multimodal texts. The development of a metalanguage focusing on the structures and grammars of visual and multimodal texts is a necessary step in supporting readers. Developing a metalanguage allows teachers to talk with students not only about what an image might mean, but how an image means. Various researchers suggest that teaching readers to use this metalanguage supports their interpretive and composing practices as they work to represent and communicate ideas and events with an array of semiotic resources and modalities (Serafini, 2010; Unsworth, 2006).