Multiformat and modality interfaces have become popular and effective tools for presenting information in training and instructional systems. Technological innovation, however, has far surpassed researchers’ understanding of how and under what circumstances these technologies are useful towards information gathering. Some recent research has begun to characterize the cognitive mechanisms that may be responsible for the comprehension and memory advantages typically seen with multimedia learning, as well as the role of individual differences in this process. Other work has defined effective pedagogical practices, such as instructional content and organization, for producing engaging and effective learning experiences. This chapter attempts to bridge these two research areas and provides concrete design recommendations for current instructional practice and directions for future research.
TopIntroduction
When I was a child, I judged how interesting books would be based upon the picture-to-page ratio. Only high ratio books would stand a chance of ending up in my backpack. Much older now, to some extent, I still adhere to this rule. In reading textbooks, I know that the ones that are more helpful to my students are those that embed pictures in the text. Perhaps not surprisingly and consistent with much research (e.g., Mayer, 2001; Paivio, 1986), smartly-combined pictures and text is inevitably a better learning medium than either format in isolation. However, it is clear that the story is not that simple. In 2006 alone, PsycINFO lists 184 multimedia publications in peer-reviewed journals, book chapters, and handbooks, demonstrating that while much is known, much is also being explored with regard to the cognitive and educational influences of multimedia across a wide range of applications. Increases in the availability of technologies with which educators, experimenters, and software companies can integrate text, images, narration, animation, and virtual reality have unfortunately not corresponded with increases in educational and cognitive psychologists’ understanding of how such technologies influence learning. The application of these technologies is as broad as the research that engenders them: from the young student studying photosynthesis to the Soldier studying maps and descriptive details about an area of operation. Across these applications, educators, designers, and engineers must ensure that information gatherers are successfully processing and comprehending, and ultimately using knowledge in appropriate ways.
To address this need four questions are posed that have broad and powerful implications for both systems design and educational effectiveness: 1) what effects, if any, do format and modality manipulations have on eventuating memory form and function, 2) which working memory mechanisms are involved in the processing, manipulation and integration of multi-format and multi-modality information, 3) how does the effectiveness of manipulations vary as a function of learning material types (e.g., facts, rules, procedures), and 4) what, if any, individual differences predict the success of various media combinations?
The present chapter attempts to answer these questions by reviewing research from cognitive, educational, and human factors psychology and pointing out gaps in knowledge that can motivate subsequent experimental investigations. Three broad domains are examined for which multimedia may prove an effective learning tool: spatial learning (e.g., maps, spatial descriptions, virtual reality), procedure learning (e.g., object assembly), and declarative learning (e.g., facts, rules, information). Theoretical motivations are provided regarding how media manipulations affect human mental representation, how these effects vary with the domain of application, and how the human mind uses working memory to manipulate and integrate spatial and verbal information towards abstract and flexible final memory forms. Finally, insights are provided into how educators may predict the success of multimedia at the level of the individual student.