A Semiotic Model of Multimedia: Theory and Evaluation

Abstract

Models of multimedia communication are attempts to classify the numerous types of media objects available, and to provide a basis for the use of unambiguous terminology in a new and expanding field. Many of these models are products of theory, rather then practical investigation, and few have been empirically studied to assess their suitability. This chapter firstly presents a novel multimedia model (called TOMUS) which is based on a common classification of semiotic representational systems, and uses three dimensions of sign, syntax and modality. By separating the classification of the nature of the text to be communicated from the nature of the technology or interaction, the model provides a focussed terminology for consistent and appropriate discussion about multimedia texts. The chapter also reports on an experiment which investigated the understandability of the dimensions comprising TOMUS. The experiment entailed subjects classifying various media objects according to the TOMUS model. Error and perceived difficulty data were collected; analysis of this data revealed which of the categories are the most difficult to comprehend. Suggestions are made as to the causes of these difficulties, and recommendations as to how the model might be correspondingly altered are proposed.