Article Preview
TopThe phenomena of combining, mixing, blending, referencing and re-using media within the context of the use of other media has become a major line of research in both contemporary media studies (cf. Bolter & Grusin, 2000; Jenkins, 2008; Grant & Wilkinson, 2009) and approaches to adaptation and ‘reworkings’ of many kinds (cf. Elliot, 2003; Hutcheon, 2006; Elleström, 2010a; Bruhn et al., 2013; Hassler-Forest & Nicklas, 2015; Bruhn, 2016). Although media have arguably always been ‘mixed’ in many ways, the sheer flexibility, extent, and speed of change for combining media evident today is unmatched. Attempts to engage theoretically with the phenomena arising in such situations are consequently of growing importance and the literature on the topic is already extensive. When seen from a semiotic perspective, however, the situation is very different and there are few semiotically sophisticated accounts directly engaging with the phenomena, their semiotic foundations, and semiotic consequences.
The development of semiotic descriptions of medially complex artifacts and performances began in earnest when ‘popular’ communication, itself medially complex, was accepted in the 1960s as a justified object of study. Starting from the seminal work of Roland Barthes on a variety of media (cf. Barthes, 1977), considerable work followed focusing on visual semiotic systems, as in, for example, Groupe μ (e.g., Groupe μ 1992), Saint-Martin (1987) and, on broadly Greimasian lines, in Floch (1981). With few exceptions (e.g., Kawama, 1990), such work continued to build on the broadly Saussuro-Hjelmslevian lines adopted by Barthes and, for several reasons that will be discussed below, are as a consequence limited in their abilities both to characterize transmediality and to explain its effectiveness. In addition, in a general move towards cultural studies within semiotic discussions, analyses of all persuasions tended to consider restricted use cases, illustrating semiotic analyses on the basis of very small collections of examples and sometimes even single cases (e.g., Nöth, 2011).
Within discussions of this kind, several crucial notions underlying the ‘medialities’ that were being combined remained unaddressed. The primary reason for this was that the concept of ‘medium’ itself was not available semiotically for discussion. To this day, semiotic accounts talking of signs and their combinations have not, in general, been refined sufficiently to distinguish media and their properties as particular kinds of meaning-making. Although there is now work that attempts to move beyond this state of affairs – particularly, for example, in the theoretical position set out in detail by Elleström (2014a, 2014b) or in approaches originating in narrative concerns (e.g., Ryan, 2005; Wolf, 2007; Rajewksy, 2010) – the accounts of media involved are still, it will be argued, limited with respect to their semiotic foundations. The task taken up here will consequently be to set out a more foundational view of media that semiotically re-constructs the notion of a medium drawing on more recent developments in the area of multimodality. This will be suggested to enhance our abilities to work with embodied materiality and its role in media combination.