Transmediality and the End of Disembodied Semiotics

Transmediality and the End of Disembodied Semiotics

John A. Bateman (Bremen University, Bremen, Germany)
Copyright: © 2019 |Pages: 23
DOI: 10.4018/IJSVR.2019070101

Abstract

The phenomena of mixing, blending, and referencing media is a major topic in contemporary media studies. Finding a sufficient semiotic foundation to characterize such phenomena remains challenging. The current article argues that combining a notion of ‘semiotic mode' developed within the field of multimodality with a Peircean foundation contributes to a solution in which communicative practices always receive both an abstract ‘discourse'-oriented level of description and, at the same time, a biophysically embodied level of description as well. The former level supports complex communication, the latter anchors communication into the embodied experience. More broadly, it is suggested that no semiotic system relevant for human activities can be adequately characterized without paying equal attention to these dual facets of semiosis.
Article Preview
Top

Introduction: The Challenge Of Transmediality

The phenomena of combining, mixing, blending, referencing and re-using media within the context of the use of other media has become a major line of research in both contemporary media studies (cf. Bolter & Grusin, 2000; Jenkins, 2008; Grant & Wilkinson, 2009) and approaches to adaptation and ‘reworkings’ of many kinds (cf. Elliot, 2003; Hutcheon, 2006; Elleström, 2010a; Bruhn et al., 2013; Hassler-Forest & Nicklas, 2015; Bruhn, 2016). Although media have arguably always been ‘mixed’ in many ways, the sheer flexibility, extent, and speed of change for combining media evident today is unmatched. Attempts to engage theoretically with the phenomena arising in such situations are consequently of growing importance and the literature on the topic is already extensive. When seen from a semiotic perspective, however, the situation is very different and there are few semiotically sophisticated accounts directly engaging with the phenomena, their semiotic foundations, and semiotic consequences.

The development of semiotic descriptions of medially complex artifacts and performances began in earnest when ‘popular’ communication, itself medially complex, was accepted in the 1960s as a justified object of study. Starting from the seminal work of Roland Barthes on a variety of media (cf. Barthes, 1977), considerable work followed focusing on visual semiotic systems, as in, for example, Groupe μ (e.g., Groupe μ 1992), Saint-Martin (1987) and, on broadly Greimasian lines, in Floch (1981). With few exceptions (e.g., Kawama, 1990), such work continued to build on the broadly Saussuro-Hjelmslevian lines adopted by Barthes and, for several reasons that will be discussed below, are as a consequence limited in their abilities both to characterize transmediality and to explain its effectiveness. In addition, in a general move towards cultural studies within semiotic discussions, analyses of all persuasions tended to consider restricted use cases, illustrating semiotic analyses on the basis of very small collections of examples and sometimes even single cases (e.g., Nöth, 2011).

Within discussions of this kind, several crucial notions underlying the ‘medialities’ that were being combined remained unaddressed. The primary reason for this was that the concept of ‘medium’ itself was not available semiotically for discussion. To this day, semiotic accounts talking of signs and their combinations have not, in general, been refined sufficiently to distinguish media and their properties as particular kinds of meaning-making. Although there is now work that attempts to move beyond this state of affairs – particularly, for example, in the theoretical position set out in detail by Elleström (2014a, 2014b) or in approaches originating in narrative concerns (e.g., Ryan, 2005; Wolf, 2007; Rajewksy, 2010) – the accounts of media involved are still, it will be argued, limited with respect to their semiotic foundations. The task taken up here will consequently be to set out a more foundational view of media that semiotically re-constructs the notion of a medium drawing on more recent developments in the area of multimodality. This will be suggested to enhance our abilities to work with embodied materiality and its role in media combination.

Complete Article List

Search this Journal:
Reset
Open Access Articles: Forthcoming
Volume 4: 2 Issues (2020): Forthcoming, Available for Pre-Order
Volume 3: 2 Issues (2019)
Volume 2: 2 Issues (2018)
Volume 1: 2 Issues (2017)
View Complete Journal Contents Listing