The Expansion of Paths in the Mutual Transformation Mechanism of Music and Narrative

The Expansion of Paths in the Mutual Transformation Mechanism of Music and Narrative

Taisuke Akimoto (Graduate School of Software and Information Science, Iwate Prefectural University, Takizawa, Japan), Jun Endo (Graduate School of Software and Information Science, Iwate Prefectural University, Takizawa, Japan) and Takashi Ogata (Faculty of Software and Information Science, Iwate Prefectural University, Takizawa, Japan)
DOI: 10.4018/ijcini.2013100103


For a mechanism of the surface expression phase in the authors' narrative generation system architecture, this paper proposes an approach to the circulative generation by the mutual transformation between music and narrative. In the concept for musical mechanism, music is not just the accompaniment for narrative and the authors extremely aim at using musical knowledge & techniques and conceptual knowledge & techniques in fusion by the medium of music. Depending on this concept, in this paper, the authosr introduce the newest prototyping system composed of eight routes of transformation modules between narrative and music including automatic composition and musical variation mechanisms. In addition, through the simple preliminary evaluations, the authors consider several insufficient points in the quality of music and the correspondence of music to narrative.
Article Preview


Many of AI based narrative generation systems automatically generate narrative texts using specific theories and techniques such as planning, story grammar, and script. In contrast, a feature of our narrative generation system architecture (Ogata, Hori, & Ohsuga, 1994; Akimoto & Ogata, 2012b), which is corresponding to the background of the system proposed in this paper, is that it is based on an integrated approach in which a variety of modules and techniques for narrative generation are unified into an organic system. This architecture is not a mere plan as many modules or mechanisms have been experimentally implemented according to the framework (Imabuchi & Ogata, 2012; Akimoto & Ogata, 2012a; Onodera, Akimoto, & Ogata, 2012) and a preliminary integrated prototyping system has been developed using the above programs (Akimoto & Ogata, 2012b). The musical mechanism in this paper is partially used in the integrated program. Strictly speaking, for example, BRUTUS system (Bringsjord & Ferrucci, 2000), which executes incremental narrative generation by using some types of methods such as theme structure, problem-solving, story grammar, and rhetorical techniques, is associated with integrated narrative generation approach. However, it deals with only one narrative theme to generate relatively high-quality natural language texts. Conversely, our approach aims at such narrative generation as having no limitation of themes and genres to be generated from the beginning. At the same time, it has other unique characteristics including the use of narratological methods (interdisciplinary approach) and the blending of expressive media with music and language-based narrative (inter-media approach). This paper focuses on the orientation of music mechanism in the narrative generation system. First, as a background of the discussion in this paper, we explain the macro structure of the narrative generation system architecture in which a music mechanism is contained. The system consists of following three big phases depending on the classification of three different types of processing for narrative generation:

  • 1.

    Story Phase: “Story” means the aspect of the content of a narrative, namely a temporal sequence of events. In the system, a story forms a tree structure including events and relations as the main elements and described by the conceptual representation. Each of the events is positioned at a node in the lowest level in a story tree, and each of the relations is equivalent to an intermediate node for binding the lower nodes by the relation. The semantics for an event and a relation are respectively described with the definition of a case frame and a semantic combining relation between some events or the lower nodes.

  • 2.

    Discourse Phase: “Discourse” means how to form a narrative for a story structure, namely a sequence of events to be narrated actually. A discourse is also a tree structure described by the conceptual representation to which the structure of story is transformed. Story processing is basically generation as the main processing is to add new parts in a story tree. By contrast, discourse processing is basically transformation as the central processing is to change the organization in a tree. However, a kind of transformation processing exists in the story phase and a kind of generation processing is used in the discourse phase. Therefore, the use of these terms in the following description is not necessarily strict.

  • 3.

    Expression Phase: “Expression” means narrative surface level including such surface representation media as natural language, visual or image media like animated movie, and music mainly. Music which is the theme in this paper is also one of the surface narrative media.

Complete Article List

Search this Journal:
Open Access Articles: Forthcoming
Volume 13: 4 Issues (2019): Forthcoming, Available for Pre-Order
Volume 12: 4 Issues (2018)
Volume 11: 4 Issues (2017)
Volume 10: 4 Issues (2016)
Volume 9: 4 Issues (2015)
Volume 8: 4 Issues (2014)
Volume 7: 4 Issues (2013)
Volume 6: 4 Issues (2012)
Volume 5: 4 Issues (2011)
Volume 4: 4 Issues (2010)
Volume 3: 4 Issues (2009)
Volume 2: 4 Issues (2008)
Volume 1: 4 Issues (2007)
View Complete Journal Contents Listing