Relations Among MOCAP and Textual Data of Motion Verbs: A Distance Calculation Perspective

Relations Among MOCAP and Textual Data of Motion Verbs: A Distance Calculation Perspective

Marietta Sionti (Athena Research and Innovation Center in Information Communication and Knowledge Technologies, Greece & Neurocognition and Action Group, Cognitive Interaction Technology Institute, Bielefeld University, Germany), Panagiotis Kouris (Athena Research and Innovation Center in Information Communication and Knowledge Technologies, Greece), Chrysovalantis Korfitis (Athena Research and Innovation Center in Information Communication and Knowledge Technologies, Greece), Vasiliki Moutzouri (Athena Research and Innovation Center in Information Communication and Knowledge Technologies, Greece) and Stella Markantonatou (Athena Research and Innovation Center in Information Communication and Knowledge Technologies, Greece)
Copyright: © 2020 |Pages: 18
DOI: 10.4018/IJACDT.2020070104

Abstract

This is a study on language and action that tries to shed light on their conceptual correspondence in terms of embodiment. The linguistic phenomenon of lexical aspect/Aktionsart is studied in connection to joint angles and time. For the purposes of this research, data concerning the usage of a set of Modern Greek verbs were collected and annotated; in addition, motion data were captured for the same verbs. First, the distance between actions and verbs was calculated revealing a strong connection between certain types of language data, lower limb motion, and time: an extended use of the lower limbs is related to longer or repetitive actions while time expresses duration.
Article Preview
Top

Introduction

According to the UNESCO Convention on the Safeguarding of Intangible Cultural Heritage and the Greek Law 3028/2002 “On the Protection of Antiquities and Cultural Heritage in general” (Official Government Gazette 15/Α’/ 28.6.2002) as intangible cultural goods are considered different facets of everyday language, such as verbal expressions, oral traditions, myths, songs, etc. (ayla culture, 2020). Our goal is to analyze language as a cultural and cognitive phenomenon related to action, under the prism of embodiment. For this reason, we attempt to ground the abstract linguistic notion of Aktionsart/lexical aspect to concrete low level kinematic features, captured with a Synertial biomechanics suit. Since the majority of relevant literature is deploying visual data, the proposed linkage is the first step of an innovative analysis with potentially far-reaching effects, for instance, the results could be embedded in robots in order to enhance embodied human-robot interaction, human activity recognition, event segmentation and labeling of sensorimotor data.

To this end, first we captured kinematic data about the actions denoted by a set of motion verbs of Modern Greek, namely actions of pushing, pulling, hitting, and beating. Next, we collected 150 sentences for each of the studied verbs from the web and annotated them drawing on the Abstract Meaning Representation framework (AMR); (Donatelli, Schneider, Croft, & Regan, 2019). Eventually, we propose an extension of the AMR schema suitable for the Modern Greek aspectual system. Next, for the same actions we obtained mocap and linguistic annotation data and calculated the distance between them. Finally, we combined kinematic and textual results, in order to ground aspectual features (such as actor, telicity, adverb for aspect declaration, etc.) to specific movements for the needs of human robot interaction. In the preliminary results, the feature “time and lower limbs” that occurs in the mocap data returns the best intersection of sets of close verb pairs obtained from the kinematics and linguistic data, suggesting that the particular lexical aspect features obtained from corpus data may be related with the internal temporal constitution of these actions.

Aspect is a core linguistic property of structures with verbal features (verbs and deverbal nominals); it is about temporal and frequency information that results from the semantics of the verbal element and the overall structure that is headed by it. Comrie (1981, p. 3) supports that aspect provides different ways of viewing the internal temporal constituency of the situation. Here we are not interested in how the aspect of a sentence is calculated (for instance, see Dowty’s Montagovian approach (1979)) but in the verbal lexical aspect or Aktionsart. Verbal lexical aspect has to do with the temporal structure of a verb’s meaning and is independent of the various structures that the verb may head. Venerated approaches to the issue are those of Zeno Vendler (1957) and Comrie (1981) while extensive work on Modern Greek can be found in Moser (2013).

This work is about verbs denoting movement; motion events, by their nature, have an inherent temporal dimension and a certain duration, which is considered to play significant role in the mental simulation of an action both in the execution of the movement per se and the linguistic expression of the real world actions (Bergen & Chang, 2005; Matlock et al. 2005; Zwaan, 1999; Barsalou, 2009), that have been previously observed and learnt by mirror neurons (Fadiga et al., 2006; Arbib, 2008).

Complete Article List

Search this Journal:
Reset
Open Access Articles: Forthcoming
Volume 10: 2 Issues (2021): Forthcoming, Available for Pre-Order
Volume 9: 2 Issues (2020)
Volume 8: 2 Issues (2019)
Volume 7: 2 Issues (2018)
Volume 6: 2 Issues (2017)
Volume 5: 2 Issues (2016)
Volume 4: 2 Issues (2014)
Volume 3: 2 Issues (2013)
Volume 2: 2 Issues (2012)
Volume 1: 2 Issues (2011)
View Complete Journal Contents Listing