Designing a Visual Language for Interaction Representation, Based on Aspects of Human Movement

Abstract

Interactions are core part of interactive computing. However, their mechanisms remain poorly understood. The tendency has been to understand interactions in terms of the results they produce rather than to provide the mechanisms that explain “how” interactions unfold in time. In this chapter, we present a framework for creating visual languages for representing interactions, which uses human movement as a source for the core concepts of the visual language. Our approach is motivated and supported by the evidence, coming from the research on kinaesthetic thinking, that constructs based on human movement support higher-level cognitive processes and can be intuitively recognised by humans. We presented an overview of the framework, an instance of a visual language design using the proposed framework and its application for representing and analysing interactions between patients and practitioners in the healthcare domain. Developed approach and the corresponding techniques target interactive computer systems for facilitating interaction-rich domains, such as health care, in particular occupational therapy, collaborative design, and learning.