Working Together with Computers: Towards a General Framework for Collaborative Human Computer Interaction

Introduction

Human Computer Interaction (HCI) tends to include every development taking place in the field of computing, together with those in the areas of Cognitive Science, Multimedia Signal Processing, Ergonomics, etc., in its own domain. Right from the appearance of the first display device or time-sharing system, to any form of ubiquitous computing through an ordinary daily use device, or social network based systems or automated sensor-based monitoring of environments can be part of HCI. However, our purpose here is to discuss the architecture of interactive systems where human agents and the computational agents interact frequently to design or solve some tasks and use their experiences to improve further interactions. The objective here is to create a general framework which can include the basic underlying processes of collaborative HCI. Although a technological development may add a new dimension to the whole framework, e.g., advent of web technologies for connecting millions of people in a virtual community, but we restrict ourselves to basic core processes. However, let us put a disclaimer at the beginning itself that there are many such attempts with different viewpoints (Gluck & Pew, 2005; Taatgen & Anderson, 2009; Chong et al., 2007; Sun, 2006). This effort is also one such attempt where collaboration among many humans and computers forms the basis of the framework and hence distances itself from the traditional or other specific frameworks, e.g., UAN or RASCALLI.

HCI includes processes (computational as well as human processes) which interact in a meaningful manner to solve a complex problem or problems on an appropriate time scale. However, it should be noted that the goal of the interaction is not just to solve one or the other problem, but to enhance the interaction itself, say, by gaining experience for solving problems in a better way in future. Hence, the definition of HCI as given by Dix and Finlay (1998), as “a field of study related to design, evaluation and implementation of interactive computer systems used by humans, which also includes research of the main phenomena that surround it” may not be sufficient. In this definition, human beings are considered as users of the interactive system; they are not part of the system, although, the human processes, e.g., collecting expert knowledge or human-like visual system, etc. may be simulated on computers. Also, the role of environment (context) needs to be analysed. The interaction takes place in an environment (real-life or synthetic) which provides the context for interaction. In dynamic environments the result of interaction can change the environment, which in turn, can affect the interaction, and so on. The problem solving process has a specific goal at that moment, which may not be the long term goal of interaction process. Thus, solving one problem does not necessarily kill the interaction process; rather it can provide important information to improve further interaction. However, in real practice, the interaction-process is designed to fulfill a goal, although learning (long term) has now become an important component of HCI systems.

Existing framework of HCI (interactive) systems talks about a two layer system, one representing the computer and the other a user (human). The thin line that separates the two is the interface level, which provides various mechanisms for input/output exchange between the two. Various types of GUIs, movement of cursors through mouse or keyboards and various other input / output devices have been analyzed to enhance the efficiency or effectiveness of such systems. Recently, some human like processes, such as visual or speech based perceptual processes based on camera or microphone inputs are also being considered, which of course will lead to automatic natural interaction for systems based on gesture recognition, eye or head tracking or speech recognition, etc. However, the interaction is not a surface level small event which stops once the information exchange takes place. Interaction may take place in a deep manner and on a continuous basis, cycle by cycle, where it will acquire knowledge, learn and modify its mechanisms and select one or more appropriate choice. Hence the interaction processes, including the knowledge acquisition and deployment, implicit and explicit learning, representing and handling various types of memories, propagating information within and outside the system, evaluation and modification of processes, etc., require to be running and getting updated regularly and must be represented as a separate layer between human and computer layers. This three layer architecture is the key to the present work.