Abstract
This chapter presents the methodology for transformation of behavioural cues into Social Signals (SSs) in human-computer interaction that consists of three steps: acquisition of behavioural cues, manual and algorithmic pre-selection of behaviour cues, and classifier selection. The methodology was used on the SS class {hesitation, no hesitation} in the interaction between a user and video-on-demand system. The first step included observation of the user during interaction and obtaining information about behavioural cues. This step was tested on several users. The second step was the manual and algorithmic pre-selection of all cues that occurred into a subset of most significant cues. Different combinations of selected cues were then used in verification process with the aim of finding the combination with the best recognition rate. The last step involved the selection of an appropriate classifier. For example, a logistic regression model was obtained in combination with four features.
TopIntroduction
The user’s interaction with communication devices in the user’s natural environment is still not a totally user-friendly experience and requires much of the user’s attention. Most approaches that can improve human-computer interaction (HCI) in handling a communication device are socially ignorant and there has been no attempt to use the user’s social signals (SSs) expressed during the user’s interaction with communication devices.
Social signal processing is a new research domain that aims to provide computers (systems) with the ability to sense and understand human SSs (Pantic, Nijholt, Pentland, & Huang, 2008; Vinciarelli, Slamin, & Pantic, 2009). SS can be expressed through a multiplicity of nonverbal behavioural cues initiated by the human body as a reaction to the current situation in everyday life. A behavioural cue is an atom action, which is atomic movement that can be described at the limb level (Poppe, 2010), or a micro movement such as a facial expression or head movement (Jokinen & Allwood, 2010). In human-to-human interaction, one person can easily ’connect’ all expressed behavioural cues of his/her interlocutor and interprets them as an SS, because the communication between two people is a natural process and human social intelligence allows people to do that. Even if the SS is expressed in many different ways, people will know how to interpret it correctly. In HCI (Figure 1), however, the computer does not know how to comprehend all the different combinations of behavioural cues that express the same SS because it is socially ignorant.
Figure 1.
The SSs are most distinctly expressed in human-to-human interaction. If we replace one person with communication device (system), we get HCI. In that case, communication device recognises the user’s SSs and uses them in interaction.
TopProblem Statement And Proposed Solution
The problem addressed in this chapter is how to determine a small number of significant behavioural cues that describe an SS in particular HCI and use them in a decision model with a high recognition rate. Since we observe the user via a camera, only behavioural cues demonstrated through visual features (such as the speed of a hand movement) are applicable. The number of cues used in the model must match the ability of human perception to recognize these cues and the reporting of decisions through a user interface in real time.
Key Terms in this Chapter
Social Signal of Hesitation: Social signal of hesitation belongs to a type of micro-movement (microslip) and can be expressed through facial expressions, head movements, etc.
Recommender systems: Recommender systems are software tools and technologies that predict user preferences and suggest useful items to a user.
Social Signal Processing (SSP): SSP is a research domain that aims to understand social interactions through machine analysis of nonverbal behaviour.
Logistic Regression: Logistic regression is a type of regression analysis used for predicting the outcome of categorical dependent variable based on one or more predictor variables.
Human-Computer Interaction: HCI involves the study, planning and design of interaction between users and computers. It is divided in simple and intelligent HCI.
Experimental Design: Design of experiment deals with planning, conducting, analysing and interpreting controlled tests to evaluate the factors that control the value of a parameter or group of parameters.
Video-on-Demand (VOD): VoD is a service that enables users to select one video content from among others.
Social Signals: Social signals are initiated by the human body and present reactions to current social situations. They are expressed with nonverbal behavioural cues.
Behavioural Cue: A behavioural cue is an atom action, which is atomic movement that can be described at the limb level, or a micro movement such as a facial expression or head movement.