Gesture Spotting Using Fuzzy Garbage Model and User Adaptation

Introduction

The shortage of caregivers for the elderly and/or people with disabilities has become a critical social issue in many countries (Kim, 2003; Stefanov, 2004). To cope with the trend, many studies have been conducted on user-friendly human-computer interaction (HCI) techniques by which the elderly and people with disabilities can operate various electrical systems more easily. A soft remote control system developed at HWRS-ERC in KAIST is an application example of enabling the users to operate home appliances by hand gestures (Do, 2002, 2005). The system utilizes ten different predefined hand gestures as a given set of commands as shown in Figure 1.

Figure 1.

Ten different hand gestures

In spite of these attempts to create a competitive and useful application of hand gestures, the recognition of command gestures is faced with various practical problems: the 3 imperative issues are listed.

• •

  A set of features and decision rules to distinguish command gestures should be carefully obtained through observation of the user’s behaviors. To cope with this requirement, Han (2006) proposed a feature selection method based on a separability index matrix (SIM) and Jeon (2008) constructed a multivariate fuzzy decision tree by using linear discriminant analysis (LDA) and information gain.

• •

  The system needs to be adapted to a specific user to enhance the performance since each person behaves in different way (inter-person variation). In addition, the characteristic of a single user may also change in a different environment. Therefore, an additional adaptation method is required for intra-person variation which occurs for a single user. This adaptation should be executed in real time to modify the temporal difference from a single user.

• •

  In actual operation of the system, it is often observed that some of the user’s ordinary but meaningless behaviors are mistakenly recognized as command gestures. In fact, those command gestures are very likely to be basic natural motions and thus similar motions can appear in the user’s ordinary behaviors. More complicated command gestures may be adopted as an alternative to maximize the different aspects between the command gestures and ordinary motions. Such an approach, however, is prone to decrease the usability of the system and can cause a cognitive burden for the user to memorize the larger set of command gestures.

Given these problems and issues, we introduce a gesture spotting method using the fuzzy garbage model to recognize the designated gesture from the user’s ordinary behaviors, while the existing approach focused on the construction of decision rules with user adaption to achieve high classification performance (Jeon, 2008).

Human gestures observed in ordinary behaviors are likely to be unstructured and occur unconsciously. Moreover, the characteristics of human behavior are different from person to person. It is thus rather difficult to express human gestures by a stochastic or statistical model. As noted in Passino and Yurkovich (1998), the sensitivity to variation of parameters or environment can be reduced by using the fuzzy logic which uses fuzzified values instead of crisp values. In addition, much of the acquired knowledge through observation is easily applied to the system. The usage of linguistic values, in fact, makes the fuzzy logic superior to other methods, since the ambiguous characteristic of unconscious human behaviors can be expressed and manipulated by linguistic variables.

To recognize a specific gesture, our proposed method utilizes two fuzzy models: one for a command gesture and the other for garbage gestures. The motion data of the user are applied to both models, and then each model gives a score which shows the degree of belongingness to each model. By comparing the scores from these models, the system decides whether the input data represent a command gesture or not.