Advanced Multimodal Frameworks to Support Human-Computer Interaction on Social Computing Environments

Introduction

The novel hardware technologies used to improve the computational capability of the current mobile (e.g. smartphones, PDAs, palmtops) and desktop (e.g. tablets, workstations, laptops) devices, jointly to the more and more high performances expressed by the current LAN (e.g. ISDN, Cable, DSL, T-Carrier) and wireless (e.g. GSM, GPRS, EDGE, UMTS) network connections have allowed the wide spread of heterogeneous social computing environments that support human activities in everyday life. More specifically, all these novel devices and related connection potentialities have contributed to expand the boundaries of the current digital collaborative and social interaction on almost all main application fields: professional, educational and entertainment.

The most current approaches (e.g. keyboards, keypads, trackballs) used to interact with these innovative devices, and related complex digital environments, do not seem able to provide an effective and exhaustive support to accomplish the increasingly needs required to face the current challenges in the advanced human-computer interaction field. Actually, there are several reasons that lead to an evolution of the current human-computer interfaces used to interact with the mentioned devices and related social computing environments. The first one regards the pervasive concept by which the user can interact, anytime and anywhere, with any kind of device and related digital environments. This reason takes on even more significant considering the increasingly main role of the current heterogeneous mobile devices. The second one concerns the amount of functionalities related to every single social computing environment. In fact, independently from the kind of the specific digital environment, the current social computing applications and services tend to have an increasing amount of functionalities which often require more suitable ways to be effectively and efficiently used. The third reason regards the growing needs to have customizable human-computer interfaces. Indeed, the customization of the access methods to both current and next generation of social computing environments is the actual challenge of the human-computer interaction.

All the mentioned aspects lead to define new types of interfaces able to interact with any kind of both device and related social application or service. In particular, these new interfaces must have three main features: intuitiveness, human-oriented and Web 2.0 designed. The first feature highlights that the user should perceive the usability criteria of the interface in natural, effortless and simple way. This means that there should be a direct interaction process with the applications or services, regardless of the characteristics of a specific interface (and related device). The second feature focuses on the need to use human-oriented styles during the interaction process. In fact, the final frontier of the human-computer interface research is to allow users to interact with any device, application or service by using the same “tools” employed during human to human interaction, such as: voice (and related tone), expressiveness by handwriting and freehand drawings, gesture, facial expressions, and so on. The last feature regards a technical aspect of the interface design. In particular, this feature expresses the need of a development process compliant with the high standard of the Web 2.0 which is strongly based on two main concepts: advanced interaction and content sharing. For this reason, the Web 2.0 technologies can be considered the natural support for the social computing interfaces.

Multimodal interfaces provide a powerful tool to solve any kind of mentioned problem tied to the interaction processes: pervasiveness, amount of functionalities and customization. Besides, they satisfy the features related to the innovative interfaces: intuitiveness, human-oriented and Web 2.0 designed. In particular, multimodal interfaces, by using human-oriented modalities (e.g. speech, handwriting, freehand drawing, gaze) enable users to express concepts, provide commands and represent ideas in order to interact with any aspect of the social computing environments. Every modality can be exploited, separately or jointly with others, in two ways. In the first one the user provides concepts, commands or ideas in order to convey inputs to the digital environments, in the second one the user receives feedbacks in order to obtain outputs from them.