Welcome to the InfoSci Platform
Reference Hub1
An Efficient Unification-Based Multimodal Language Processor for Multimodal Input Fusion

An Efficient Unification-Based Multimodal Language Processor for Multimodal Input Fusion

Fang Chen, Yong Sun
Copyright: © 2009 |Pages: 29
ISBN13: 9781605663869|ISBN10: 1605663867|ISBN13 Softcover: 9781616924805|EISBN13: 9781605663876
DOI: 10.4018/978-1-60566-386-9.ch004
Cite Chapter Cite Chapter

MLA

Chen, Fang, and Yong Sun. "An Efficient Unification-Based Multimodal Language Processor for Multimodal Input Fusion." Multimodal Human Computer Interaction and Pervasive Services, edited by Patrizia Grifoni, IGI Global, 2009, pp. 58-86. https://doi.org/10.4018/978-1-60566-386-9.ch004

APA

Chen, F. & Sun, Y. (2009). An Efficient Unification-Based Multimodal Language Processor for Multimodal Input Fusion. In P. Grifoni (Ed.), Multimodal Human Computer Interaction and Pervasive Services (pp. 58-86). IGI Global. https://doi.org/10.4018/978-1-60566-386-9.ch004

Chicago

Chen, Fang, and Yong Sun. "An Efficient Unification-Based Multimodal Language Processor for Multimodal Input Fusion." In Multimodal Human Computer Interaction and Pervasive Services, edited by Patrizia Grifoni, 58-86. Hershey, PA: IGI Global, 2009. https://doi.org/10.4018/978-1-60566-386-9.ch004

Export Reference

Mendeley
Favorite

Abstract

Multimodal user interaction technology aims at building natural and intuitive interfaces allowing a user to interact with computers in a way similar to human-to-human communication, for example, through speech and gestures. As a critical component in a multimodal user interface, multimodal input fusion explores ways to effectively derive the combined semantic interpretation of user inputs through multiple modalities. Based on state–of-the-art review on multimodal input fusion approaches, this chapter presents a novel approach to multimodal input fusion based on speech and gesture; or speech and eye tracking. It can also be applied for other input modalities and extended to more than two modalities. It is the first time that a powerful combinational categorical grammar is adopted in multimodal input fusion. The effectiveness of the approach has been validated through user experiments, which indicated a low polynomial computational complexity while parsing versatile multimodal input patterns. It is very useful for mobile context. Future trends in multimodal input fusion will be discussed at the end of this chapter.

Request Access

You do not own this content. Please login to recommend this title to your institution's librarian or purchase it from the IGI Global bookstore.