A Context-Aware Smart TV System with Body-Gesture Control and Personalized Recommendation

Introduction

In recent years, smart TV becomes more and more popular in the market. Different from the traditional TV focusing on media broadcasting, smart TV systems are to deliver diverse multimedia contents from networked devices directly to the end-users, and allow them to access the contents through a friendly interface. It also provides interactive Internet-based services, such as media-on-demand, social networking, on-line gaming, etc. Smart TV is continuously developing to offer more and more functions and services. In this work, we address three important issues closely related to smart TV, including human-machine interfacing, personalized content recommendation, and context awareness, and develop the corresponding mechanisms to further enhance the performance of smart TV system.

Traditionally, people used to adopt device-based control to operate different consumer electronics at home. Afterwards, some researchers have also implemented systems that utilized personal hand-held devices to work as controllers, for example (Wang, Chung, & Yan, 2011; Lee, Wang, 2004). Lately, to provide natural control over the equipment, different ways of human-machine interactions (such as voice-based and gesture-based control) have been proposed to command the equipment without any remote control devices (Wachs, Kolsch, Stern, & Edan, 2011; Dumas, Lalanne, & Oviatt, 2009). Moving towards an even more natural way for interacting with machines, in this work we further design a body control mechanism through which the smart TV can response to how the human users move.

Though the smart TV system can easily present different types of multimedia contents to end-users, however, the large amount of contents leads to the problem of information overload. It is thus important to develop personalization techniques to recommend most suitable contents to users (Kim, Pyo, Park, & Kim, 2011; Angelides, 2003). Many recommendation methods have been proposed, ranging from content-based user modeling to group-based collaboration, and generally speaking the collaboration-based approach is considered more efficient and effective than the content-based user modeling approach. The current trend of organizing and sharing digital contents through user-created metadata (i.e., social tags) indicates that the performance of collaborative recommendation can be further improved by using such metadata to reason about how the users likes specific items. In this work, we will adopt such metadata for multimedia annotation, use the tag information to analyze how the user likes specific items, and exploit such user information to perform collaborative recommendation.

In addition to the recommendation techniques that focus on the multimedia items, context is another important issue to be considered in personalized recommendation. This is especially critical when the mobile smart TVs become popular, and the handheld devices can use a variety of sensory technology to collect and analyze information about their human user. In general, context awareness means the ability of computing systems to acquire and reason about the context information and adapt the corresponding applications accordingly. In other words, it is about capturing a broad range of contextual attributes (such as the user’s locations, activities, and their surrounding environments) to better understand what the user is trying to accomplish, and what content suits the user the most in that context (Baldauf, Dustdar, & Rosenberg, 2007; Baltrunas, Ludwig, Peer, & Ricci, 2012; Adomavicius, Sankaranarayanan, Sen, & Tuzhilin, 2005). By integrating context information into the application service, a recommendation system can fulfill the user’s needs more efficiently and practically.

Taking the previous issues into account, in this work we develop a smart TV system with several unique features. First, we design a Kinect-based system to recognize human body gestures for TV control. Second, we compare different computational methods for making personalized recommendation on multimedia items. Following the current trend of community-based information sharing, we also propose to exploit social tags to annotate multimedia items for the improvement of recommendation performance. Experimental results show that the tag-based method outperforms other conventional methods. Finally, we implement a context-aware platform and present how to integrate various environmental situations into the proposed method to perform item recommendation accordingly.