Integrating Interactive TV Services and the Web through Semantics

Introduction

During the last two decades, we witnessed many radical changes in the computing paradigms and environments. The recent advances in many areas of Information Technology (IT) seem to converge to a new global computing infrastructure that will be based on an enhanced World Wide Web (WWW). This enhanced WWW will be the corner-stone for human communication, interoperable computing systems, intelligent ultra-wide scale applications and, eventually, improved used experience. Among the aforementioned advances and trends we can distinguish: a) the technologies and social networking principles of Web 2.0, b) the huge online collections of multimedia content, c) the Service Oriented Architectures (SOA), d) the networked multimedia applications (e.g., IPTV, interactive TV) and, finally, e) the explicit representation of content and application semantics through structured metadata. This semantically enhanced application framework is, generally, referred to as Web 3.0. These new technologies can alter the current human-computer interaction practices and facilitate the consolidation of diverse IT solutions into really innovative products and services.

To better illustrate the type of services we refer to, let us consider a sample service that continuously displays information relevant to a currently playing documentary in a small window in a corner of the TV screen. We assume that each scene of the film is annotated with special metadata describing the entities presented (e.g., place, objects, time, persons, activities). These are correlated with the profile of the user as set prior to using the service so that the system decides what information to display. The actual information may be retrieved by different sources in the Web, such as Wikipedia, Flickr, IMDb, etc. Hence, we eventually have a kind of a “Web mashup” in the residential environment created through representation and reasoning of semantics. Another example service can be an advanced parental control service, where the rules for characterizing the content are based on its metadata. Such service would enable the characterization of specific scenes instead of the entire film. We will describe in more detail such service in a following section.

However, despite the maturity of most of the aforementioned technologies, their integration into platforms able to deliver services like those just described has not made much progress. One reason for this is that most researchers focused on specific technology integration tasks. For example, some researchers addressed the integration of TV and Web content while others focused on personalizing specific TV services, usually the EPG (Electronic Program Guide). On the other hand, most of the works ignore the semantics of the content or users. The only metadata they exploit are simple tags or keywords. One reason for this is that rich and formal semantic metadata are hardly available. There are no easy-to-use tools and the publicly available metadata collections do not usually refer to audiovisual content. Unfortunately, this lack of semantics data results in greater effort for developing sophisticated algorithms. Our thesis is that by exploiting the semantics metadata of TV content, we can rely on declarative ways of personalizing services that simplify a lot the development of such systems.

In this article, we describe the application of the Web 3.0 paradigm to a residential infotainment environment that is developed over interactive digital TV (iTV). The discussed approach, implemented in the context of the POLYSEMA project (POLYSEMA, 2009), employs Web technologies and services, multimedia semantics and an extensible residential infrastructure that can support a flexible service lifecycle. Specifically, the main contributions of our solution are:

• 1.

  Semantic annotation of audiovisual content through Semantic Web ontologies (e.g., MPEG-7). This results in formal metadata descriptions and advanced support for multimedia content indexing and retrieval. Based on an existing ontology we adapted it and we built a tool for annotating video files (Valkanas et al., 2007).

• 2.

  Declarative representation of user profiles and preferences with (Semantic Web-compliant) rules.

• 3.

  A testbed for standardized industry technologies (e.g., DVB-T, DVB-MHP, OSGi) and research techniques and tools (context-aware Web content retrieval, rule-based personalization).

• 4.

  An open platform that exploits the Web 3.0 framework in the home infotainment domain. The design principles of this platform facilitate the introduction of residential SOA architectures and the introduction of more flexible business models.