Web Content Recommendation Methods Based on Reinforcement Learning

Web Content Recommendation Methods Based on Reinforcement Learning

Nima Taghipour (Amirkabir University of Technology, Iran) and Ahmad Kardan (Amirkabir University of Technology, Iran)
DOI: 10.4018/978-1-60566-306-7.ch011
OnDemand PDF Download:


Information overload is no longer news; the explosive growth of the Internet has made this issue increasingly serious for Web users. Recommender systems aim at directing users through this information space, toward the resources that best meet their needs and interests. In this chapter the authors introduce their novel machine learning perspective toward the Web recommendation problem, based on reinforcement learning. Our recommendation method makes use of the Web usage and content data to learn a predictive model of users’ behavior on the Web and exploits the learned model to make Web page recommendations. Unlike other recommender systems, this system does not use the static patterns discovered from Web usage data, instead it learns to make recommendations as the actions it performs in each situation. In the proposed method the authors combined the conceptual and usage information in order to gain a more general model of user behavior and improve the quality of web recommendations. A hybrid Web recommendation method is proposed by making use of the conceptual relationships among Web resources to derive a novel model of the problem, enriched with semantic knowledge about the usage behavior. The method is evaluated under different settings and it is shown how this method can improve the overall quality of recommendations.
Chapter Preview


The amount of information available on-line is increasing rapidly with the explosive growth of the World Wide Web and the advent of e-Commerce. Although this surely provides users with more options, at the same time makes it more difficult to find the “relevant” or “interesting” information from this great pool of information. This problem is commonly known as information overload: The state of having too much information to make a decision or remain informed about a topic. To address the problems caused by information overload, recommender systems have been introduced (Resnick & Varian, 1997). These systems can be defined as the personalized information technologies used to predict a user evaluation of a particular item (Deshpande & Karypis, 2004) or more generally as systems that guide users toward interesting or useful objects in a large space of possible options (Burke, 2002).

Recommender systems have been used in various applications ranging from predicting the products a customer is likely to buy (Shany et al., 2005), movies, music or news that might interest the user (Konstan et al., 1998; Zhang & Seo, 2001) and web pages that the user is likely to seek (Cooley et al., 1999; Fu et al., 2000; Joachims et al., 1997; Mobasher et al., 2000a), which is also the focus of this chapter. Web page recommendation is considered a user modeling or web personalization task (Eirinaki et al., 2004). One research area that has recently contributed greatly to this problem is web mining. Most of the systems developed in this field are based on web usage mining which is the process of applying data mining techniques to the discovery of usage patterns form web data (Srivastava et al., 2000). These systems are mainly concerned with analyzing web usage logs, discovering patterns from this data and making recommendations based on the extracted knowledge (Fu et al., 2000; Mobasher et al., 2000a; Shahabi et al., 1997; Zhang & Seo, 2001). One important characteristic of these systems is that unlike traditional recommender systems, which mainly base their decisions on user ratings on different items or other explicit feedbacks provided by the user (Deshpande & Karypis, 2004; Herlocker et al., 2000), these techniques discover user preferences from their implicit feedbacks, e.g. the web pages they have visited. More recently, systems that take advantage of domain knowledge, e.g. a combination of content, usage and even structure information of the web, have been introduced and shown superior results in the web page recommendation problem (Li & Zaiane, 2004; Mobasher et al., 2000b; Nakagawa & Mobasher, 2003).

Complete Chapter List

Search this Book:
Table of Contents
Bamshad Mobasher
Max Chevalier, Christine Julien, Chantal Soule-Dupuy
Chapter 1
Laurent Candillier, Kris Jack, Françoise Fessant, Frank Meyer
The aim of Recommender Systems is to help users to find items that they should appreciate from huge catalogues. In that field, collaborative... Sample PDF
State-of-the-Art Recommender Systems
Chapter 2
Neal Lathia
Recommender systems generate personalized content for each of its users, by relying on an assumption reflected in the interaction between people... Sample PDF
Computing Recommendations with Collaborative Filtering
Chapter 3
Edwin Simpson, Mark H. Butler
The increasing amount of available information has created a demand for better, more automated methods of finding and organizing different types of... Sample PDF
Analyzing Communal Tag Relationships for Enhanced Navigation and User Modeling
Chapter 4
Adaptive User Profiles  (pages 65-87)
Steve Cayzer, Elke Michlmayr
A major opportunity for collaborative knowledge management is the construction of user models which can be exploited to provide relevant... Sample PDF
Adaptive User Profiles
Chapter 5
Eugene Santos Jr., Hien Nguyen
In this chapter, we study and present our results on the problem of employing a cognitive user model for Information Retrieval (IR) in which a... Sample PDF
Modeling Users for Adaptive Information Retrieval by Capturing User Intent
Chapter 6
Mihaela Brut, Florence Sedes, Corinne Zayani
Inside the e-learning platforms, it is important to manage the user competencies profile and to recommend to each user the most suitable documents... Sample PDF
Ontology-Based User Competencies Modeling for E-Learning Recommender Systems
Chapter 7
Colum Foley, Alan F. Smeaton, Gareth J.F. Jones
Traditionally information retrieval (IR) research has focussed on a single user interaction modality, where a user searches to satisfy an... Sample PDF
Combining Relevance Information in a Synchronous Collaborative Information Retrieval Environment
Chapter 8
Charles Delalonde, Eddie Soulier
This research leverages information retrieval activity in order to build a network of organizational expertise in a distributed R&D laboratory. The... Sample PDF
DemonD: A Social Search Engine Built Upon the Actor-Network Theory
Chapter 9
Hager Karoui
In this chapter, the authors propose a case-based reasoning recommender system called COBRAS: a Peer-to-Peer (P2P) bibliographical reference... Sample PDF
COBRAS: Cooperative CBR Bibliographic Recommender System
Chapter 10
Zehra Cataltepe, Berna Altinel
As the amount, availability, and use of online music increase, music recommendation becomes an important field of research. Collaborative... Sample PDF
Music Recommendation by Modeling User's Preferred Perspectives of Content, Singer/Genre and Popularity
Chapter 11
Nima Taghipour, Ahmad Kardan
Information overload is no longer news; the explosive growth of the Internet has made this issue increasingly serious for Web users. Recommender... Sample PDF
Web Content Recommendation Methods Based on Reinforcement Learning
Chapter 12
Angela Carrillo-Ramos, Manuele Kirsch Pinheiro, Marlène Villanova-Oliver, Jérôme Gensel, Yolande Berbers
The authors of this chapter present a two-fold approach for adapting content information delivered to a group of mobile users. This approach is... Sample PDF
Collaborating Agents for Adaptation to Mobile Users
Chapter 13
Cristina Gena, Liliana Ardissono
This chapter describes the user-centered design approach we adopted in the development and evaluation of an adaptive Web site. The development of... Sample PDF
A User-Centered Approach to the Retrieval of Information in an Adaptive Web Site
Chapter 14
Antonella Carbonaro, Rodolfo Ferrini
Active learning is the ability of learners to carry out learning activities in such a way that they will be able to effectively and efficiently... Sample PDF
Personalized Information Retrieval in a Semantic-Based Learning Environment
Chapter 15
Hanh Huu Hoang, Tho Manh Nguyen, A Min Tjoa
Formulating unambiguous queries in the Semantic Web applications is a challenging task for users. This article presents a new approach in guiding... Sample PDF
A Semantic Web Based Approach for Context-Aware User Query Formulation and Information Retrieval
About the Contributors