Problems and Pitfalls in the Evaluation of Adaptive Systems

Problems and Pitfalls in the Evaluation of Adaptive Systems

Stephan Weibelzahl (Fraunhoffer IESE, Germany)
Copyright: © 2005 |Pages: 15
DOI: 10.4018/978-1-59140-567-2.ch015
OnDemand PDF Download:


Empirical studies with adaptive systems offer many advantages and opportunities. Nevertheless, there is still a lack of evaluation studies. This chapter lists several problems and pitfalls that arise when evaluating an adaptive system, and provides guidelines and recommendations for workarounds or even avoidance of these problems. Among other things the following issues are covered: relating evaluation studies to the development cycle; saving resources; specifying control conditions, sample, and criteria; asking users for adaptivity effects; reporting results. An overview of existing evaluation frameworks shows which of these problems have been addressed and in which way.

Complete Chapter List

Search this Book:
Table of Contents
Peter Brusilovsky
Sherry Y. Chen, George D. Magoulas
Chapter 1
Hanna Stelmaszewska, Ann Blandford, George Buchanan
Hypermedia systems allow information to be created, stored, accessed, and manipulated in a variety of ways. One example of such a system is a... Sample PDF
Designing to Change Users' Information Seeking Behaviour
Chapter 2
Eelco Herder, Betsy van Dijk
The analysis of the structure of Web sites and patterns of user navigation through these sites is gaining attention from different disciplines, as... Sample PDF
Site Structure and User Navigation: Models, Measures and Methods
Chapter 3
Geert-Jan Houben, Lora Aroyo, Paul De Bra, Darina Dicheva
This chapter presents main issues and the state of the art of research on adaptation engineering in adaptive concept-based systems. Adaptive... Sample PDF
Adaptation Engineering in Adaptive Concept-Based System
Chapter 4
Carlos Miguel Tobar, Ivan Luiz Marques Ricarte
This chapter presents the Extended Abstract Categorisation Map, which provides a conceptual tool to evaluate the coverage of adaptive hypermedia... Sample PDF
The Extended Abstract Categorization Map (E-ACM)
Chapter 5
Alexandros Paramythis, Constantine Stephanidis
This chapter introduces a framework intended for facilitating the implementation of Web-based adaptive hypermedia systems. The framework is... Sample PDF
A Generic Adaptation Framework for Web-Based Hypermedia Systems
Chapter 6
Martin E. Muller
This chapter demonstrates the use of machine learning techniques in adaptive hypermedia systems. A generic machine learning scenario is described... Sample PDF
Learning Adaptive Behaviour
Chapter 7
Martha Koutri, Nikolaos Avouris, Sophia Daskalaki
This chapter discusses Web usage mining techniques that can be applied for building adaptive hypermedia systems. These techniques are used for... Sample PDF
A Survey of Web-Usage Mining: Techniques for Building Web-Based Adaptive Hypermedia Systems
Chapter 8
Miguel-Ángel Sicila, Elena García Barriocanal
daptive hypermedia applications are aimed at tailoring hypermedia structures according to some form of user model, in an attempt to increase the... Sample PDF
Information Imperfection as an Inherent Characteristic of Adaptive Hypermedia: Imprecise Models of Users and Interactions
Chapter 9
Yutaka Matsuo
This chapter presents discussion of word weighting algorithms in user modelling and adaptive information systems. We specifically address two types... Sample PDF
Work Weighting Based on User's Browsing History
Chapter 10
George Lepouras, Costas Vassilakis
This chapter presents an architecture for supporting the creation of adaptive virtual reality museums on the Web. It argues whether the task of... Sample PDF
Adaptive Virtual Reality Museums on the Web
Chapter 11
Serge Garlatti, Sébastien Iskal, Philippe Tanguy
This chapter presents SCARCE, a flexible adaptive hypermedia environment based on virtual document and the semantic Web. After a short state of the... Sample PDF
SCARCE: An Adaptive Hypermedia Environment Based on Virtual Documents and Semantic Web
Chapter 12
Liana Razmerita
This chapter focuses on the role of user models and user modelling for enhanced, personalised user support within knowledge management systems... Sample PDF
User Modelling and Personalisation of Knowledge Management Systems
Chapter 13
Judith Masthoff, Lyn Pemberton
Adaptive hypermedia techniques have until now been applied mainly to Web pages populated by text and images. In this chapter, we explore how these... Sample PDF
Adaptive Hypermedia for Personalised TV
Chapter 14
George D. Lekakos, George M. Giaglis
In this chapter, we discuss personalisation of advertisements in the digital TV environment and propose an effective personalisation approach... Sample PDF
Personalisation of Advertisements in the Digital TV Context
Chapter 15
Stephan Weibelzahl
Empirical studies with adaptive systems offer many advantages and opportunities. Nevertheless, there is still a lack of evaluation studies. This... Sample PDF
Problems and Pitfalls in the Evaluation of Adaptive Systems
Chapter 16
Stephen Sobol, Catherine Stones
Web-based content is increasingly delivered via dynamic methods. Visualisation tools are required which reveal how users interact with such data... Sample PDF
Evaluating Usage of Dynamic Web Site Structures Through Visualisation
Glossary of Terms
About the Authors