The Evaluation of Adaptive Systems

Abstract

This chapter discusses the evaluation of adaptive systems, as it is important to investigate the effectiveness of adaptation and how it may be optimised. It argues that evaluation is often neglected because of difficulties in attributing cause, defining effectiveness, obtaining significant results and getting resources. Using example evaluations of an interactive instruction system, it shows how these issues can be tackled by separating concerns, using learning curves, splitting subjects into groups, and using experts and models instead of subjects. The author hopes that these methods will be more widely adopted for evaluating adaptive systems (not just adaptive instruction systems), producing more insight in experimental data and its causes, delivering more significant results, and making evaluations less costly.