Assessment of Task-Specific Expertise

Abstract

Main implication of the expertise reversal effect is the need to tailor instructional techniques and procedures to changing levels of learner expertise in a specific task domain. In order to design adaptive procedures capable of tailoring instruction in real-time, it is necessary to have online measures of learner expertise. Such measures should be rapid enough to be used in real time. At the same time, they need to have sufficient diagnostic power to detect different levels of task-specific expertise. One of the previously mentioned reasons for low practical applicability of the results of studies in Aptitude-Treatment Interactions were inadequate aptitude measures. Most of the assessment methods used in those studies were psychometric instruments designed for selection purposes (e.g., large batteries of aptitude tests based on artificially simplified tasks administered mostly in laboratory conditions). Another suggested reason was unsuitability of those methods for dynamic, real-time applications while learners proceeded through a single learning session. This chapter describes a rapid diagnostic approach to the assessment of learner task-specific expertise that has been intentionally designed for rapid online application in adaptive learning environments. The method was developed using an analogy to experimental procedures applied in classical studies of chess expertise mentioned in Chapter I. In those studies, realistic board configurations were briefly presented for subsequent replications. With the described diagnostic approach, learners are briefly presented with a problem situation and required to indicate their first solution step in this problem situation or to rapidly verify suggested steps at various stages of a problem solution procedure.