Assessing Engagement During the Online Assessment of Real-World Skills

Assessing Engagement During the Online Assessment of Real-World Skills

J. Christine Harmes (Assessment Consultant, USA) and Steven L. Wise (Northwest Evaluation Association, USA)
DOI: 10.4018/978-1-5225-2584-4.ch004


The assessment of real-world skills will often require complex and innovative types of computer-based test items to provide more authentic assessment. Having information about how students remain engaged with the various innovative elements during an assessment is useful in both assessing the utility of different types of innovative test items and assessing the validity of the inferences made about the test scores of individual students. This chapter introduces the Item Engagement Index (IEI) and the Student Engagement Index (SEI) and demonstrates their use with a variety of innovative items that were pilot tested for a nursing licensure exam. The IEI provided useful information about the amount of student effort each innovative item received, while the SEI was found useful in identifying disengaged test takers.
Chapter Preview

Innovative Items

One of the most exciting and promising aspects of the use of technology in assessment is the set of expanded possibilities afforded by innovative item formats. In general, innovative items are technology-based test items or tasks that include functionality and features that extend beyond traditional text-based multiple-choice items. These innovative items have also been referred to as “technology-enhanced items” (Zenisky & Sireci, 2013). Helpful discussions of the range of item types and functionalities that can be used in the assessment of real-world skills can be found in Parshall, Harmes, Davey, and Pashley (2010), Scalise (2012), Scalise and Gifford (2006), and Sireci and Zenisky (2006).

Complete Chapter List

Search this Book: