Article Preview
Top1. Introduction To Evidence-Centered Design And Stealth Assessment
Today’s students are expected to develop 21st century skills, such as problem solving, creativity, and critical thinking (Partnership for 21st Century Learning, 2012). Such higher-order skills are necessary to be successful and productive in school, work, and life in general. It is thus important for educators to be able to accurately assess students on these complex skills. Assessments can help educators determine not only students’ current levels of these competencies, but also their strengths and weaknesses on particular facets of the skills. This information can assist educators in supporting their students to develop 21st century skills, as well as other important competencies such as content knowledge and dispositions. However, traditional formats for assessing learning and achievement, such as multiple-choice tests, often measure superficial skills and are stripped of the context in which knowledge and skills are applied (Shute, Leighton, Jang, & Chu, in press). Thus, an ongoing problem in education involves finding more authentic and valid, yet efficient, ways to assess students on these complex competencies. Stealth assessment (Shute, 2011) has been proposed as one of the most promising methods for assessing complex skills. It is the process of embedding assessments seamlessly into a computer-based learning or gaming environment such that the learner is unaware he or she being assessed.
Researchers generally agree that the development of an assessment has to follow a principled assessment design framework (AERA, APA, NCME, 1999; Kane, 2006) to be valid and reliable. Some leading principled assessment design frameworks include evidence-centered design (ECD), cognitive design system (CDS), and assessment engineering (AE). These three design frameworks are similar in their end goals, but vary in the processes they use to arrive at the goals (Shute, Leighton, Jang, & Chu, in press). In this paper, we discuss the hurdles we faced when using the evidence-centered design framework to implement stealth assessment and how we overcame those hurdles. Based on these hurdles, we make recommendations for stealth assessment best practices. We also present an ongoing project in which we are applying the lessons we have learned to more effectively and efficiently develop and implement stealth assessment.