Lessons Learned and Best Practices of Stealth Assessment

Lessons Learned and Best Practices of Stealth Assessment

Lubin Wang (Florida State University, Tallahassee, FL, USA), Valerie Shute (Florida State University, Tallahassee, FL, USA) and Gregory R. Moore (Florida State University, Tallahassee, FL, USA)
DOI: 10.4018/IJGCMS.2015100104
OnDemand PDF Download:
No Current Special Offers


Stealth assessment provides an innovative way to assess and ultimately support knowledge, skills, and other personal attributes within learning and gaming environments without disrupting students' flow. In this paper, the authors briefly discuss two challenges they encountered during the development of stealth assessments in two past projects (i.e., utility issues related to log files and validation issues related to in-game measures). They also present successful examples of designing and testing stealth assessments and describe the steps they are taking to apply the lessons they have learned to the ongoing development of a stealth assessment for problem solving skills. The authors conclude with suggestions for future research.
Article Preview

1. Introduction To Evidence-Centered Design And Stealth Assessment

Today’s students are expected to develop 21st century skills, such as problem solving, creativity, and critical thinking (Partnership for 21st Century Learning, 2012). Such higher-order skills are necessary to be successful and productive in school, work, and life in general. It is thus important for educators to be able to accurately assess students on these complex skills. Assessments can help educators determine not only students’ current levels of these competencies, but also their strengths and weaknesses on particular facets of the skills. This information can assist educators in supporting their students to develop 21st century skills, as well as other important competencies such as content knowledge and dispositions. However, traditional formats for assessing learning and achievement, such as multiple-choice tests, often measure superficial skills and are stripped of the context in which knowledge and skills are applied (Shute, Leighton, Jang, & Chu, in press). Thus, an ongoing problem in education involves finding more authentic and valid, yet efficient, ways to assess students on these complex competencies. Stealth assessment (Shute, 2011) has been proposed as one of the most promising methods for assessing complex skills. It is the process of embedding assessments seamlessly into a computer-based learning or gaming environment such that the learner is unaware he or she being assessed.

Researchers generally agree that the development of an assessment has to follow a principled assessment design framework (AERA, APA, NCME, 1999; Kane, 2006) to be valid and reliable. Some leading principled assessment design frameworks include evidence-centered design (ECD), cognitive design system (CDS), and assessment engineering (AE). These three design frameworks are similar in their end goals, but vary in the processes they use to arrive at the goals (Shute, Leighton, Jang, & Chu, in press). In this paper, we discuss the hurdles we faced when using the evidence-centered design framework to implement stealth assessment and how we overcame those hurdles. Based on these hurdles, we make recommendations for stealth assessment best practices. We also present an ongoing project in which we are applying the lessons we have learned to more effectively and efficiently develop and implement stealth assessment.

Complete Article List

Search this Journal:
Open Access Articles
Volume 14: 4 Issues (2022): Forthcoming, Available for Pre-Order
Volume 13: 4 Issues (2021): 2 Released, 2 Forthcoming
Volume 12: 4 Issues (2020)
Volume 11: 4 Issues (2019)
Volume 10: 4 Issues (2018)
Volume 9: 4 Issues (2017)
Volume 8: 4 Issues (2016)
Volume 7: 4 Issues (2015)
Volume 6: 4 Issues (2014)
Volume 5: 4 Issues (2013)
Volume 4: 4 Issues (2012)
Volume 3: 4 Issues (2011)
Volume 2: 4 Issues (2010)
Volume 1: 4 Issues (2009)
View Complete Journal Contents Listing