The TPACK framework provides researchers with a robust framework for conducting research on technology integration in authentic environments, i.e., intact classrooms engaged in standards-aligned instruction. Researchers who wish to identify the value added by a promising technology-supported instructional strategy will need to assess student learning outcomes in these environments; unfortunately, collecting valid and reliable data on student learning in classroom research is extremely difficult. To date, few studies using TPACK in K-12 classrooms have included student learning outcomes in their research questions, and researchers are therefore left without models to guide their development, implementation, and analysis of assessments. This chapter draws upon the literature and our own research and assessment experiences in technology-integrated, standards-aligned classroom instruction to give examples and advice to researchers as they develop, analyze, and write up their observations of student learning outcomes. In particular, we focus on standard items, specifically multiple choice items, as an accepted (if limited) method for assessing student understanding. We seek to fill an existing gap in the literature between assessment advice for educational psychologists (who typically work outside of classroom settings) and advice given to teachers (who have lower thresholds for issues such as validity and reliability). Classroom researchers will benefit from this advice to develop, validate, and apply their own objective assessments. We focus on the content areas of science and social studies, but this advice can be applied to others as well.
Research on the impact of instructional technology on learning outcomes have a history of finding no significant difference—the technology did not provide a measureable effect on students’ performance (e.g., Clark, 1983; Dynarksi et al., 2007). Kozma (1994) provided a counterpoint, noting that the proper focus is not the technology alone but the technology and the instructional method employed by the teacher. Mishra and Koehler’s framework of Technological Pedagogical Content Knowledge (TPACK; Mishra & Koehler, 2006) provides an even more comprehensive set of variables as it broadens Kozma’s interest in instructional method into the richer, more descriptive context of Shulman’s Pedagogical Content Knowledge (1987). Researchers can use TPACK to frame studies that attend to the interacting variables of content, pedagogy, and technology in the authentic environment of technology integration: classroom teachers engaged in standards-aligned instruction with intact groups of primary and secondary students. These more fully-contextualized, real-world studies may be able to shed light on which combinations offer no relative advantage and which do, indeed, afford a significant difference in learning outcomes.
As Schrum et al. (2007) noted in their discussion of TPACK, “Until the pedagogical methods that uniquely take advantage of a technology’s pedagogical affordances to achieve content-specific learning objectives are identified, it will not be possible to prepare teachers to make effective use of current and emerging technologies” (p. 460 [emphasis added]).The goal of this chapter is to review the state of the field in TPACK-related research to see whether and how this task is being carried out and to offer constructive, specific guidance to future research. Specifically, we will
Examine the extent to which TPACK-informed research has sought to observe the relative advantage of technology integration strategies in terms of student learning in elementary and secondary classrooms,
Identify and evaluate the methodology used in this research to identify exemplars, and
Advise TPACK-informed researchers as they conduct their own assessments of student learning in the authentic environment of technology integration: intact primary and secondary classrooms engaged in standards-aligned instruction.
In the spirit of TPACK’s attention to context, we will focus on the content areas of science and social studies—these are the areas of our own instructional and research expertise and are therefore the ones in which we are most competent to review others’ work with a critical but constructive lens. Our analysis and advice can inform work in other content areas as well but will directly address only these designated areas.