Assessment in Authentic Environments: Designing Instruments and Reporting Results from Classroom-Based TPACK Research

Introduction

Research on the impact of instructional technology on learning outcomes have a history of finding no significant difference—the technology did not provide a measureable effect on students’ performance (e.g., Clark, 1983; Dynarksi et al., 2007). Kozma (1994) provided a counterpoint, noting that the proper focus is not the technology alone but the technology and the instructional method employed by the teacher. Mishra and Koehler’s framework of Technological Pedagogical Content Knowledge (TPACK; Mishra & Koehler, 2006) provides an even more comprehensive set of variables as it broadens Kozma’s interest in instructional method into the richer, more descriptive context of Shulman’s Pedagogical Content Knowledge (1987). Researchers can use TPACK to frame studies that attend to the interacting variables of content, pedagogy, and technology in the authentic environment of technology integration: classroom teachers engaged in standards-aligned instruction with intact groups of primary and secondary students. These more fully-contextualized, real-world studies may be able to shed light on which combinations offer no relative advantage and which do, indeed, afford a significant difference in learning outcomes.

As Schrum et al. (2007) noted in their discussion of TPACK, “Until the pedagogical methods that uniquely take advantage of a technology’s pedagogical affordances to achieve content-specific learning objectives are identified, it will not be possible to prepare teachers to make effective use of current and emerging technologies” (p. 460 [emphasis added]).The goal of this chapter is to review the state of the field in TPACK-related research to see whether and how this task is being carried out and to offer constructive, specific guidance to future research. Specifically, we will

• 1.

  Examine the extent to which TPACK-informed research has sought to observe the relative advantage of technology integration strategies in terms of student learning in elementary and secondary classrooms,

• 2.

  Identify and evaluate the methodology used in this research to identify exemplars, and

• 3.

  Advise TPACK-informed researchers as they conduct their own assessments of student learning in the authentic environment of technology integration: intact primary and secondary classrooms engaged in standards-aligned instruction.

In the spirit of TPACK’s attention to context, we will focus on the content areas of science and social studies—these are the areas of our own instructional and research expertise and are therefore the ones in which we are most competent to review others’ work with a critical but constructive lens. Our analysis and advice can inform work in other content areas as well but will directly address only these designated areas.