Assessment in Authentic Environments: Designing Instruments and Reporting Results from Classroom-Based TPACK Research

Assessment in Authentic Environments: Designing Instruments and Reporting Results from Classroom-Based TPACK Research

Thomas C. Hammond (Lehigh University, USA), R. Curby Alexander (University of North Texas, USA) and Alec M. Bodzin (Lehigh University, USA)
DOI: 10.4018/978-1-60960-750-0.ch003
OnDemand PDF Download:


The TPACK framework provides researchers with a robust framework for conducting research on technology integration in authentic environments, i.e., intact classrooms engaged in standards-aligned instruction. Researchers who wish to identify the value added by a promising technology-supported instructional strategy will need to assess student learning outcomes in these environments; unfortunately, collecting valid and reliable data on student learning in classroom research is extremely difficult. To date, few studies using TPACK in K-12 classrooms have included student learning outcomes in their research questions, and researchers are therefore left without models to guide their development, implementation, and analysis of assessments. This chapter draws upon the literature and our own research and assessment experiences in technology-integrated, standards-aligned classroom instruction to give examples and advice to researchers as they develop, analyze, and write up their observations of student learning outcomes. In particular, we focus on standard items, specifically multiple choice items, as an accepted (if limited) method for assessing student understanding. We seek to fill an existing gap in the literature between assessment advice for educational psychologists (who typically work outside of classroom settings) and advice given to teachers (who have lower thresholds for issues such as validity and reliability). Classroom researchers will benefit from this advice to develop, validate, and apply their own objective assessments. We focus on the content areas of science and social studies, but this advice can be applied to others as well.
Chapter Preview


Research on the impact of instructional technology on learning outcomes have a history of finding no significant difference—the technology did not provide a measureable effect on students’ performance (e.g., Clark, 1983; Dynarksi et al., 2007). Kozma (1994) provided a counterpoint, noting that the proper focus is not the technology alone but the technology and the instructional method employed by the teacher. Mishra and Koehler’s framework of Technological Pedagogical Content Knowledge (TPACK; Mishra & Koehler, 2006) provides an even more comprehensive set of variables as it broadens Kozma’s interest in instructional method into the richer, more descriptive context of Shulman’s Pedagogical Content Knowledge (1987). Researchers can use TPACK to frame studies that attend to the interacting variables of content, pedagogy, and technology in the authentic environment of technology integration: classroom teachers engaged in standards-aligned instruction with intact groups of primary and secondary students. These more fully-contextualized, real-world studies may be able to shed light on which combinations offer no relative advantage and which do, indeed, afford a significant difference in learning outcomes.

As Schrum et al. (2007) noted in their discussion of TPACK, “Until the pedagogical methods that uniquely take advantage of a technology’s pedagogical affordances to achieve content-specific learning objectives are identified, it will not be possible to prepare teachers to make effective use of current and emerging technologies” (p. 460 [emphasis added]).The goal of this chapter is to review the state of the field in TPACK-related research to see whether and how this task is being carried out and to offer constructive, specific guidance to future research. Specifically, we will

  • 1.

    Examine the extent to which TPACK-informed research has sought to observe the relative advantage of technology integration strategies in terms of student learning in elementary and secondary classrooms,

  • 2.

    Identify and evaluate the methodology used in this research to identify exemplars, and

  • 3.

    Advise TPACK-informed researchers as they conduct their own assessments of student learning in the authentic environment of technology integration: intact primary and secondary classrooms engaged in standards-aligned instruction.

In the spirit of TPACK’s attention to context, we will focus on the content areas of science and social studies—these are the areas of our own instructional and research expertise and are therefore the ones in which we are most competent to review others’ work with a critical but constructive lens. Our analysis and advice can inform work in other content areas as well but will directly address only these designated areas.

Complete Chapter List

Search this Book:
Editorial Advisory Board
Table of Contents
Robert N. Ronau, Christopher R. Rakes, Margaret L. Niess
Robert N. Ronau, Christopher R. Rakes, Margaret L. Niess
Chapter 1
Margaret L. Niess
Technology, pedagogy, and content knowledge (TPACK) is a dynamic lens that describes teacher knowledge required for designing, implementing, and... Sample PDF
Teacher Knowledge for Teaching with Technology: A TPACK Lens
Chapter 2
Matthew J. Koehler, Tae Seob Shin, Punya Mishra
In this chapter we reviewed a wide range of approaches to measure Technological Pedagogical Content Knowledge (TPACK). We identified recent... Sample PDF
How Do We Measure TPACK? Let Me Count the Ways
Chapter 3
Thomas C. Hammond, R. Curby Alexander, Alec M. Bodzin
The TPACK framework provides researchers with a robust framework for conducting research on technology integration in authentic environments, i.e.... Sample PDF
Assessment in Authentic Environments: Designing Instruments and Reporting Results from Classroom-Based TPACK Research
Chapter 4
Robert N. Ronau, Christopher R. Rakes
In this study, we examine the validity of the Comprehensive Framework for Teacher Knowledge (CFTK) through a systematic review and meta-analysis.... Sample PDF
A Comprehensive Framework for Teacher Knowledge (CFTK): Complexity of Individual Aspects and Their Interactions
Chapter 5
Lynn Bell, Nicole Juersivich, Thomas C. Hammond, Randy L. Bell
Effective teachers across K-12 content areas often use visual representations to promote conceptual understanding, but these static representations... Sample PDF
The TPACK of Dynamic Representations
Chapter 6
Erica C. Boling, Jeanine Beatty
This chapter informs teacher educators and individuals involved in teacher professional development about the tensions that frequently arise when... Sample PDF
Overcoming the Tensions and Challenges of Technology Integration: How Can We Best Support our Teachers?
Chapter 7
John K. Lee, Meghan M. Manfra
To address the myriad effects that emerge from using technology in social studies, we introduce in this chapter the concept of vernaculars to... Sample PDF
TPACK Vernaculars in Social Studies Research
Chapter 8
Stephen J. Pape, Karen E. Irving, Clare V. Bell, Melissa L. Shirley, Douglas T. Owens, Sharilyn Owens, Jonathan D. Bostic, Soon Chun Lee
Classroom Connectivity Technology (CCT) can serve as a tool for creating contexts in which students engage in mathematical thinking leading to... Sample PDF
Principles of Effective Pedagogy within the Context of Connected Classroom Technology: Implications for Teacher Knowledge
Chapter 9
Christopher J. Johnston, Patricia S. Moyer-Packenham
Multiple existing frameworks address aspects of teachers’ knowledge for teaching mathematics with technology. This study proposes the integration of... Sample PDF
A Model for Examining the Criteria Used by Pre-Service Elementary Teachers in Their Evaluation of Technology for Mathematics Teaching
Chapter 10
Joseph M. Piro, Nancy Marksbury
With the continuing shift of instructional media to digital sources occurring in classrooms around the world, the role of technology instruction in... Sample PDF
Technologizing Teaching: Using the WebQuest to Enhance Pre-Service Education
Chapter 11
Travis K. Miller
This chapter details a theoretical framework for effective implementation and study of technology when used in mathematics education. Based on... Sample PDF
A Theoretical Framework for Implementing Technology for Mathematics Learning
Chapter 12
David A. Slykhuis, Rebecca McNall Krall
In this review of recent literature on the use of technology to teach science content, 143 articles from 8 science education journals were selected... Sample PDF
Successful Implementation of Technology to Teach Science: Research Implications
Chapter 13
Irina Lyublinskaya, Nelly Tournaki
A year-long PD program was provided to four NYC integrated algebra teachers. The PD comprised of teacher authoring of curriculum that incorporated... Sample PDF
The Effects of Teacher Content Authoring on TPACK and on Student Achievement in Algebra: Research on Instruction with the TI-Nspire™ Handheld
Chapter 14
Robert N. Ronau, Christopher R. Rakes
This chapter examines issues surrounding the design of research in educational technology and teacher knowledge. The National Research Council... Sample PDF
Making the Grade: Reporting Educational Technology and Teacher Knowledge Research
About the Contributors