Qualitative and Quantitative Methods as Complementary Assessment Tools

Qualitative and Quantitative Methods as Complementary Assessment Tools

Sheila S. Thompson (Metropolitan State College of Denver, USA) and Annemarie Vaccaro (University of Rhode Island, USA)
DOI: 10.4018/978-1-60566-667-9.ch007
OnDemand PDF Download:
$37.50

Abstract

The purpose of this chapter is to address epistemological and methodological approaches to assessing assessment. The authors’ intent is to show how moving beyond paradigm wars and using multiple methods makes for good assessment. The authors explore ways qualitative and quantitative methods are complementary, as opposed to competing concepts, arguing that these methodologies in collaboration provide a much richer form of higher education assessment. The chapter begins with a review of the literature on qualitative, quantitative, and mixed method designs, which includes an overview of the history and contemporary landscape of the qualitative-quantitative debate. The chapter also highlights successful examples of mixed-method assessment at a mid-sized, private university, presented in general frameworks which can be used on any campus. The chapter concludes with recommendations for practitioners and future trends.
Chapter Preview
Top

Introduction

This chapter addresses epistemological and methodological approaches to assessment, with the intent to show how moving beyond paradigm wars and using multiple methods makes for good assessment. Higher education institutions are complex entities and thus require equally complex and comprehensive forms of assessing student learning and development. In this chapter, the authors explore the ways qualitative and quantitative methods are complementary, as opposed to competing concepts. Using both methodologies in collaboration provides a much richer form of assessment. Our approach to assessing assessment considers epistemological perspectives and presents practical mixed method examples.

The chapter is comprised of three main sections. The first section is a review of the literature on qualitative, quantitative, and mixed method designs, which includes an overview of the history and contemporary landscape of the qualitative-quantitative debate. The second section of this chapter highlights successful examples of mixed-method assessment at a mid-sized university. The examples provide not only specific instances, but also general frameworks which can be used on any campus. Finally, the chapter concludes with future trends and recommendations for practitioners. Specific strategies for maximizing the benefits of both assessment methods in any higher education setting are offered.

Given its location within the assessing assessment section of this text, the objectives of this chapter include:

  • 1.

    Encouraging readers to consider multiple epistemological perspectives as they select assessment methods.

  • 2.

    Encouraging readers to critically evaluate the potential effectiveness of quantitative and qualitative assessment techniques for their projects and carefully select rigorous techniques which can stand up to scrutiny.

  • 3.

    Presenting a variety of specific mixed-method assessment examples within a series of frameworks which can be adapted for use in any higher education environment.

  • 4.

    Offering practical suggestions for readers to consider prior to implementing their assessment projects, which are often high stakes activities.

The authors fully acknowledge the challenges of addressing the vast epistemological perspectives and language usage issues related to qualitative and quantitative methods. While a brief overview of these issues is included here, the intent of this chapter is not to provide an in-depth philosophical analysis. Instead, we strive to provide useful insight and recommendations for practitioners who are faced with assessment decisions on a daily basis.

Top

Background

Over the past twenty-five years, assessment has become a standard practice in higher education. Ideas of research and scholarship have historically been associated with the professorate, but they are rarely used for program planning and assessment purposes. Both research and assessment are a reality in higher education and have many overlapping procedures and functions (Erwin, 1991). Both require the systematic, rigorous collection and analysis of data to answer a question or solve a problem. Pike (2002) reminds us that there are “five elements of research that are essential for effective assessment: asking good questions, identifying appropriate methods, using appropriate measures, selecting representative participants, and communicating results effectively” (pp. 131-132). A major difference between assessment and research, however, is that research, especially quantitative research methods, are typically used to generate theory or conclusions applicable to many settings. Conversely, the primary intent of assessment is to understand the impact of and improve local and specific higher education practices. However, it should be recognized that assessment findings can also impact accountability concerns, including funding allocations, program development, and public perception.

Key Terms in this Chapter

Focus Group: A method of data collection where a small group (4-8) of people are collectively interviewed or engage in an informal, but documented discussion.

Qualitative: “Qualitative research is a situated activity that locates the observer in the world. It consists of a set of interpretive, material practices that make the world visible. These practices transform the world. They turn the world into a series of representations, including fieldnotes, interviews, conversations, photographs, recordings, and memos to the self. At this level, qualitative research involves an interpretive, naturalistic approach to the world…Qualitative researchers study things in their natural settings, attempting to make sense of, or interpret, phenomena in terms of the meanings people bring to them” (Denzin & Lincoln, 2005, p. 3).

Method: “A set of procedures and techniques for gathering and analyzing data.” (Strauss & Corbin, 1998, p. 3)

Constructivism: Researchers who use this way of understanding the world “develop subjective meanings of their experiences—meanings directed toward certain objects or things. These meanings are varied and multiple, leading the researcher to look for the complexity of views rather than narrow the meanings into a few categories or ideas. The goal of research, then, is to rely as much as possible on the participants’ views of the situation” (Creswell, 2007, p. 20).

Quantitative: A form of research which uses numeric values to represent relationships between different phenomena. Depending on the sampling method, findings may be generalized to larger populations.

Mixed Methods: A form of data collection and analysis which utilizes both quantitative and qualitative data in a single study.

Methodology: “A way of thinking about and studying social reality.” (Strauss & Corbin, 1998, p. 3)

Positivism: Based on Auguste Comte’s (French philosopher) original philosophical thinking, which “held that all genuine knowledge is based on sense experience and can only be advanced by means of observation and experiment…” (Cohen, Manion, & Morrison, 2000, p. 8)

Rubric: An assessment tool which allows for numerical judgments to be made about complex skills or phenomenon.

Epistemology: A branch of philosophy which describes what knowledge is and how or if it can be known.

Complete Chapter List

Search this Book:
Reset
Dedication
Editorial Advisory Board
Table of Contents
Preface
Christopher S. Schreiner
Acknowledgment
Christopher S. Schreiner
Chapter 1
Melissa A. Dyehouse, John Y. Baek, Richard A. Lesh
This chapter describes a model for evaluating complex organizations or systems. The design assessment model the authors propose is a response to... Sample PDF
Multi-Tier Design Assessment in the Development of Complex Organizational Systems
$37.50
Chapter 2
Hedva Lewittes
In this chapter critical thinking is assessed using two critical thinking learning outcomes that were required for the State University of New... Sample PDF
A Critical Thinking Rubric as the Basis of Assessment and Curriculum
$37.50
Chapter 3
Suzanne Pieper, Erika Edwards, Brandon Haist, Walter Nolan
The purpose of this chapter is to review literature over the past ten years regarding technology tools that are being used in higher education to... Sample PDF
A Survey of Effective Technologies to Assess Student Learning
$37.50
Chapter 4
John Baer, Sharon S. McKool
The Consensual Assessment Technique is a powerful tool used by creativity researchers in which panels of expert judges are asked to rate the... Sample PDF
Assessing Creativity Using the Consensual Assessment Technique
$37.50
Chapter 5
Christine Charyton, Zorana Ivcevic, Jonathan A. Plucker, James C. Kaufman
This chapter discusses creativity assessment as a means for evaluating skills required in higher education. Creativity is assessed in the context of... Sample PDF
Creativity Assessment in Higher Education
$37.50
Chapter 6
Asao B. Inoue
This chapter articulates writing assessment as a technology, theorized with three aspects (power, parts, and purpose), accounting for the ways in... Sample PDF
The Technology of Writing Assessment and Racial Validity
$37.50
Chapter 7
Sheila S. Thompson, Annemarie Vaccaro
The purpose of this chapter is to address epistemological and methodological approaches to assessing assessment. The authors’ intent is to show how... Sample PDF
Qualitative and Quantitative Methods as Complementary Assessment Tools
$37.50
Chapter 8
Teresa Flateby
The development of the Cognitive Level and Quality of Writing Assessment online system is described in this chapter. Beginning with needs identified... Sample PDF
Effects of Assessment Results on a Writing and Thinking Rubric
$37.50
Chapter 9
Barbara D’Angelo, Barry Maid
Outcomes-based assessment provides data for programs to demonstrate student learning as a result of their enrollment in the program and to assess... Sample PDF
Assessing Outcomes in a Technical Communication Capstone
$37.50
Chapter 10
Sonya Borton, Alanna Frost, Kate Warrington
As Jacqueline Jones Royster articulated at the 2006 Conference on College Composition and Communication, English departments are already assessing... Sample PDF
Assessing the Composition Program on Our Own Terms
$37.50
Chapter 11
Joan Aitken
This chapter uses a case study to exemplify one approach to assessment of three instructional delivery formats: (a) online, (b) distance, satellite... Sample PDF
A Case Study of Instructional Delivery Formats
$37.50
Chapter 12
Victor W. Brunsden
The author present a case-study of a classroom technique that allows assessment and some remediation of several shortcomings of college student... Sample PDF
Inverting the Remedial Mathematics Classroom with Alternative Assessment
$37.50
Chapter 13
David A. Eubanks
This chapter describes Coker College’s subjective performance assessment program to rate student thinking and communication skills. It uses a... Sample PDF
A Case Study of Authentic Assessment
$37.50
Chapter 14
P. Tokyo Kang, David Gugin
This chapter reports an outcomes assessment study conducted at the University of Guam. The assessment project was conducted during the 2006-07 and... Sample PDF
Outcomes Assessment in Japanese Language Instruction
$37.50
Chapter 15
Barika Barboza, Frances Singh
This chapter describes an outcomes assessment study completed in a basic composition course at a small urban open admissions community college. The... Sample PDF
Assessing the Effectiveness of a Basic Writing Course
$37.50
Chapter 16
Lorraine Gilpin, Yasar Bodur, Kathleen Crawford
Peer assessment holds tremendous potential to positively impact the development of preservice teachers. The purpose of this chapter is to describe... Sample PDF
Peer Assessment for Development of Preservice Teachers
$37.50
Chapter 17
Charlotte Brammer, Rhonda Parker
In 2002, Samford University began working on a long-term learning assessment plan designed to evaluate its undergraduates’ competencies in written... Sample PDF
Workshops and E-Portfolios as Transformational Assessment
$37.50
Chapter 18
Daniel F. Chambliss
This chapter describes how the trend favoring assessment initiatives of a system-wide scope such as program review and collegiate learning... Sample PDF
A Neglected Necessity in Liberal Arts Assessment: The Student as the Unit of Analysis
$37.50
Chapter 19
Deirdre Pettipiece, Timothy Ray, Justin Everett
Perhaps due to its applicability as a performance of skill sets in virtually all disciplines, writing as a mechanism for measuring student success... Sample PDF
Redefining Writing Reality Multi-Modal Writing and Assessment
$37.50
Chapter 20
Sean A. McKitrick
This chapter introduces methods that can be used to engage faculty in the assessment process, working within a shared governance structure in... Sample PDF
Engaging Faculty as a Strategic Choice in Assessment
$37.50
Chapter 21
Steven M. Culver, Ray VanDyke
There is much in the assessment literature about the necessity of developing a culture of assessment and mandates from accrediting bodies include... Sample PDF
Developing a Receptive and Faculty-Focused Environment for Assessment
$37.50
Chapter 22
John Wittman
This chapter argues that as primary stakeholders in writing program assessment, students and instructors need to be included proactively in... Sample PDF
New Collaborations for Writing Program Assessment
$37.50
Chapter 23
Mya Poe
The study of racial-ethnic group differences on educational tests has yielded a substantial body of research internationally in the last decade. In... Sample PDF
Reporting Race and Ethnicity in International Assessment
$37.50
Chapter 24
Joan Hawthorne, Tatyana Dumova, April Bradley, Daphne Pederson
In this chapter the authors describe a method developed to assess the outcome of a “cultural familiarity” general education goal. Challenges in... Sample PDF
Method Development for Assessing a Diversity Goal
$37.50
About the Contributors