The purpose of this chapter is to address epistemological and methodological approaches to assessing assessment. The authors’ intent is to show how moving beyond paradigm wars and using multiple methods makes for good assessment. The authors explore ways qualitative and quantitative methods are complementary, as opposed to competing concepts, arguing that these methodologies in collaboration provide a much richer form of higher education assessment. The chapter begins with a review of the literature on qualitative, quantitative, and mixed method designs, which includes an overview of the history and contemporary landscape of the qualitative-quantitative debate. The chapter also highlights successful examples of mixed-method assessment at a mid-sized, private university, presented in general frameworks which can be used on any campus. The chapter concludes with recommendations for practitioners and future trends.
This chapter addresses epistemological and methodological approaches to assessment, with the intent to show how moving beyond paradigm wars and using multiple methods makes for good assessment. Higher education institutions are complex entities and thus require equally complex and comprehensive forms of assessing student learning and development. In this chapter, the authors explore the ways qualitative and quantitative methods are complementary, as opposed to competing concepts. Using both methodologies in collaboration provides a much richer form of assessment. Our approach to assessing assessment considers epistemological perspectives and presents practical mixed method examples.
The chapter is comprised of three main sections. The first section is a review of the literature on qualitative, quantitative, and mixed method designs, which includes an overview of the history and contemporary landscape of the qualitative-quantitative debate. The second section of this chapter highlights successful examples of mixed-method assessment at a mid-sized university. The examples provide not only specific instances, but also general frameworks which can be used on any campus. Finally, the chapter concludes with future trends and recommendations for practitioners. Specific strategies for maximizing the benefits of both assessment methods in any higher education setting are offered.
Given its location within the assessing assessment section of this text, the objectives of this chapter include:
Encouraging readers to consider multiple epistemological perspectives as they select assessment methods.
Encouraging readers to critically evaluate the potential effectiveness of quantitative and qualitative assessment techniques for their projects and carefully select rigorous techniques which can stand up to scrutiny.
Presenting a variety of specific mixed-method assessment examples within a series of frameworks which can be adapted for use in any higher education environment.
Offering practical suggestions for readers to consider prior to implementing their assessment projects, which are often high stakes activities.
The authors fully acknowledge the challenges of addressing the vast epistemological perspectives and language usage issues related to qualitative and quantitative methods. While a brief overview of these issues is included here, the intent of this chapter is not to provide an in-depth philosophical analysis. Instead, we strive to provide useful insight and recommendations for practitioners who are faced with assessment decisions on a daily basis.Top
Over the past twenty-five years, assessment has become a standard practice in higher education. Ideas of research and scholarship have historically been associated with the professorate, but they are rarely used for program planning and assessment purposes. Both research and assessment are a reality in higher education and have many overlapping procedures and functions (Erwin, 1991). Both require the systematic, rigorous collection and analysis of data to answer a question or solve a problem. Pike (2002) reminds us that there are “five elements of research that are essential for effective assessment: asking good questions, identifying appropriate methods, using appropriate measures, selecting representative participants, and communicating results effectively” (pp. 131-132). A major difference between assessment and research, however, is that research, especially quantitative research methods, are typically used to generate theory or conclusions applicable to many settings. Conversely, the primary intent of assessment is to understand the impact of and improve local and specific higher education practices. However, it should be recognized that assessment findings can also impact accountability concerns, including funding allocations, program development, and public perception.
Key Terms in this Chapter
Focus Group: A method of data collection where a small group (4-8) of people are collectively interviewed or engage in an informal, but documented discussion.
Qualitative: “Qualitative research is a situated activity that locates the observer in the world. It consists of a set of interpretive, material practices that make the world visible. These practices transform the world. They turn the world into a series of representations, including fieldnotes, interviews, conversations, photographs, recordings, and memos to the self. At this level, qualitative research involves an interpretive, naturalistic approach to the world…Qualitative researchers study things in their natural settings, attempting to make sense of, or interpret, phenomena in terms of the meanings people bring to them” (Denzin & Lincoln, 2005, p. 3).
Method: “A set of procedures and techniques for gathering and analyzing data.” (Strauss & Corbin, 1998, p. 3)
Constructivism: Researchers who use this way of understanding the world “develop subjective meanings of their experiences—meanings directed toward certain objects or things. These meanings are varied and multiple, leading the researcher to look for the complexity of views rather than narrow the meanings into a few categories or ideas. The goal of research, then, is to rely as much as possible on the participants’ views of the situation” (Creswell, 2007, p. 20).
Quantitative: A form of research which uses numeric values to represent relationships between different phenomena. Depending on the sampling method, findings may be generalized to larger populations.
Mixed Methods: A form of data collection and analysis which utilizes both quantitative and qualitative data in a single study.
Methodology: “A way of thinking about and studying social reality.” (Strauss & Corbin, 1998, p. 3)
Positivism: Based on Auguste Comte’s (French philosopher) original philosophical thinking, which “held that all genuine knowledge is based on sense experience and can only be advanced by means of observation and experiment…” (Cohen, Manion, & Morrison, 2000, p. 8)
Rubric: An assessment tool which allows for numerical judgments to be made about complex skills or phenomenon.
Epistemology: A branch of philosophy which describes what knowledge is and how or if it can be known.