Shopping Cart | Login | Register | Language: English

Handbook of Research on Assessment Technologies, Methods, and Applications in Higher Education

Release Date: May, 2009. Copyright © 2009. 500 pages.
Select a Format:
Hardcover
$265.00
Current Promotions:
Free E-Access*
In Stock. Have it as soon as Aug. 1 with express shipping*.
DOI: 10.4018/978-1-60566-667-9, ISBN13: 9781605666679, ISBN10: 160566667X, EISBN13: 9781605666686
Cite Book

MLA

Schreiner, Christopher S. "Handbook of Research on Assessment Technologies, Methods, and Applications in Higher Education." IGI Global, 2009. 1-500. Web. 29 Jul. 2014. doi:10.4018/978-1-60566-667-9

APA

Schreiner, C. S. (2009). Handbook of Research on Assessment Technologies, Methods, and Applications in Higher Education (pp. 1-500). Hershey, PA: IGI Global. doi:10.4018/978-1-60566-667-9

Chicago

Schreiner, Christopher S. "Handbook of Research on Assessment Technologies, Methods, and Applications in Higher Education." 1-500 (2009), accessed July 29, 2014. doi:10.4018/978-1-60566-667-9

Export Reference

Mendeley
Favorite
Handbook of Research on Assessment Technologies, Methods, and Applications in Higher Education
Access on Platform
Browse by Subject
Top

Description

Educational institutions across the globe have begun to place value on the technology of assessment instruments as they reflect what is valued in learning and deemed worthy of measurement.

The Handbook of Research on Assessment Technologies, Methods, and Applications in Higher Education combines in-depth, multi-disciplinary research in learning assessment to provide a fresh look at its impact on academic life. A significant reference source for practitioners, academicians, and researchers in related fields, this Handbook of Research contains not only technological assessments, but also technologies and assumptions about assessment and learning involving race, cultural diversity, and creativity.

Top

Table of Contents and List of Contributors

Search this Book: Reset
Table of Contents
Preface
Christopher S. Schreiner
Chapter 1
Melissa A. Dyehouse, John Y. Baek, Richard A. Lesh
This chapter describes a model for evaluating complex organizations or systems. The design assessment model the authors propose is a response to... Sample PDF
Multi-Tier Design Assessment in the Development of Complex Organizational Systems
$37.50
Chapter 2
Hedva Lewittes
In this chapter critical thinking is assessed using two critical thinking learning outcomes that were required for the State University of New... Sample PDF
A Critical Thinking Rubric as the Basis of Assessment and Curriculum
$37.50
Chapter 3
Suzanne Pieper, Erika Edwards, Brandon Haist, Walter Nolan
The purpose of this chapter is to review literature over the past ten years regarding technology tools that are being used in higher education to... Sample PDF
A Survey of Effective Technologies to Assess Student Learning
$37.50
Chapter 4
John Baer, Sharon S. McKool
The Consensual Assessment Technique is a powerful tool used by creativity researchers in which panels of expert judges are asked to rate the... Sample PDF
Assessing Creativity Using the Consensual Assessment Technique
$37.50
Chapter 5
Christine Charyton, Zorana Ivcevic, Jonathan A. Plucker, James C. Kaufman
This chapter discusses creativity assessment as a means for evaluating skills required in higher education. Creativity is assessed in the context of... Sample PDF
Creativity Assessment in Higher Education
$37.50
Chapter 6
Asao B. Inoue
This chapter articulates writing assessment as a technology, theorized with three aspects (power, parts, and purpose), accounting for the ways in... Sample PDF
The Technology of Writing Assessment and Racial Validity
$37.50
Chapter 7
Sheila S. Thompson, Annemarie Vaccaro
The purpose of this chapter is to address epistemological and methodological approaches to assessing assessment. The authors’ intent is to show how... Sample PDF
Qualitative and Quantitative Methods as Complementary Assessment Tools
$37.50
Chapter 8
Teresa Flateby
The development of the Cognitive Level and Quality of Writing Assessment online system is described in this chapter. Beginning with needs identified... Sample PDF
Effects of Assessment Results on a Writing and Thinking Rubric
$37.50
Chapter 9
Barbara D’Angelo, Barry Maid
Outcomes-based assessment provides data for programs to demonstrate student learning as a result of their enrollment in the program and to assess... Sample PDF
Assessing Outcomes in a Technical Communication Capstone
$37.50
Chapter 10
Sonya Borton, Alanna Frost, Kate Warrington
As Jacqueline Jones Royster articulated at the 2006 Conference on College Composition and Communication, English departments are already assessing... Sample PDF
Assessing the Composition Program on Our Own Terms
$37.50
Chapter 11
Joan Aitken
This chapter uses a case study to exemplify one approach to assessment of three instructional delivery formats: (a) online, (b) distance, satellite... Sample PDF
A Case Study of Instructional Delivery Formats
$37.50
Chapter 12
Victor W. Brunsden
The author present a case-study of a classroom technique that allows assessment and some remediation of several shortcomings of college student... Sample PDF
Inverting the Remedial Mathematics Classroom with Alternative Assessment
$37.50
Chapter 13
David A. Eubanks
This chapter describes Coker College’s subjective performance assessment program to rate student thinking and communication skills. It uses a... Sample PDF
A Case Study of Authentic Assessment
$37.50
Chapter 14
P. Tokyo Kang, David Gugin
This chapter reports an outcomes assessment study conducted at the University of Guam. The assessment project was conducted during the 2006-07 and... Sample PDF
Outcomes Assessment in Japanese Language Instruction
$37.50
Chapter 15
Barika Barboza, Frances Singh
This chapter describes an outcomes assessment study completed in a basic composition course at a small urban open admissions community college. The... Sample PDF
Assessing the Effectiveness of a Basic Writing Course
$37.50
Chapter 16
Lorraine Gilpin, Yasar Bodur, Kathleen Crawford
Peer assessment holds tremendous potential to positively impact the development of preservice teachers. The purpose of this chapter is to describe... Sample PDF
Peer Assessment for Development of Preservice Teachers
$37.50
Chapter 17
Charlotte Brammer, Rhonda Parker
In 2002, Samford University began working on a long-term learning assessment plan designed to evaluate its undergraduates’ competencies in written... Sample PDF
Workshops and E-Portfolios as Transformational Assessment
$37.50
Chapter 18
Daniel F. Chambliss
This chapter describes how the trend favoring assessment initiatives of a system-wide scope such as program review and collegiate learning... Sample PDF
A Neglected Necessity in Liberal Arts Assessment: The Student as the Unit of Analysis
$37.50
Chapter 19
Deirdre Pettipiece, Timothy Ray, Justin Everett
Perhaps due to its applicability as a performance of skill sets in virtually all disciplines, writing as a mechanism for measuring student success... Sample PDF
Redefining Writing Reality Multi-Modal Writing and Assessment
$37.50
Chapter 20
Sean A. McKitrick
This chapter introduces methods that can be used to engage faculty in the assessment process, working within a shared governance structure in... Sample PDF
Engaging Faculty as a Strategic Choice in Assessment
$37.50
Chapter 21
Steven M. Culver, Ray VanDyke
There is much in the assessment literature about the necessity of developing a culture of assessment and mandates from accrediting bodies include... Sample PDF
Developing a Receptive and Faculty-Focused Environment for Assessment
$37.50
Chapter 22
John Wittman
This chapter argues that as primary stakeholders in writing program assessment, students and instructors need to be included proactively in... Sample PDF
New Collaborations for Writing Program Assessment
$37.50
Chapter 23
Mya Poe
The study of racial-ethnic group differences on educational tests has yielded a substantial body of research internationally in the last decade. In... Sample PDF
Reporting Race and Ethnicity in International Assessment
$37.50
Chapter 24
Joan Hawthorne, Tatyana Dumova, April Bradley, Daphne Pederson
In this chapter the authors describe a method developed to assess the outcome of a “cultural familiarity” general education goal. Challenges in... Sample PDF
Method Development for Assessing a Diversity Goal
$37.50
Top

Reviews and Testimonials

The purpose of the research in this book, then, is to help educators make their own assessment efforts more articulate and systematic by demonstrating applied research in cases and initiatives.

– Christopher S. Schreiner, University of Guam, Guam

There is a stunning range of inquiry in this well-edited book, which certainly exceeds the bounds of the usual handbook in so far as it is always readable and stimulating. Every department chair and assessment coordinator needs a copy, but so do faculty members seeking to get aboard the assessment train that has already left the station. Bravo to IGI Global and the editor for gathering such exceptional essays and articles under one cover!

– Dr. Michel Pharand, The Disraeli Project, Queen's University, Canada

This new and comprehensive Handbooks of Research is a mine of information that anyone in the education and information technology fields will want at their elbow as well as all libraries on the shelves. Assessment is THE topic of the hour: administrators and educatos cannot afford to be wihtout this crucial guide.

– Michel Pharand, Queen's University, Canada
Top

Topics Covered

  • Assessment applications and initiatives
  • Assessment technologies and instruments
  • Collaborations for writing program assessment
  • Communication workshops and e-portfolios
  • Creativity assessment in higher education
  • Effective technologies to assess student learning
  • Faculty-focused environment for assessment
  • Instructional delivery formats
  • Method development for assessing a diversity goal
  • Multi-tier design assessment
  • Reporting race and ethnicity in international assessment
  • Technology of writing assessment and racial validity
Top

Preface

Many who have not learnt Reason, nevertheless live according to reason.
—Democritus of Abdera (5th-4th century BC) More than a few readers will likely discern a tautology in our title phrase “assessment technologies.” Assessment instruments and methods are all technologies when technology is understood in its etymological sense as techne, craft or technique (Partridge, 1958, p. 698). When colleagues sheepishly confess, “Well, I do assessment, but I don’t use any technology,” I ask them to explain what it is they do, and inevitably a use of technology emerges. Even in its earliest Latinate sense assessment was technical. In Roman culture, an assessor provided the sovereign with technical advice regarding the value of things to be taxed (Steen, 1999). The assessor helped value the amount to be taxed, and he did so technically. This process of valuation is of interest today, for assessment instruments reflect what we value in learning, deem worthy of measurement. Hence it would be wise to make explicit, in each case, the technology of assessment, to better grasp how it reflects our values and the degree to which the technology determines the results of our studies—all for the sake of clarifying our values and refining our use of technology to serve learning. The confession by a colleague reveals the given or tacit presence of technology. Is there anything more slippery to put in perspective than technology, except perhaps one’s native language which by no means operates independently of everything technical?

Understanding through Research

Technologies of assessment are put in perspective in diverse ways by the authors in the present handbook. Many chapters assembled here demonstrate the curricular and institutional integration of instruments for effective assessment of thinking, writing, learning, and organizational structure in higher education. Other chapters evince a heightened methodological awareness with regard to the relative appropriateness and usability of technologies, what they lack, overlook, but also the opportunities they create. The professional research contained here represents diverse stakeholders whose broad constituency from composition studies to psychology to mathematics, liberal arts colleges to major research institutions, is heartening for those who promote the growth of assessment culture. Creativity receives attention in these pages, as do multi-modal practices and instruments familiar to a new generation of students. The result is to make instruments and methods of assessment more relevant, appear less intrusive and alien, more “ours” and less “theirs.” This research should help allay the pervasive concern, warranted or otherwise, among many faculty members that their daily practice has been commandeered by assessment imperatives in counter-intuitive ways. We know that teachers regularly practice assessment without calling it such. They assess intuitively as it were, putting truth before method. They also make adjustments in their teaching based on their assessments that experts call “closing the loop.” They call it something else—changing the lesson plan; updating the syllabus; refining learning objectives. The purpose of the research in this book, then, is to help educators make their own assessment efforts more articulate and systematic by demonstrating applied research in cases and initiatives. Such applied research is beneficial for readers and practitioners. The historian Droysen’s expression, forschend zu verstehen, “understanding through (re)search,” is apt here (Grondin, 1994, p. 81). It means that in research one relearns and becomes more methodologically explicit about what one already knows and does, creating a new resource. A case in point is when an utterly devoted instructor received help to formalize his grading criteria, carried around in his head for over a decade, at a rubric design workshop where the latest design research is learned and applied. He marveled at the rubric table he produced, which, now objectively sharable, he brought to class and distributed. His intent was not to officially inform his students how he graded and ranked them, but to ask their opinion how they might refine the criteria after a thorough discussion of learning objectives. The result was that the teacher and student collaboratively produced a rubric of which the students had co-ownership, and which raised their awareness of what they were in class to learn. This example confirms that the best teachers use assessment and evaluation to help students learn (Bain, 2004, p. 151).

Yet some educators will remain methodologically vague by choice, their pedagogy romantic, convinced that the intrinsic multiplicity (ambiguity) of the learning experience is immeasurable but true, and any attempt to reach an unequivocal assessment outcome is always already false. Romantics, it is important to remind ourselves (for some of us are them, or have been, or will be on occasion), are comfortable with inexplicit evidence—that which seems to be the case. They suspect that when an assessment technology like Accuplacer informs them of what is in fact the case, method has supplanted truth and technology has become sovereign.

Truth and Method

This sort of technical usurpation is allegorized in Kafka’s story, “A Visit to a Mine” (1976), narrated by miners whose work is interrupted by visiting engineers and inspectors—an accreditation team like WASC if you will, performing an audit of an academic program. “The management has issued some instructions or other about boring new galleries, and so the engineers arrived to make the initial survey.” The miners stare, transfixed by the procession of engineers and their machine replete with measuring instruments, an “extremely expensive apparatus.” The inspection group represents a different professional class. “One, a lively man with black hair, has eyes that take in everything. A second with a notebook makes jottings as he goes, looks around him, compares, notes down.” It is the ninth engineer, however, who embodies instrumental reason in his relation with a perambulator full of instruments.

    Obviously he understands the apparatus thoroughly and seems to be really the man in charge of it. From time to time, without stopping the vehicle, he takes up a part of some instrument, peers through it, screws it open or shut, shakes and taps it, holds it to his ear and listens; and finally, while the man pushing the instruments usually stands still, he lays the small thing, which one can scarcely discern at a distance, back into its packing with great care. This engineer is a little domineering, but only in the service of his instruments. Ten paces ahead of the perambulator we have to give way to it at a wordless sign of his finger, even where there is no room for us to make way. (Kafka, 1976, p. 406)

This is not an “invitation to dialogue,” to invoke a phrase from the WASC Handbook of Accreditation (2001). The engineer is engrossed with his instruments and does not communicate with the miners except to soundlessly signal his obtrusive approach. To be sure, there can be a romance with measuring technology as much as with the immeasurable, but in the engineer’s case the relation exposes his domination. The intimate nature of his technical enthrallment, which separates him from others, is perceived through the respectfully objective gaze of the miners, imparting an air of absurdity to the otherwise sober behavior of the engineer. The almost fetish-like character of his relation to his tools is noteworthy, for they are his tools to measure their work space, tools with regard to which the miners can make no claim of ownership.

The reified behavior of the inspector is something before which many educators are most sensitive if not mistrustful. I have been using inspection, audit, and assessment interchangeably here to make a point about the relation between faculty members and assessment instruments and their representatives. I was the administrator when our Division of English and Applied Linguistics was subjected to an academic audit, so I have a concrete impression of the faculty response. The word “audit” is unpleasantly redolent of taxes; no one looks forward to an audit. We saw earlier that assessment is etymologically linked to tax collection. Some faculty members are wary not of mining machines or tax audits but assessment technologies such as Collegiate Learning Assessment (CLA) enthusiastically purchased by their universities. They have heard that Accuplacer and e-Write use an Artificial Intelligence program to produce their scores. For these educators the very idea of such systems and the uncritical bureaucracy that endorses them brings to mind the Skynet computer program in the “Terminator” films. This is writ large a narrative of the Verselbstandigung der Apparatur or autonomization of the apparatus (Adorno, 1998). The program takes over everything and destroys the world. As darkly comical as it sounds, this is not far from the sort of development feared by more than a few faculty members when confronted with assessment mandates and methods.

From Reactive to Active

“We’ve got to determine what happens, rather than adjust to what they do. It’s definitely an important game.” –University of Alabama football player

It would be pleasant to say that most educators have gotten involved with assessment because it is interesting and worthy of attention, sort of like a dissertation topic. And it is interesting and worthy of our attention, as the present book demonstrates. But it is rather the case that assessment sort of befalls many a teacher and administrator who in fact did not study it in graduate school. It comes down on many a head like climatic change, forcing a shift in lifestyle, or marches by in some official capacity that commands our attention, like the inspectors in Kafka’s “Visit to a Mine” who transfix the gazes of the miners with their somber procession. As a result, there is often little intellectual distance between educators and their assessment activity; they simply react to a mandate, grab a tool or template as one would an umbrella or pair of boots, mimic usage and do what needs to be done. They update a set of learning objectives without asking what the objective of learning is in the first place, a much tougher assignment. Pascal’s famous wager, Il faut parier, means, “You must do something; nonparticipation is not an option.” Well yes, but there is busy work and intellectual work, and most of us prefer the latter. The problem is that reactive behavior is not the most effective approach to a serious and complex state of affairs, or for that matter to an academic discipline. Nor does it bode well for the growth of a culture. It would be more active to take ownership of assessment and find in it an intellectually compelling set of problems and challenges for administrators, faculty members, and students to share. Several of our contributors argue this point, most notably Sean McKitrick, John Wittman, and Steve Culver. Motivation would thereby be more intrinsic, related to the matter at stake, and not mandated by the agenda of the newest political majority. Hitherto perfunctory tasks would point back to questions for reevaluating the conditions of learning and the possibilities of improvement in a time of scarce resources. Only in this way can we avoid the constricted cognition described by Collingwood (1978, p. 106): “He was trying to see only those elements in the situation which he already knew how to deal with, and was shutting his eyes to anything which might convince him that his ready-made rules were not an adequate guide to the conduct of life.” So it stands with assessment, whose technologies and assessments try to keep pace with yet inevitably lag behind the new epistemologies of student learning, webs of relations and ways of knowing so browsed, blogged, hypertexted, instant messaged, gamed, and hacked, as to be irreducible to any one identity, data set, or system. Sue Pieper and colleagues, and Deirdre Petttipiece and her group, are some of the authors in our book who look closely as these emerging technologies insofar as they impact or are integral to the assessment process.

The desire to turn from a reactive to active context for assessment is essentially the origin of the present book. In 2006, at the CCAS conference in Boston I heard many deans complain how difficult it is to get faculty members involved in assessment without rancor or resistance. One of the most heavily attended sessions was “Getting Faculty Excited about Assessment.” I realized something obvious, that for assessment to be done with enthusiasm by faculty members it would have to look like their other academic writing and have the same scholarly value, be integrated with their professional personae and rewarded on the scale of promotion. Hence it became my project to solicit substantial research papers in assessment for a book-length manuscript. Here would be my supporting evidence for saying to my colleagues, “Write up assessment projects not as internal memoranda, but with the flair, polish, and intellectual intensity you put into your scholarship.” This is apparently more difficult in practice than it sounds, for one rarely sees essays on assessment; it is rather the case that we have technical treatises and studies that do not, by any stretch, belong to the genre of the essay. A simple name change from “report” to “reflective essay,” recommended by WASC for purposes of documenting the institutional self-review process, does not automatically qualify the actual documentation being produced as an essay, at least as typically understood by its practitioners (Atkins, 2005). Nonetheless to even speak of an essay format for documenting educational assessment is a step in the right direction, and concretely confirms the sincerity of WASC’s stated intention “to maintain a posture of experimentation in the years ahead” (WASC, 2001).

Somewhat incredulous then, not unlike some of my colleagues, I wanted to see if assessment has depths. I knew that the place to look is in the research. Once I knew the answer, I could earnestly persuade colleagues to become engaged or involved with assessment, as is argued by several authors in the present book. Furthermore, few of my colleagues, however incredulous of assessment initiatives, would oppose the idea that as scholars we are still learning. Howard Gardner makes a good case for this in his description of what he calls “the disciplined mind,” which demonstrates “continuing mastery of one’s professional or employment roles, including the acquisition of additional disciplinary or interdisciplinary acumen” (Gardner, 2007). It would be beneficial then if assessment is viewed as an element of this new knowledge, which is very much conceived as an active learning process. In its active modality, alternative and authentic assessments become feasible (McMillan, 2001), as demonstrated in this handbook by Eubanks and Brunsden. This shift from passive to active awakens new kinds of thinking when one no longer resists but engages. One recalls here the protagonist in Kono Abe’s The Woman of the Dunes (1991) who at first seeks escape from a dwelling that is under constant threat of burial by sand dunes. He refuses to dig until he realizes that not digging sand is not a solution and the digging must be integrated harmoniously into his daily existence. At the point where his digging becomes active, his behavior evinces inventiveness; he builds a water collection device to make life more endurable.

Myth Interrupted

Happily, the chapters that appear here, arising from serious research, will disabuse the incredulous of their suspicions. As Henry James said in defense of his novels, “there are depths” to assessment, and, one has to say, a richness of conceptualization. I wanted to see if there are depths, not only shallows—not that shallows can’t be interesting and yield their own depths. But how many pre-and post tests based on ten items learned over fifteen weeks must we administer before we realize we need to swim more deeply to assess deep understanding, reasoning, and academic skills? But to pause for reflection, to gauge the relevance and appropriateness of a method is not as convenient as applying it and reaping data, however trivial. Let us remember what such methods do for us: as is said of myth, “it is a system of the elimination of arbitrariness” (Blumenberg, 1985, p. 43). Assessment, like myth, alleviates anxiety, gives us a semblance of measure and order where there seems disorder, simplicity where the situation is complex or nuanced.

In a recent essay in which Les Perelman seeks to put Collegiate Learning Assessment in perspective, he says the following about the Voluntary System of Accountability that was the bureaucratic precondition for the CLA: “This enterprise, however, of trying to measure and then compare the common benefits of a college education among widely differing educational institutions through two-hour standardized tests reaffirms the truth of H. L. Mencken’s observation that ‘For every complex problem there is an answer that is clear, simple, and wrong.’ These assessments are, at best, useless, and, at worst, subversive to the complex set of abilities that should inform undergraduate education. (Perelman, 2008, p. 1). It seems to me that what Perelman is doing is demythologizing assessment by warning us about blind faith in system-wide applications. Something like CLA is not a magic formula that is universally applicable. Bob Broad, who contributed the Foreword to this book, has done equally critical work with rubrics (2003). Another one of our contributors, Daniel Chambliss, moves in the same direction when he speaks of the individual as a neglected unit of analysis and when he examines the bureaucratic language that invokes assessment as a cure-all (2007). Likewise Stephen Ehrmann (1998) when he argues for a “unique uses” perspective to bring proportion to a paradigm that favors a uniform (average) impact perspective. Not solely enthralled with deriving data associated with what is average, he is encouraged by the fact that diverse outcomes are possible in the same course because of the different life paths students follow; a course is seen less as a set of fixed objectives than as a flexible learning opportunity. This is the case with delayed outcomes whose truth is untimely. Delayed outcomes can appear abysmal in present evidence, only becoming positive eight years hence, as when a student takes an advanced philosophy course as a sophomore, barely passes it, flops in exams and papers, yet is instilled with an obscure thirst for speculative inquiry that later matures into a passion and vocation in philosophy proper. A “C” student becomes a renowned teacher of philosophy. We have all heard of such cases.

Let us rephrase and say that the scholars cited above do not so much demythologize as interrupt myth in assessment, to borrow a phrase from Jean-Luc Nancy (1991), prevent it from becoming a reified practice by practicing critique as integral to its effective implementation. Other contributors to this handbook, such as Asao Inoue, Mya Poe, and John Wittman, are forging ahead early in their careers with critically aware research as they focus on the related issues of race, cultural diversity and technology in assessment.

The KISS of Death

All of the scholars mentioned here, and many handbook contributors as well, deserve our attention because they practice critique while fully involved with the matter of assessment. By so doing they “desimplify” assessment, to borrow a catchword of the heyday of deconstruction. Much of the resistance to assessment arises from the problem that Mencken, cited by Perelman, observes above. The oft repeated slogan “KISS” (Keep it Simple Stupid) has done more harm than good, the kiss of death, further convincing already skeptical colleagues that “assessment culture” is less a watchword than an oxymoron. Many educators suspect assessment to be intellectually vacuous not only because of the KISS slogan with which it is associated, but because they often hear that it is focused on outcomes. Those who have privileged process over product are understandably perturbed. But it is not assessment per se that is profound or not, since it can take many forms and styles, but rather certain outcomes--say, a simple pre-and post test assessment project. These simple outcomes don’t condemn assessment as an activity; they merely challenge practitioners to invent new and more encompassing assessment instruments, such as the critical thinking and/or writing rubrics described in detail by Hedva Lewittes and Teresa Flateby, to name just two of our contributors.

Chapter Organization

The chapters have been organized in three sections. In all but a few chapters the research is applied, that is, methodologically worked out in concrete frameworks of data collection and analysis, either with actual data sets or literature searches associated with a concrete matter at stake. A focus on assessment design and instrumental qualification in an applied framework characterizes the chapters in Section One. Although many of the chapters here are case studies, most of those which explicitly declare themselves as such, and which were not primarily concerned with instrument design so much as implementation, comprise Section Two. It is always nonsensical to separate applied from theoretical chapters for many reasons, but those with more overarching or ostensibly critical viewpoints, as well as those focused on workplace and assessment culture issues, were placed in Section 3.

Section I

Assessment Technologies and Instruments

Chapter I describes a model for evaluating complex organizations or systems. The design assessment model proposed is a response to current notions of assessment. Purdue University’s strategic planning process provides a context for describing how design assessment takes place in a higher education setting.

Chapter II describes how critical thinking is assessed using two critical thinking learning outcomes that were required for the State University of New York’s General Education program. As part of this process faculty developed a rubric that delineated the criteria for rating critical thinking on a four-point scale.

Chapter III reviews literature over the past ten years regarding technology tools that are being used in higher education to assess student learning. Three commonly used technology tools are examined: electronic portfolios, course management systems, and audience response systems.

Chapter IV explains the Consensual Assessment Technique, discusses how it has been used in research on creativity, and explores ways it might be employed in assessment in higher education. Unlike other measures of creativity, such as divergent-thinking tests, the Consensual Assessment Technique is not based on any particular theory of creativity, which means that its validity (which has been well established empirically) is not dependent upon the validity of any particular theory of creativity.

Chapter V discusses creativity assessment as a means for evaluating skills required in higher education. Creativity is assessed in the context of the creative person, process, product and press or environment. A historical view of creativity assessment is addressed with a substantive approach to understanding the construct of creativity, its measurement and evaluation, and the broader implications for use in higher education settings.

Chapter VI conceptualizes writing assessment as a technology, accounting for the ways in which assessment dialectically constructs and is constructed by its historical environment. Seeing writing assessment as a technology provides a full account of assessment as an environment of conflict and social reproduction, but most importantly, it accounts for racial formations existing around it and because of it.

Chapter VII explores ways qualitative and quantitative methods are complimentary, as opposed to competing concepts. The chapter reviews the literature on qualitative, quantitative, and mixed method designs, then highlights successful examples of mixed-method assessment at a mid-sized, private university, presented in general frameworks which can be used on any campus.

Chapter VIII describes the development of the Cognitive Level and Quality of Writing Assessment online system. Beginning with needs identified in a learning community program, the system evolved from a classroom analytic writing and thinking assessment rubric to an online system for classroom assessment and instructional purposes.

Section II

Assessment Applications and Initiatives

Chapter IX reports on the process used to develop an outcomes assessment initiative for the Multimedia Writing and Technical Communication Program at Arizona State University. Discussed herein are the development of outcomes, the mapping of outcomes to the curriculum, the use of electronic portfolios to assess student writing using Phase 2 scoring procedures, and how results from the first three semesters of implementation are being used to evaluate and improve the program’s curriculum.

Chapter X details the Composition program self-assessment conducted at the University of Louisville in anticipation of a review conducted by the Southern Association of Colleges and Schools (SACS) in 2006. The chapter includes a comprehensive analysis of internal assessment rationale, theoretical foundations, methodologies, and results.

Chapter XI uses a case study to report assessment of three instructional delivery formats: (a) online, (b) distance, satellite campuses, and (c) traditional on-ground format. Student competencies on learning outcomes in a basic interpersonal communication college course were analyzed in a core assessment project (a course assignment portfolio) using a department-approved assessment rubric. This assessment effort provides an example of how faculty can employ assessment as part of a continuous improvement cycle.

Chapter XII presents a case study of a college classroom strategy that enables assessment and some remediation of student problem solving skills in mathematics. It reports the outcomes of implementing this technique and its associated processes in several lower-level mathematics courses in the calculus sequence at Penn State- Altoona.

Chapter XIII describes Coker College’s subjective performance assessment program to rate student thinking and communication skills. It uses a discussion of the epistemology of assessment to motivate an emphasis on direct observation by experts as the basis for “authentic” assessment for complex learning outcomes. The purpose of this chapter is to provide a philosophical framework and practical methods that can help institutions assess liberal arts learning outcomes.

Chapter XIV describes an assessment project conducted during the 2006-07 and 2007-08 academic years at the University of Guam in Elementary Japanese I and II. This chapter presents the results of this assessment and a reflection on the assessment process itself, including its rationale, methodology and consequences in terms of on-going and future assessment in the Japanese Studies program and Foreign Language instruction.

Chapter XV narrates an outcomes assessment study completed in a basic composition course at a small urban open admissions community college. The course was a pilot course designed in response to marginally remedial performance on a standardized writing instrument and solidly exempted performance on the standardized reading instrument. It will be shown how data was used to guide decision making about curricular change on our campus.

Chapter XVI reports research on the impact of different forms of peer observation and feedback on preservice teachers’ skills in analyzing classroom teaching and their perceptions of their experience with peer assessment. According to this study, peer observation and feedback is beneficial to preservice teachers’ learning. However, to maximize its effectiveness, a culture of peer assessment should be established in teacher education programs.

Chapter XVII offers a detailed description of an innovative senior capstone program developed by the Department of Communication Studies at Samford University. The program incorporates student-driven communication workshops and electronic portfolios, and uses qualitative and quantitative methods to assess Transformational Learning Abilities or TLAs. The program is shown to serves as a catalyst for student engagement, faculty development, and departmental transformation.

Section III

Assessing Assessment

Chapter XVIII argues that the trend favoring assessment initiatives of a system-wide scope such as program review and collegiate learning tend to overlook the specific, highly concrete learning experiences of individual students in the liberal arts. These individual cases offer a rich source of data. The insights that can be derived from a rigorous analysis of such individual experience can tell educators much about learning outcomes, teaching quality, and curricular effectiveness.

Chapter XIX introduces a variety of multi-modal writing assessment instruments and methods, and discusses potential ways to determine the value of the student texts produced with them. New instruments associated with computerized scoring and distance technology make multi-modal writing assessments readily available, affordable, replicable, and transferable, but the value of the texts produced must be carefully ascertained.

Chapter XX introduces methods that can be used to engage faculty in the assessment process, working within a shared governance structure in institutions of higher education. It begins by identifying assumptions about including faculty in the assessment process, placing special emphasis on social capital and networking theories often used in communication and sociological research. The chapter t identifies six methods that might be used to engage faculty strategically in the assessment process, and then used three case studies to help explain these methods.

Chapter XXI suggests five elements to consider when developing an environment for assessment that successfully engages faculty: structure of assessment; qualifications of those in assessment; focus of assessment conversations; faculty development; and linkages with other areas within the institution.

Chapter XXII argues that as primary stakeholders in writing program assessment, students and instructors need to be included proactively in assessment research. However, little research has been done to determine how to accomplish this methodologically even though assessment affects pedagogical practices, student populations, and public opinion about what constitutes good writing. Instead of traditional quantitative, psychometric research, this chapter argues that assessment practitioners need to utilize local opportunities to discover native needs.

Chapter XXIII is guided by evidence that the study of racial-ethnic group differences on educational tests has yielded a substantial body of research internationally in the last decade. It maps current research about racial-ethnic minorities and educational testing in the English-language educational assessment literature. From an initial search yielding 420 articles, 78 were identified for further analysis of the frameworks for defining race-ethnicity and discourses found in the research literature.

Chapter XXIV describes a method developed to assess the outcome of a “cultural familiarity” general education goal. Challenges in defining, measuring, and providing summary information on variables of interest are discussed. The authors review the process of developing their own “oral examination” assessment method, explain their rationale for using this particular method, and suggest that locally-developed methods – this one and others – may have particular benefits that make them especially useful for program review and revision.

Concluding the book there is a glossary of assessment instruments, many of which are not fully invoked, studied, or referenced in the preceding chapters, yet which are relevant to studies and conversations about assessment.

Not the Final Word

Most of the contributors to the present work understand that assessment is not the final word in the total effort to raise the quality of learning in our institutions of higher education, no more than the final exam is the high point for even the best students in a well attended and dynamic seminar. For my own colleagues, and for many faculty members I have met elsewhere, knowledge of the goals and methods of assessment helps them survive the visits of the accrediting agencies. End of story. Many faculty members go back to their teaching and research until the next round of visits. Understandably, no few think it is a passing fad; they have seen many fads over the years. Only so many construe and value assessment itself as an area in which they will conduct and publish research. The latter are the ones who have contributed to the present volume, and we are grateful to them for doing so.

Top

Author(s)/Editor(s) Biography

Christopher S. Schreiner is Professor of English and Chair of the Division of English and Applied Linguistics at the University of Guam. Before teaching on Guam, he was a Professor of Literature at Fukuoka Women’s University in Japan, and a Professor of Integrated Arts and Sciences at Hiroshima University. He has coordinated assessment for the Division of English and Applied Linguistics in preparation for the WASC visit, and authored the summary assessment report for the grant-funded Project HATSA in the College of Liberal Arts and Social Sciences at the University of Guam. One of his recent articles, “Scanners and Readers: Digital Literacy and the Experience of Reading” appeared in the IGI Global book, Technology and Diversity in Higher Education (2007).
Top

Editorial Board

  • David Gugin, University of Guam, Guam
  • Asao B. Inoue, California State University – Fresno, USA
  • Robert Kellerman, University of Maine—Augusta, USA
  • Michel Pharand, Queen’s University, Canada
  • Mya Poe, Massachusetts Institute of Technology, USA
  • James Sellmann, University of Guam, Guam
  • Jason Vest, University of Guam, Guam
  • John Wittman, California State University, USA