Abstract
The development of the Cognitive Level and Quality of Writing Assessment online system is described in this chapter. Beginning with needs identified in a learning community program, the system evolved from a classroom analytic writing and thinking assessment rubric to an online system for classroom assessment and instructional purposes. Reflecting the assessment cycle, the system is equally appropriate for program or institutional assessment of student learning. Over a period of twelve years, assessment, survey, and research data guided changes and additions to the rubric and system. Preliminary data suggest using the system for peer review improves students’ writing and thinking.
TopInitial Development And Use Of Findings
Based upon commonly used writing handbooks, such as St. Martin’s Handbook, Harbrace College Handbook, and Scott Foresman Handbook for Writers, the initial writing rubric was a five point analytic scale with levels one, three, and five defined. Several factors influenced the decision to develop an analytic scale, in which individual components are judged separately, rather than a holistic scale in which a paper is assigned a single score. An analytic scale provides both a common set of criteria faculty could use to guide their evaluation of students’ writing and flexibility to select from the criteria. Furthermore, White (1998), a composition theorist and early proponent of holistic scoring, cautioned that while judgments can be made holistically (choosing one level as a representation of all criteria), faculty must teach analytically. While giving a grade represents a holistic perception, feedback about the specific criteria used to make the holistic judgment provides students with the information necessary to improve. Also, through conversations with the learning community faculty, including composition instructors, they revealed that students often are better with some components of writing than others.
Although based upon writing handbooks, the categories also were affected by assessment results as the rubric was initially used. For example, the components that comprised the original category of “Organization and Development” were divided into two separate categories: one pertaining to structure and the other reflecting reasoning and evidence supplied. We observed that while many beginning students’ essays had an appealing structure (five paragraph essays which students learned to produce for standardized testing), the quality of content and the reasoning demonstrated in the essays were often weak. These results were used to refine the rubric and more clearly reflect writing traits we hoped to foster.
Key Terms in this Chapter
Peer Review: An approach used to provide peers (students) with feedback to improve their performance.
Holistic Scoring: A single score is used to indicate the overall achievement level of a performance (e.g. writing assignments).
Formative Assessment: The collection of data to determine if progress is being made toward learning outcomes. If weaknesses are identified, results are used to make changes in the curriculum or in courses.
Summative Assessment: The collection of data to determine if leaning outcomes are achieved.
CLAQWA: The Cognitive Level and Quality of Writing Assessment is a two-part scale developed to analytically evaluate writing and thinking and to judge the cognitive level reached in student writing.
Assessment: The use of valid and reliable data to determine achievement of measurable student learning outcomes or to improve achievement of student learning outcomes. This is a process that typically seen as a mechanism to improve student learning through curricular or instructional changes. .
Analytic Scoring: Multiple scores are used to indicate performance on specific criteria used to judge a performance.