Comparison of Three Kinds of Alternative Essay-Rating Methods to the ESL Composition Profile

Comparison of Three Kinds of Alternative Essay-Rating Methods to the ESL Composition Profile

Shin'ichiro Ishikawa (Kobe University, Hyogo, Japan)
DOI: 10.4018/IJCALLT.2018100103

Abstract

Although the ESL Composition Profile (CP) has been widely used by scholars as a well-balanced rubric for learner essay evaluation, it is not necessarily easy for L2 teachers to rate students' essays based on the five criteria of the CP. This indicates the necessity to explore reliable alternatives to the CP. This article, therefore, compares three kinds of alternative approaches: (1) using the simplified version of the CP, (2) using edited essays, and (3) using model essays, paying attention to the correlation between each of the possible alternatives and the original CP. Our learner-corpus-based analysis has shown that simplifying the CP and paying attention to the organization of learner essays appears to be the most effective method.
Article Preview
Top

Background

Concerning learners’ L2 essay evaluation, many approaches have been proposed to date. Recent studies emphasize the merits of a portfolio evaluation, which enables teachers to see the whole of learners’ writing process and also gives learners a chance to get more actively involved in assessment and learning process (Lee, 2017). However, many of the EFL teachers still rely on rubric-based direct rating of learner essays, which is relatively less complicated and therefore easier to be incorporated into their teaching routines. In this section, we will have a quick look at the CP and other rubrics.

Holistic vs. Analytical Evaluation

Many scoring guides define different levels of performance or product with specific indicators and descriptors, which are usually called rubrics. Rubrics can be classified into two types: a holistic rubric and an analytical rubric, each of which has its own merits and demerits.

A holistic rubric, which is “a single, overall rating for an entire performance or product” (Arter, 2010), has its own strength in practicality, especially in large-scale testing contexts (East, 2009). Raters, however, have a tendency to focus on only one or two superficial traits (Sakyi, 2001), such as essay length, handwriting, lexical diversity / richness, or errors in word choice or spelling (Charney, 1984; Grobe, 1981; Engber, 1995). Meanwhile, an analytical rubric, which is based on an evaluation of “several different, important dimensions” of a performance or a product (Arter, 2010), can be more objective and reliable. The limitation is that it is often too time-consuming and, consequently, difficult to apply to large-scale testing or everyday assessment in L2 classes.

When comparing a holistic rubric and an analytical one, much of the literature has regarded the latter as being better balanced and, therefore, a relatively more reliable scoring guide for evaluating learner essays (Jacobs et al., 1981; Bacha, 2001).

Complete Article List

Search this Journal:
Reset
Open Access Articles: Forthcoming
Volume 10: 4 Issues (2020): 1 Released, 3 Forthcoming
Volume 9: 4 Issues (2019)
Volume 8: 4 Issues (2018)
Volume 7: 4 Issues (2017)
Volume 6: 4 Issues (2016)
Volume 5: 4 Issues (2015)
Volume 4: 4 Issues (2014)
Volume 3: 4 Issues (2013)
Volume 2: 4 Issues (2012)
Volume 1: 4 Issues (2011)
View Complete Journal Contents Listing