Abstract
Automated writing evaluation (AWE) has become increasingly popular in the assessment of writing. The study in this chapter examines the extent to which EFL learners' overall narrative writing performance improves through the AWE feedback system (i.e., Pigai). Eighteen university participants were required to write one paragraph narratives on the web-based Pigai system every week over the course of a month. Findings show a significant improvement in overall scores between the first and last writing task. The analysis of lexical profile further shows a significant improvement in lexical richness, clause density, and paragraph length between the first and last narrative task. The study also reported that the primary error types that occurred in learner narrative writing were lexical, mechanical, and syntactic errors. Results of post-writing interviews also showed a positive attitude towards Pigai. Finally, a positive correlation was observed between automated Pigai scores and human rating scores, supporting the reliability of the AWE system.
TopIntroduction
The rapid development of the internet and intelligent technology over the past twenty years has revolutionized the learning and teaching of English as a foreign language (EFL). During this time, computer-assisted language learning tools for automated writing evaluation (AWE) have become increasingly popular in EFL writing instruction across different educational levels (Huang & Renandya, 2018; Link & Hegelheimer, 2015; Zhang & Hyland, 2017). To improve English language writing, established commercialized programs such as Criterion (e.g., Attali & Burstein, 2006), MY Access (e.g., Vantage Learning, 2007), Project Essay Grade (e.g., Page, 2003) and WriteToLearn (e.g., Liu & Kunnan, 2016) have been used by large numbers of students and teachers internationally to improve writing skills for both native English speakers and EFL learners. These programs have proven useful, allowing English language learners to independently and immediately revise and improve their written work (Chen & Cheng, 2008). AWE tools were also designed to benefit teachers by reducing heavy grading loads by providing immediate, automatically-generated quantitative assessments and qualitative diagnostic feedback on large numbers of written works (Aryadoust & Riazi, 2017; Bai & Hu, 2017; Chen & Cheng, 2008). Recent AWE tools such as LightSide (Mayfield & Rosé, 2013), Writing Pal (Roscoe et al., 2017), CorrectEnglish (Wang et al., 2013) and Pigai (Li, Lu, & Li, 2015; Lu & Li, 2016) have been developed to be innovative learning platforms that provide individualized feedback, particularly for EFL contexts.
Key Terms in this Chapter
Recommended Expression: The reference of extended learning which provides some better or more formal words similar to the word that users use in writing.
Lexical Sophistication: Words are categorized by Pigai for different levels of (e.g., frequently used word, academic word, and informal word). Lexical sophistication refers to the number of words used are from the academic word list.
Extended Differentiation: The reference of extended learning which tells users how many times the word they use occurs in the corpus or which helps users make the differentiation between similar words.
Automated Writing Evaluation: The technology to evaluate essays and provide scores and feedback for users.
Extended Learning: Different kinds of references for users to learn and revise their essays (e.g., recommended expression, synonyms learning, learning hints, and extended differentiation).
Lexical Richness: Type-token ratio, that is, the number of unique words divided by the number of total words.
Pigai: A web-based system which users can submit their writings in and receive an overall score and sentence-by-sentence comments and feedback.
Lexical profile: The analytical statistics of writings based on corpora and cloud computing in Pigai and categorized into various aspects (e.g. average word length, lexical richness, lexical sophistication, average sentence length, clause density, and paragraph length).