Advances in Automated Scoring of Writing for Performance Assessment

Writing For Performance Assessments

Writing is the ability to create meaning through symbols (e.g., Kellogg, 1999) and to communicate that meaning to others. Communicating information through writing is considered one of the key 21^st Century skills and has been incorporated as a critical component in many national standards (e.g., Ananiadou & Claro, 2009; OECD, 2013). For example, the U.S. Common Core State Standards require students to develop more rigorous writing skills with a stronger emphasis on the ability to synthesize and summarize informational text, formulate arguments, as well as respond appropriately to source documents. This puts greater emphasis on writing for a purpose, with students linking ideas to texts and argumentation and writing across the curriculum. With the advent of information technologies, writing also plays a more central role in everyday communication. Information can be readily published on the web and communication via writing has become more ubiquitous through email, chat, and other collaborative tools. Thus, within an information economy, writing is not just an academic pursuit, it is the primary conduit for effective transmission of the vital information. As such, writing is a crucial skill across almost all career paths.

Educational practice has also recognized that writing is not just a matter of teaching compositional writing skills. Training of writing typically occurs both through more domain general means, such as in language arts classes and as integrated components within particular curricula to train domain specific writing skills. Over the past few decades, greater emphasis has been placed in training writing across the curriculum (e.g., McCleod, 1992), by emphasizing that students are not just learning to write, but by embedding writing in curricula, they are writing to learn. Indeed, over half of U.S. higher education institutions have formal programs where writing is integrated across the curriculum (Thaiss & Porter, 2010). Thus, writing activities must occur within natural performance tasks in order to promote the abilities that will be needed and used in and out of the classroom.

Learning to write well is a multi-faceted ability. It requires language, organization and expression skills as well as domain knowledge, and it develops slowly over a person’s educational career. It is a well-known adage that in order to become a good writer, one needs to do a lot of writing. However, while time on task is typically a strong predictor of performance gains in reading and writing, receiving timely feedback is critical (e.g., Black & William, 1998; Hattie & Timperley, 2007; Shute, 2008). Meta-analyses of studies of formative writing (e.g., Graham, Harris, & Hebert, 2011; Graham & Hebert, 2010; Graham & Perin, 2007) have shown that instructional practices can have strong effects on helping students with learning to write. For example, supporting students with feedback had an effect size of 0.77, and providing students with instruction in strategies for planning, revising and editing their compositions had an effect size of 0.82 (Graham & Perin, 2007), indicating that each instructional practice can be highly valuable. Other findings from these studies indicate that having teachers actively monitor a student’s writing progress and teaching students to monitor their own writing also significantly improve student performance. Thus, it is not good enough to just tell students to write more or write more often. The writing must be monitored and the students need to receive timely feedback and instruction for it to be most effective.