Abstract
Computer-Aided Assessment (CAA) is becoming an increasingly popular method for assessing students in their mathematics courses in higher education. This article examines six lecturers' practices of using CAA on their mathematics courses. The interviews with these lecturers revealed that the CAA system did provide many of the benefits that were promised; however, there were some important aims not satisfied by the system, which limited the scope of its effectiveness. Using a model for effective assessment, which draws upon ideas from the assessment literature and cultural-historical activity theory, the lecturer interviews give an insight into what stops this assessment tool from remaining effective. This study shows that the CAA system was reasonably effective to an extent, and lecturers had achieved a relatively stable practice that they were satisfied to maintain; however, there were shortcomings with the existing system that limited the scope of its effectiveness, which led to diverse practices and a desire to change system.
TopIntroduction
Computer-aided assessment (CAA) is a means of offering assessment electronically (Bull & McKenna, 2003), offering immediate and automated feedback. There are several CAA systems in use across UK higher education institutions (HEIs) that assess students in mathematics, such as HELM, QM Perception and STACK (Green, Harrison, Palipana, Pidcock & Ward, 2004, Greenhow and Gill, 2005, Sangwin, 2007). There have been calls in the literature for a systematic and rigorous review of its use (McKenna & Bull, 2000, Sangwin, 2003).
Lecturers assess students for many reasons and Samuelowicz and Bain (2002) encountered many in their study, including: to encourage students to study, to differentiate between students of different abilities, and to guide students towards a professional standard. There are also institutional demands, constraints and rules that shape lecturers’ aims and choices for assessment (Hunt, 2006).
In the United Kingdom, there are changing expectations in a developing higher education landscape. Students demand more time from their lecturers and are less willing to work independently (Rolfe, 2002). The number of students attending universities has increased substantially since the 1960s (Blanden & Machin, 2004) and cohort sizes are further increasing.
Amid these changes and competing demands for time, lecturers face increasing pressure over assessment. Traditional forms of assessment used in mathematics learning that are marked by hand and offer personalized feedback take substantial time (Biscomb, Devonport & Lane, 2008). CAA offers the facility to provide feedback to students quickly and are relatively straightforward to administer. It permits lecturers to spend more time on their other commitments, such as their research interests.
However, while lecturers at HEIs have adopted CAA for use with engineering and mathematics students, it does not necessarily follow that the CAA systems they use satisfy all their aims for assessment.
This research study focuses on lecturers’ use of a CAA system with first year students at one particular HEI: with one bank of questions for mathematics students and one for engineering students. The questions were in multiple choice and numerical input formats; the feedback comprised either the full solution to the question or the solution to a generalized form of the question. The lecturers were able to offer practice tests to students prior to a summative test.
Among the lecturers of these first year modules, not all had adopted CAA and practices were diverse (Broughton, Robinson & Hernandez-Martinez, 2013). The system had been in place for ten years and had not been evaluated formally in that time. The dearth of studies evaluating the effect CAA systems have had on learning and teaching in the literature suggested this situation was common. Therefore, there remained a need to establish whether these CAA systems were proving to be useful for lecturers and students. The authors have reported on how students have used this CAA system and where it has been effective (Broughton, Hernandez-Martinez & Robinson, 2012); it is the purpose of this chapter to report on the experiences of six lecturers that have been using the CAA system with their mathematics and engineering students.
The research questions for this study are:
- •
Why did these lecturers use the CAA system?
- •
How did they arrive at their current practice?
- •
For what aims was the CAA system effective?
Key Terms in this Chapter
Subject: The subject of the activity is the primary actor in an activity system. It is the subject that is striving to achieve the object of the activity. AU24: Reference appears to be out of alphabetical order. Please check
Formative Assessment: Although there continue to be discussions and disagreements in the literature, formative assessment is usually taken to mean a zero-stakes assessment in which students receive feedback.
Object: The object of the activity in CHAT is the overarching aim of an activity, which comes from a subject’s need.
Computer-Aided Assessment: A test that is taken via a computer terminal, usually conducted via the Internet.
Summative Assessment: Summative assessments are tests taken by a student to evaluate performance, knowledge or understanding. They are usually taken at the end of a unit of learning.
Contradiction: In CHAT, a contradiction occurs within or between nodes and activity systems. They are conflicts that destabilize an activity system and inspire changes to resolve the conflict.
Tool: Tools are socially created artefacts from an activity that are used in other activities. For example, the CAA system is a tool that has been created by lecturers and developers, which is now used by lecturers to assess their students.
Activity: In cultural-historical activity theory (CHAT), an activity is a unit of analysis, in which an actor (the subject) undertakes actions directed towards achieving the object.