Good assessment assures attainment and drives learning. In vocational and practical programmes, the important learning outcomes are non-cognitive skills and attitudes - for example, dexterity, situational awareness, professionalism, compassion, or resilience. Unfortunately, these domains are much more difficult to assess. There are three main reasons. First, the constructs themselves are tacit - making them difficult to define. Second, performance is highly variable and situation-specific. Third, significant assessor judgement is required to differentiate between good and poor performance, and this brings subjectivity. The chapter reviews seven existing strategies for addressing these problems: delineating the constructs, using cognitive assessments as a proxy, making the subjective objective, sampling across performances and opinions, using outcome measures as a proxy, using meta-cognition as a proxy, and abandoning the existing measurement paradigm. Given the limitations of these strategies, the author finishes by offering three promising ways forward.
TopIntroduction
Some rise by sin, and some by virtue fall. (Measure for measure, Shakespeare & Lever, 1997)
Good assessment can serve many purposes that might broadly be defined as summative (for example assessing for licensure, regulation, selection or ranking), formative (through transparent performance targets, developmental feedback, and self-monitoring as part of self-regulation), and evaluative (faculty or institutional quality assurance). It is precisely because assessment is so purposeful and powerful that this book matters.
Definitions of assessment vary widely in emphasis. In every case however, assessment claims to be something more than unstructured consideration or appraisal; it claims to make measurements with reasonable measurement characteristics that can be trusted. When assessment fails to tell the truth, there are negative consequences. The educational assessment community has developed a technical language to describe the many ways in which assessments can fail. Assessment results can be unreliable – prone to error so that they yield different outcomes when repeated. They can be invalid – in that they estimate or reflect some characteristic that is not what was intended. They may have an adverse educational impact – driving learning away from its intended outcomes. It is also possible for assessment to have favourable measurement characteristics, but to be unaffordable, unacceptable or unfeasible to implement. Figure 1 illustrates the measurement concepts of reliability and validity.
Figure 1.
A visual illustration of reliability and validity (Gareis & Grant, 2008)
Bloom, and many subsequent educators, have broadly categorised learning as cognitive (knowledge), psychomotor (skills), and affective (attitudes) (Bloom, 1971). Assessing the cognitive domains has been the primary business of higher education institutions for most of their history. However, the competency movement, and the expanding scope of higher education programmes, bring the new and particular challenge of teaching and assessing the non-cognitive domains (skills and attitudes). The non-cognitive domains are central to good performance in many vocational courses (e.g. medicine, dentistry, law) and many practical courses (e.g. the building and mechanical trades). A failure to design good assessments to cover these domains is as serious as a failure to design domain-appropriate teaching and learning activities. It will render the assessment programmes invalid as a reflection of the intended learning, and it will undermine the impact of the taught curriculum.
Furthermore, authentic practice in each of the vocational and practical fields above doesn’t draw on the cognitive, psychomotor and affective domains one at a time; it draws on them simultaneously and in an integrated way. Epstein, for example, defines competence in the profession of medicine as:
…the habitual and judicious use of communication, knowledge, technical skills, clinical reasoning, emotions, values, and reflection in daily practice for the benefit of the individuals and communities being served. (Epstein & Hundert, 2002)
It is therefore likely that authentic assessment will need to reflect this integration; it may not be sufficient to design psychomotor and affective measures to apply in parallel with knowledge measures.
This chapter will address the particular challenges of assessing the non-cognitive domains. Most of the examples and evidence will be drawn from the field of medicine because this is the literature with which the author is most familiar. However, the principles apply similarly across all the disciplines.
TopThe Challenges Of Assessing The Non-Cognitive Domains
At their simplest, the challenges of assessing the non-cognitive domains are threefold. First, the objects of assessment are tacit; second, performance is highly contextual and variable; and third assessment relies on subjective judgement. Each of these threatens reliability and validity – and thus the trustworthiness of assessment. Each problem is expanded below: