Abstract
The assessment of competences requires an approach where knowledge, abilities, and attitudes are integrated, naturally implying the resource to a variety of assessment strategies. Within this context, we have seen the emergence of what has been called by several authors, the Assessment Culture. Furthermore, Higher Education e-learning environments have also promoted the use of new e-assessment strategies. Therefore, it is important to reconsider the concept of quality assessment in Higher Education online contexts, and particularly how to develop it in the present learning landscapes. In this chapter, the authors present a new conceptual framework for digital assessment in Higher Education supported by four dimensions—authenticity, consistency, transparency, and practicability—each composed by a set of criteria, aimed at promoting the quality of the assessment strategies being used. This framework was developed based on the expansion of the concept of validity supported by edumetric qualities.
TopIntroduction
Nowadays, it is demanded from Institutions of Higher Education (IHE) to integrate the complex needs of a labor market characterized by a variety of diverse contexts and contribute to the development of active and autonomous citizens and professionals. It is expected that the students will develop complex problem solving competences, strategies to cope with frequent changes and innovations, and be able to reflect about their own learning (metacognition) and so engage in a variety of contexts and situations. Moreover, in this setting, IHE are being confronted with the fast growing use Information and Communication Technologies (ICT) and the exponential development of distance education and elearning.
These challenges require not only a shift in the perceived goals of Higher Education, but also in the selection of the methodologies used, as well as a drastic change in the assessment strategies to implement in order to foster a competence-based curriculum.
In this new context, new roles are demanded from both teachers and students, where students are empowered to actively develop their knowledge and competences, and teachers are responsible for creating learning environments that nurture deep learning, anchored in real contexts. Taking into account these new contexts, “assessment will have to go beyond measuring the reproduction of knowledge” (Dierick & Dochy, 2001, p. 301) requiring the development of a new kind of assessment design. This new approach towards assessment has been labeled as an “assessment culture” (as opposed to a “testing culture”) and is characterized essentially by:
- •
Emphasis in the integration of assessment and teaching. (Birenbaum, 1996)
- •
Student engagement in the development of his own assessment in a continuous dialogue with the instructor. (McConnell, 2006)
- •
Assessment of both the product and process of learning. (Linn, Baker & Dunbar, 1991)
- •
Assessment in a variety of non-standardized formats associated with experienced instructional practices. (Dierick & Dochy, 2001)
- •
Using assessment tasks similar to real life contexts. (Resnick, 1987; Herrington & Herrington, 1998)
- •
Complex challenges and an emphasis on research. (Gulikers, Bastiaens, & Kirchner, 2004)
- •
Supporting student reflection about their learning. (McConnell, 2006)
- •
Valuing qualitative feedback over a bland quantitative classification. (Birenbaum, 1996)
Dierick and Dochy (2001) discussed this new assessment culture through the analysis of several assessment strategies (portfolios, OverAll tests, and assessments in which students have a role) supported by an edumetric perspective of assessment. According to these authors, the current societal and technological contexts require education to change, stressing that “the explicit objective is to interweave assessment and instruction in order to improve education” (p. 321).
In this chapter, we start by discussing the concepts of competence, assessment culture, and edumetrics, as well as the changes introduced by technology into assessment. Moving forward we clarify the concept of e-assessment, introduce the new concept of alternative digital assessment and present a new conceptual framework for e-assessment in Higher Education. This conceptual framework introduces a set of four quality dimensions and fifteen criteria for the development and use of e-assessment in Higher Education virtual environments supported by the discussed edumetric qualities.
Key Terms in this Chapter
Consistency: This dimension takes into account the traditional demands for validity and reliability, associated with psychometric indicators. It recognizes that the assessment of competences requires the implication of a variety of assessment methods, in diverse contexts, by different assessors, as well as the adequacy of the employed strategies. This dimension is comprised by four criteria: instruction-assessment alignment, multiplicity of indicators, relevant criteria, competences-assessment alignment.
Alternative Digital Assessment: Is used as a strategy that refers to all technology-enabled assessment tasks where the design, performance, and feedback must be mediated by technologies. The design includes the definition of competences to be assessed, as well as instructions about the task to be completed, including the use of electronic devices for its accomplishment. In this context the student performance must be completed using digital technologies and requiring the students to participate in technologically mediated activities (such as e-portfolios, digital reports, digital forums, etc.) either on a personal computer, on the webWeb, or using the webWeb as a resource, or on any other electronic device (tablets, cell phones, digital cameras, etc.). The feedback component includes not only the professors comments but also the student´s self and peer assessment.
Practicability: Is related with the feasibility of the assessment strategy. This dimension is particularly important when designing a competence assessment program given the complexity of its design. It implies an effective management in terms of time and cost/efficiency balance for both assessors and organizations. Furthermore, it should assure that the students/learners consider the assessment tasks as being doable, relevantrelevant, and contributing to their learning. It is composed of three criteria: costs, efficiencyefficiency, and sustainability.
Authenticity: Is related to the degree of similarity between the competences being assessed and the ones required in real/professional life. In this dimension are included four reference criteria contributing to the degree of assessment authenticity: similarity, complexity, adequacyadequacy, and significance.
Assessment Design: Includes the complete assessment program developed by the instructor, either for a program or course, and should comprise the definition of the strategies, instruments, assessment criteria and scoring sheets to be used, as well as the definition of every participants’ role (e.g.: self-assessment, peer assessment, co-assessment) and complete schedule.
Transparency: This dimension intends to make the entire competence assessment program visible and comprehensible for all participants. For this reason, it is important that the students/learners are able to understand the fairness of their assessment, as precisely as their instructors, requiring them to have a complete knowledge of all assessment criteria and their relative weights. For this dimensiondimension, four criteria were considered: democratization, engagement, visibilityvisibility, and impact.
Competence: Is defined as the capacity to respond to individual, or societal, demands in order to perform an activity or complete a given task. They are developed through acting and interacting in both formal and informal educational or professional contexts, and require going beyond the mere reproduction of acquired knowledge. At its highest level, this conceptualization of competence implies to choose and adapt from within the acquired processes those ones necessary to solve an unknown complex task or problem.
Competence-Based Assessment: Involves the use of multiple forms of authentic assessment, that are carried out without time pressure and using the tools or other helping materials that are also used in real life. Competence assessment requires, from the student, active participation and production.
Assessment Culture: Arose from the growing criticism of traditional testing methods relating to the unrealistic nature of the tests, the loss of faith in them as valid measures of learning, and an over-reliance on tests as the ultimate goal of the instruction process. The assessment culture is based on social constructivism, in which learning is thought of as the active construction of schemes in order to understand the material. The student is recognized as an active participant, who shares responsibility for the learning process, practices self-assessmentself-assessment, and reflection, and collaborates with the teacher and other students. Multiple forms of assessment of student learning are used, which are generally less standardized than the formats used in the testing culture.