Abstract
In the educational field, evaluation is a very complex activity due to the intrinsically multidimensional nature of the processes to be evaluated. Several variables must be taken into consideration, and they interact and influence one another: the object and the goal of the evaluation determines the criteria, the methods, and the data to be used for the evaluation. In this chapter, we will focus on evaluation in computer-supported collaborative learning (CSCL). In this field, a primary role is played by the monitoring process, which allows us to gather important information about the learning process while it takes place. Indeed, monitoring serves three purposes: it provides real-time data about group dynamics so that they can be used by tutors to facilitate learning and stimulate collaboration among trainees; it provides designers and evaluators with data about learning system usage that are needed to evaluate its effectiveness; finally, it supplies information about the learning process and its outcomes, thereby informing assessment. Hence, monitoring can be seen as a sort of common denominator between the methods used to foster collaborative learning and those that allow the gathering of data for the two types of evaluation.
TopBackground
Evaluation was defined by Hadji (1995) as the formulation of a value judgment on a given reality that is the object of the evaluation. More precisely, evaluation is the consequence of ascribing meaning to facts, data, and information associated with that reality. The judgment is generally based on a definition of suitable criteria and on the identification of the quality models used as a reference.
In education, a distinction is usually made on the basis of the object of the judgment: when the focus is on changes in individual competences, we talk about learning assessment, while if the object of judgment is the learning system, the teaching methods and the learning resources, the term evaluation is generally used (Ellington, Percival, & Race, 1993). When looking at the evaluation activity from the point of view of its goals, another distinction is usually made: formative evaluation aims to obtain both general and detailed information in order to improve the object of evaluation, while summative evaluation aims to formulate a comprehensive judgment on the object to be evaluated, often with certifying purposes. In accordance with these different goals, formative evaluation is usually carried out in itinere, that is, during the learning process, so that any problems are identified and dealt with as early as possible. Summative evaluation, on the other hand, is usually carried out at the end of the learning process, or at particular stages where a global judgment of the results is needed. In spite of this connotation of summative evaluation, it may also be based on data that has been collected during the learning process, not just at the end.
The above-mentioned terms and concepts have been used for over 50 years in both face-to-face and distance education (Bloom, Hastings, & Madaus, 1971; Scriven, 1967). The meanings of the terms have remained basically unvaried while the methods used to carry out evaluation and assessment continue to undergo major changes due to the evolution of learning theories, methods, and techniques. According to constructivist learning theories, for example, learners should be actively and increasingly involved in formative evaluation practices and consequent decisions in order to promote the metacognitive and self-regulation skills needed for effective personal and professional development. It is for this reason that evaluation strategies are being increasingly integrated into the learning process, and self-evaluation and peer evaluation are frequently practiced along with hetero-evaluation (Ranieri, 2005).
Key Terms in this Chapter
Competence-Based Education: In the 1990s, the definition of learning outcomes in instructional design saw a radical shift of attention from the specification of behavioral objectives to the formulation of the aims in terms of competences to be developed. Competences include skills and knowledge needed to adequately perform identified jobs and tasks, but they also comprise higher level abilities and social attitudes that are considered important in a given context or profession. These aspects cannot easily be measured through traditional examinations, let alone tests. However, evidence of competences can be collected from self-evaluation, peer evaluation, and observation by others.
Summative Assessment: It aims to produce a comprehensive judgment on the learning outcomes of an individual student. Typically, summative assessment is quantitative and uses numeric scores or letter grades.
Formative Assessment: Its purpose is to improve the quality of student learning rather than judging or grading students. For this reason, it is generally carried out during (or even at its beginning) of the learning process. Formative assessment provides diagnostic information for teachers and students to make the necessary adjustments to the teaching and learning process, including the use of alternative instructional approaches or the provision of more opportunities for practice.
Three Generations of Distance Education: In the literature, three generations of distance education have been identified (Garrison, 1985; Nipper, 1989). The first generation mostly involved individual learning and entailed the delivery of printed material to the students by mail. Communication between learner and tutor was based on a one-to-one model, and it entailed a very slow, sparse exchange of documents. The second generation was based on the use of multimedia teaching materials including videos, radio programs, open TV, cable TV, and educational software. These learning resources could be delivered via mail or through specialized telecommunication channels. In this case, tutor–student communication was based on a one-to-one or one-to-many model. The third generation, also called online education, takes full advantage of computer mediated communication systems and is heavily based on virtual communities and collaborative learning strategies. In this case, communication is based on a many-to-many model.
Summative Evaluation: Provides information on the quality of the learning system with particular reference to its ability/suitability to achieve its objectives. It usually produces a global judgment of the system, considering all of its components and taking a very general point of view. However, it may also focus on one particular aspect of the system. For example, summative evaluation of the learning outcomes or summative evaluation of students’ acceptance.
Formative Evaluation: “Typically conducted during the development or improvement of a program or product (or person, and so on), it is conducted, often more than once, for in-house staff of the program with the intent to improve. The reports normally remain in-house; but serious formative evaluation may be done by an internal or an external evaluator or preferably, a combination; of course, many program staff are, in an informal sense, constantly doing formative evaluation” (Scriven, 1991).
Bloom’s Taxonomy of Educational Objectives: The American psychologist B.S. Bloom and his co-workers compiled a set of taxonomies of learning objectives, classified into three broad “domains”: the cognitive, affective, and psychomotor domains. The most famous is the taxonomy proposed for the cognitive domain where six levels are identified, ranging from the simple/concrete to the complex/abstract. They are knowledge, comprehension, application, analysis, synthesis, and evaluation (Bloom,1956). According to Bloom, the six subdivisions should not be regarded as rigidly distinct or mutually exclusive, but rather as a continuum spanning the whole domain (Ellington et al., 1993).