A Model for Monitoring and Evaluating CSCL

Introduction To The Cscl Field

This paper addresses the issue of monitoring and evaluating collaborative learning processes that take advantage of computer communication technology and are typically carried out at a distance. In particular, we focus on CSCL (Computer Supported Collaborative Learning) approaches based on social constructivism (Dillenbourg, 1999; Kanuka & Anderson, 1999; Vanderbilt, 1991; Scardamalia & Bereiter, 1994), where knowledge building takes place through social negotiation. In this approach, discussion with peers and teachers is the main learning method because it encourages critical thinking and, hence, understanding. Interaction is usually based on written asynchronous message exchanges in specific environments known as Computer Mediated Communication (CMC) systems, that allow the organization of communication along separate, topic-oriented discussion “forums” or “conferences”, and provide the valuable benefit of recording most significant events for subsequent revisitation and analysis. Even in those cases where interaction occurs synchronously (e.g., through chats), the underlying communication system usually allows one to record a transcript that participants in the learning community can later retrieve.

Social constructivism promotes a shift from the traditional teacher-centred approach to a setting where students are the main actors on the learning stage and take direct responsibility for their knowledge building processes. In this kind of setting the terms “teacher / instructor”, that usually indicate the actors in charge of designing, delivering and evaluating the learning event, recede in favour of two different terms: “designer”, to indicate the person in charge of setting up the course, and ”tutor”, to identify the person in charge of orchestrating and moderating online discussions among the members of the learning community (students, experts, etc.). Accordingly, tutors are not experts who fulfil their role by dispensing domain knowledge, but rather they are in charge of setting up and maintaining collaborative processes. Of course, while orchestrating and moderating online interactions, each tutor will deploy their own personal tutoring style, but to ensure the effectiveness of their action, tutors must always have a clear picture of what is happening, who is participating and how, in order to anticipate and deal with problems, to identify different learning styles, and to help students to exploit their own individual abilities. When the learning community is large and participation is high, gathering and systematically organising these kinds of information can be quite a labour intensive task.

Currently, CMC systems provide generic statistic tools that can be used to track participant behaviours, even though most of them only provide a quantitative picture of events (EduTools, 2008). In contrast, what is needed is a complete, both quantitative and qualitative picture, able to inform the assessment of student individual learning (both in progress and at the end of the course) and the evaluation of the whole instructional initiative.

More specifically, formative assessment of students usually takes place during the course and is an essential condition to ensure a good degree of personalization of the learning process. As such, it relies heavily on information gathered through monitoring. Summative assessment of students is a frequent context requirement of formal learning institutions and generally includes not only product evaluation, that can be used as an indicator of effectiveness, but also process evaluation, based on the same data as those that inform formative assessment.