In order to understand potential applications of collaborative problem solving (CPS) assessment tasks, it is necessary to examine empirically the multi-faceted student performance that may be distributed across collaboration methods and purposes of the assessment. Ideally, each student should be matched with various types of group members and must apply the skills in varied contexts and tasks. One solution to these assessment demands is to use computer-based (virtual) agents to serve as the collaborators in the interactions with students. This paper proposes human-to-agent (H-A) approach for formative CPS assessment and describes an international pilot study aimed to provide preliminary empirical findings on the use of H-A CPS assessment to inform collaborative learning. Overall, the findings showed promise in terms of using H-A CPS assessment task as a formative tool for structuring effective groups in the context of CPS online learning.
TopIntroduction
This chapter is capitalizing on the affordances of digital tools to deepen and extend the kinds of science learning and assessment highlighted in the Framework for K–12 Science Education and the Next Generation Science Standards (National Research Council, 2012), Programme for International Student Assessment (PISA) Framework for Science and Collaborative Problem Solving (OECD, 2013a, 2013b). On one hand, a scientifically literate person is willing to engage in reasoned discourse about science and technology which requires the competencies to explain phenomena scientifically, evaluate and design scientific enquiry, interpret data and evidence scientifically, and analyze and evaluate data, claims and arguments in a variety of representations and draw appropriate scientific conclusions (OECD, 2013a). On the other hand, collaborative problem solving (CPS) competency is characterized by the capacity of an individual to effectively engage in a process whereby two or more agents attempt to solve a problem by sharing the understanding and effort required to come to a solution and pooling their knowledge, skills, and efforts to reach that solution (OECD, 2013b). That is, the OECD frameworks for assessment build upon an individual assessment of problem-solving, which was already defined and well understood in earlier PISA assessments (Chauncey, & Azevedo, 2010), and conjoins that definition with a new domain framework of collaboration made operational in a simulated collaborative context. An agent could be considered either a human agent or a computer agent that interacts with the student. The competency is typically assessed by evaluating how well the individual collaborates with agents during the problem-solving process. This includes establishing and maintaining shared understanding, taking appropriate actions to solve the problem, and establishing and maintaining group organization. CPS refers to problem-solving activities that involve collaboration among a group of individuals (O’Neil, Chuang, & Baker, 2010; Zhang, 1998). It is a conjoint construct consisting of collaboration, or: “coordinated, synchronous activity that is the result of a continued attempt to construct and maintain a shared conception of a problem” (Roschelle, & Teasley, 1995, p. 70), and problem solving, or: “cognitive processing directed at achieving a goal when no solution method is obvious to the problem solver” (Mayer, & Wittrock, 1996). CPS has both cognitive and social aspects, and the outcomes from a CPS tasks are generally the results of the interaction of both. Liu, et al. (2015) proposed a conceptual model that documents a matrix of individual and social skills involved in CPS, which provides a basis for designing rich performance assessments tasks in the context of science. The individual cognitive skills are used to complete tasks independently of other team members. In this individual dimension of the CPS skills contextualized to inquiring knowledge, the following cognitive skills: conceptual understanding and inquiry skills in science (e.g., data collection, data analysis, prediction making, evidence-based reasoning). The second dimension in the CPS skills matrix is social skills, which are often acquired through social interactions with peers and can affect both group and individual performance. There are four major categories of social skills, namely sharing information, assimilating and accommodating knowledge, regulating problem-solving activities, and maintaining a positive communication atmosphere.