Abstract
Often in our daily lives we learn and work in groups. In recognition of the importance of collaborative and problem solving skills, educators are realizing the need for effective and scalable learning and assessment solutions to promote the skillset in educational systems. In the settings of a comprehensive collaborative problem solving assessment, each student should be matched with various types of group members and must apply the skills in varied contexts and tasks. One solution to these assessment demands is to use computer-based (virtual) agents to serve as the collaborators in the interactions with students. The chapter presents the premises and challenges in the use of computer agents in the assessment of collaborative problem solving. Directions for future research are discussed in terms of their implications to large-scale assessment programs.
TopIntroduction
Collaborative problem solving is recognized as a core competency for college and career readiness. Students emerging from schools into the workforce and public life will be expected to work in teams, cooperate with others, and resolve conflicts in order to solve the kinds of problems required in modern economies. They will further need to be able to use these skills flexibly with various group compositions and environments (Davey, et al., 2015; Griffin, Care, & McGaw, 2012; O’Neil, & Chuang, 2008; Rosen, & Rimor, 2013; Roseth, et al., 2006). Educational programs in K-12 have focused to a greater extent on the advancement of learning and the assessment of collaborative problem solving as a central construct in theoretical and technological developments in educational research (National Research Council, 2011, 2013; OECD, 2013a; National Assessment Governing Board, 2013; U.S. Department of Education, 2010). Collaborative skills are included within the major practices in the 2014 U.S. National Assessment of Educational Progress (NAEP) Technology and Engineering Literacy (National Assessment Governing Board, 2013). In NAEP Technology and Engineering Literacy assessment program, students are expected to show their ability in collaborating effectively with computer-based (virtual) peers and experts and to use appropriate information and communication technologies to collaborate with others on the creation and modification of knowledge products. Similarly, the Israeli national program of adopting the educational system to the 21st century illustrates a multi-year program with the goal of leading the implementation of innovative pedagogy and assessment in schools, including collaboration, communication, and problem solving (Israel Ministry of Education, 2011). Collaborative problem solving is one of the areas that the Organisation for Economic Co-operation and Development (OECD) emphasized for major development in the Programme for International Student Assessment (PISA) in addition to scientific, math, and reading literacy for the 2015 assessment. Collaborative problem solving refers to problem solving activities that involve collaboration among a group of individuals (O’Neil, Chuang, & Baker, 2010; Zhang, 1998). In the PISA 2015 Framework (OECD, 2013b), collaborative problem solving competency is defined as “the capacity of an individual to effectively engage in a process whereby two or more agents attempt to solve a problem by sharing the understanding and effort required to come to a solution and pooling their knowledge, skills, and efforts to reach that solution” (p. 6). This definition treats the competency as conjoint dimension collaboration skills and the skills needed to solve a problem. For the PISA assessment, the focus is on individual capacities within collaborative situations. Thus, the effectiveness of collaborative problem solving depends on the ability of group members to collaborate and to prioritize the success of the group over individual successes. At the same time, this ability is still a trait in each of the individual members of the group. Development of a standardized computer-based assessment of collaborative problem solving skills, specifically for large-scale assessment programs, remains challenging. Unlike some other skills, collaborative problem solving typically requires using complex performance tasks, grounded in varied educational domains, with interaction among students. These factors can affect the level of control that can be applied to ensure accurate assessment of students.
Key Terms in this Chapter
Openness: The degree to which a problem is “well-defined” (e.g., all the information is at hand for the problem solver) vs. “ill-defined” (e.g., the problem solver must discover or generate new information in order for the problem to be solved).
Collaboration: Coordinated, synchronous activity that is the result of a continued attempt to construct and maintain a shared conception of a problem.
Problem Space: The space in which the actions are carried out to solve the problem. Can be explicitly or implicitly visible to team members.
Conflict Situation: Situation in which there are disagreements between the team members on a solution as reflected in communication or actions taken by the team members.
Symmetry of Status: The degree to which the status of team members is the same or is of different rank (e.g., peers vs. supervisor and subordinate relationships).
Semantic Richness: The degree to which the problem provides a rich problem context that relates to the external world.
Symmetry of Roles: The degree to which team members are assigned similar or different roles in a problem scenario.
Agent: Either a human or a computer-simulated participant in a collaborative problem solving group.
Referentiality: A problem’s context may have high referentiality to the outside world and real-world contexts or, at the other end of the spectrum, a low referentiality with little reference to external knowledge.
Perspective Taking: the ability to place oneself in another’s position, which can lead to adaptation and to modification of communication to take the other’s perspective into consideration.
Problem Solving: Cognitive processing directed at achieving a goal when no solution method is obvious to the problem solver.
Collaborative Problem Solving: The capacity of an individual to effectively engage in a group process whereby two or more agents attempt to solve a problem by sharing knowledge and understanding, organizing the group work and monitoring the progress, taking actions to solve the problem, and providing constructive feedback to group members.
Computer Agent: An avatar with a preprogrammed profile, actions, and communication. Computer agents can be capable of generating goals, performing actions, communicating messages, sensing environment, adapting to changing environments, and learning.