Using Discourse Analysis to Assess Student Problem-Solving in a Virtual World

Introduction

In the higher education sector, substantial resources and time has been devoted to the development of virtual and game-based environments. While numerous studies have been undertaken on student and staff perspectives of the technologies – see, for example, Freitas and Neumann (2009) and Wouters, Spek, and Oostendorp (2011) – little attention has been placed on conducting and evaluating assessment in these environments.

Of the small body of work that directly examines assessment in these environments, one of the cited benefits of using virtual worlds in higher education is that they provide students with an opportunity to interact with an authentic context and develop the skills and knowledge necessary to succeed in professional practice. For example, Tuten (2009) in her study on authentic assessment in business education found that the use of Second Life for business students provided an authentic opportunity for the students to develop marketing plans, which enhanced task significance. Similarly, Forsberg, Georg, Ziegert, and Fors (2011) noted that using virtual cases in nursing was well-accepted by the students as the experience reflected specific tasks that they would encounter in their nursing practice. In a study on the use of Second Life to support language learning, Grant and Clerehan (2011) found that the complexity of the virtual task, which required students to order food and converse in Mandarin in a virtual restaurant, may have had an impact on students’ ability to complete the assessment. What these three case studies all identified was that the authentic nature of the task was perceived to be of value to the students, but that the complexity of the environment may have caused a tension between learning and technical proficiency.

What is needed here is an understanding of how the students actually interact with the environment so that strategies to support students can be embedded into the learning activity and assessment. As Clarke-Midura and Dede (2010) explain that while an assessment can serve multiple purposes, it is simply not possible for one assessment to meet all purposes within the context of a student’s learning experience. Hence, the actual design and purpose of the assessment needs to be carefully considered so that mastering the technology is not a barrier to undertaking the assessment. Innovative assessment formats, such as is made possible in virtual and game-based environments, can measure complex knowledge, professional skills and inquiry that is not possible in paper-based formats (Clarke-Midura & Dede, 2010). However, a discussion on the range and potential measure of student learning in assessments needs to be broadened into include innovative methods in eResearch.

In this chapter, we investigate students’ real time communication whilst they engage in a virtual activity. An analysis of six groups’ interactions while they participated in a virtual problem-solving activity is presented. The rationale for this is that by examining student talk, we can see how students establish roles, identify the rules of the environment plan and execute goals and overcome problems. This chapter is positioned within the research on computer-supported collaborative learning (CSCL) and discusses the value of discourse analysis as a means to assess students’ in-world actions. Our focus in this chapter is on tense, modality and pronouns.