3D Virtual Worlds: Assessing the Experience and Informing Design

Introduction

Over one billion hours of collaborative play have been logged in the game Halo (Grossman, 2007). Put into easily understood commercial terms, that’s about 526,315 person years of work effort; or the entire population of Ljubljana, Slovenia playing Halo instead of going to work, school or anywhere else during the day for an entire year. This milestone underscores the increasing acceptance of virtual worlds as legitimate, compelling places for humans to interact with one another, making it likely that 3D interaction technologies will be adopted beyond the realm of play. In fact, there is evidence that 3D virtual worlds are becoming a prominent framework for human-computer interaction (HCI) to support distributed, collaborative work (Kaptelinin & Czerwinski, 2007).

Much of what is known about HCI in 3D emerges from the study of video games. Qualitative studies of how gamers experience virtual worlds show that new games are adopted faster if they follow familiar interaction styles (Clarke & Duimering, 2006), and that game play sometimes leads to a new category of virtual community (Nardi & Harris, 2006). These studies provide preliminary guidance for HCI designers searching for new metaphors that might support deeper engagement in collaborative work among geographically dispersed groups (Carroll, 2003; Dyck, Pinelle, Brown, & Gutwin, 2003; Rapeepisarn, Wong, Fung, & Depickere, 2006). Designing software for distributed group work is recognized as a wicked problem with many challenging dimensions (Fitzpatrick, 1998). Grudin (1994) identified eight of the most significant challenges of designing software for collaborative work. Each of Grudin’s eight challenges takes a slightly different form as technology changes. In this paper, we specifically address one of Grudin’s eight challenges for the 3D generation of collaboration technologies: Evaluation.

Before explaining how evaluation of 3D environments might be different than evaluation of other types of collaborative work systems, we need to understand how users experience 3D collaboration differently. Dyck et al.’s (2003) analyses of gaming as an interaction metaphor reveals the core dimensions of effortless community, learning by watching, deep customizability and fluid system-human interaction. These dimensions of 3D games present an opportunity for 3D collaboration software designers to transfer ideas from gaming (Rapeepisarn et al., 2006). Wrapped up with this opportunity are challenges to long accepted heuristics for interface design, including simplicity, consistency & ease of use for all users. Dyck et al (2003) describe how these accepted HCI design heuristics do not transfer to 3D interaction design. HCI convention is further undermined by evidence that gamers enjoy learning a game quickly but then wish for it to become more difficult as they advance through different stages.

Gaming style interactions like those encountered in 3D environments are more compelling than conventional windowing systems (Paiva et al., 2002; Larson, 2007) and could form the basis for more engaging and productive HCI. The evaluation heuristics that we communicate to designers of these types of systems must, then, necessarily reflect both the fundamental differences and potential advantages of 3D collaboration. First, we must make a critical distinction between 3D virtual worlds and games: In the 3D environments tuned for work, collaboration is measured by creative output and group efficacy, not by a body count or other discrete score. One of the premises underlying the work presented here is that while the HCI community might learn from the example of game developers, building worlds to support creative collaboration is distinct from the challenge of creating a really great first person shooter (FPS) game. While the gaming community knows what works (Crawford, 1982; Rollings & Adams, 2003), the development and evaluation of collaborative 3D environments are in a more formative state.