Article Preview
TopIntroduction
As smart phones become almost ubiquitous, User Experience (UX) is increasingly recognised as a significant differentiator for applications in a crowded marketplace. Thus, the inclusion of User Centered Design (UCD) processes which can deliver excellent user experience is ever more significant in the design and development of mobile and social Information and Technology (ICT) services. Understanding users’ expectations, experiences and perceptions is particularly important in the case of context-aware pervasive computing, where context-of-use is directly meaningful for human system interactions.
While traditional usability approaches, such as Think Aloud (Wright & Monk, 1991), or Cognitive Walkthrough (Wharton, Rieman, Lewis, & Polson, 1994), evolved to learn about one user’s interactions with a particular pre-defined set of tasks in one computing application, these and other usability approaches are not easily adapted for multiple-user multiple-service mobile social and pervasive systems, such as the SOCIETIES platform and third party services which we use for a case study discussion in this paper. Attempting to list tasks and functionalities is more complex in the case of multiple user social and context aware systems, as network effects such as feedback and community dynamics and creative appropriation (which are properties that emerge in such systems through relevant data population and usage) are not easily modeled. Social and pervasive systems are evolutionary, and data gathered and generated through monitoring users’ actions in particular contexts, and interactions with services and other users is: 1) particular to the specific users, enabling personalisation; and 2) significant to the emergence of opportunities for sharing of data, services and things offered through the identification of shared interests across communities of users in particular contexts. Therefore, it is difficult to present random users with opportunities that feel sufficiently authentic for them to experience the system from within.
Large extended field trials are often proposed as the preferred evaluation approach for context aware pervasive or social systems using mobile applications. Field trials allow for ecological validity which is important for complex social technical systems—which means facilitating trial participants’ natural usage of services in everyday life, rather than in controlled laboratory conditions (Eagle & Pentland, 2006; Oulasvirta, 2011; Rogers et al., 2007a). It also implies participants’ interactions with the services and systems under investigation will be competing for attention with the myriad of distractions and interruptions associated with active usage ‘in the wild’. However, in the case of early prototypes, which are not yet sufficiently robust or mature for extended deployment with multiple users, more creative solutions are required.
In order to evaluate two medium fidelity Enterprise-focused services designed for a conference scenario, based on a novel social and pervasive platform, with a small number of participants in a workshop setting over the course of one afternoon, we elected to use scripted role-play, combined with other participatory demonstrations and games. A mixed methods approach, including surveys, cognitive walkthrough, games, role play and focus group discussion, afforded us the necessary flexibility and adaptability to glean insights and invite feedback from participants at various different points of the evaluation, from initial expectations and impressions, to immersion in usage, and final considered reflections. Role-play allowed us to simulate situated use and condense conference type interactions to a few hours. We found that it enabled participants to engage in-depth with the presented technologies, whilst also facilitating developers to gain useful insights.