Participant Experiences of Mobile Device-Based Diary Studies

Introduction

As well as being a target of HCI research, mobile devices have a tremendous value as a tool for HCI research (Consolvo & Walker, 2003; Carter & Mankoff, 2007; Klasnja et al., 2008). Mobile devices support a whole range of rich data collection opportunities, be that through event and location logging, through response to mobile-based surveys, or through the capture of user-generated text, audio, images and video. More than ever, people are familiar with such devices. Indeed, there are 1.08 billion mobile smart phones in use in the world (AnsonAlex, 2012), allowing a research ‘app’ to be loaded directly on to their own device. The result is a potentially less obtrusive way to capture behaviour, attitudes and perceptions at the time and place where they are occurring.

The ability to collect data in the field has a number of advantages. At a general level, it should lead to more valid data (e.g. automatic data capture such as place and time) particularly for activities, such as travel behaviour or health behaviour, where there is a potential disconnect between stated preferences, intentions, and actual behaviour (Gardner, 2009; Shiffman et al., 2008). Another advantage is that it allows prolonged data capture without the need for a researcher to be present. Not only does this increase the number of data points, it allows a researcher to see how patterns of behaviour change over time (Bolger et al., 2003; Shiffman et al., 2008). Finally, mobile devices support flexible data capture, often using a number of media such as text, audio and video, at the point when the behaviour in question occurs, and the later use of that data for reflection with an investigator. This type of diary study is referred to as an ‘elicitation study’ (Carter & Mankoff, 2005), in contrast to ‘feedback’ studies which use constrained, predefined questions asked either at certain intervals or during certain events.

While there is guidance available on how to approach paper and technology-based diary and experience sampling studies (Bolger et al., 2003; Consolvo & Walker, 2003; Shiffman et al., 2008), there is less known about the participant experience of mobile data capture and how that can be fed into the design of methodology. In this paper, we shed light on the value of different approaches in the domain of HCI by drawing upon three case studies: a) an experience sampling application to capture serendipitous information acquisition b) an application to record travel motivations and track travel behaviours and c) an application to monitor the use of data sources on a mobile devices to understand users’ ‘contextual footprint’ (i.e. their pattern of data use over time). While each case varies in domain and specific data collection approach, all had the same aim of using a mobile application to capture contextual data for user requirements and subsequent technology development.

The particular focus of this paper is participants’ experience of mobile data collection, their attitude, preference and problems of using such an application and how the design of mobile diary methods can be adapted and improved. The paper therefore has three contributions. First, it provides three cases of mobile data capture as exemplars, and uses these to highlight some of the variability of approaches that can be used (e.g. elicitation versus feedback, active versus passive data capture). Second, it provides a categorisation of the major perceptions participants have of being involved in mobile-device based diary studies, relating to issues such as usability, privacy, and motivation. Third, based on these perceptions we present some of the potential issues, and offer solutions, that need to be considered when electing to use a mobile-device based diary study.