The development of appropriate lab-based evaluation techniques for mobile technologies requires continued research attention. In particular, experimental design needs to account for the environmental context in which such technologies will ultimately be used. This requires, in part, that relevant environmental distractions be incorporated into evaluations. This chapter reflects on different techniques that were used in three separate lab-based mobile evaluation experiments to present visual distractions to participants and to measure the participants’ cognizance of the distractions during the course of mobile evaluations of technology. The different techniques met the different needs of the three studies with respect to the fidelity of the data captured, the impact of acknowledging distractions on the evaluation task, and the typical context of use for the technology being evaluated. The results of the studies showed that the introduction of visual distractions did have an impact on the experimental task and indicate that future work is required in this area.
Key Terms in this Chapter
Interfering Distractions: Distractions incorporated within a lab-based experiment which may be passive or active and they interfere with a user’s ability to effectively interact with a mobile device.
Context: The intended context of use for a mobile application.
Field Study: The evaluation of a mobile application that takes place in the actual context of use. The advantage of such evaluations is that problems that only arise in the particular context will be detected. The disadvantages of such evaluations include difficulties in controlling the environment and capturing evaluation data.
Visual Distraction: Distractions that require participants’ visual attention.
Passive Distractions: Distractions incorporated within a lab-based experiment which distract users but require no active response.
Lab Evaluation: The evaluation of a mobile application that takes place in a laboratory. The advantages of such evaluations include ease of controlling the environment and data capture. Disadvantages include difficulties in creating an appropriately realistic evaluation setting.
Active Distractions: Distractions incorporated within a lab-based experiment which require a user to respond or react in some way (the required response varying according to the nature of the distraction).
Complete Chapter List
Anxo Cereijo Roibás, Stephen Johnson
Hanna Stelmaszewska, Bob Fields, Ann Blandford
Hyowon Lee, Cathal Gurrin, Gareth J.F. Jones, Alan F. Smeaton
Amy K. Karlson, Benjamin B. Bederson, Jose L. Contreras-Vidal
Martina Ziefle, Susanne Bay
Susanne Bay, Martina Ziefle
Chris Barber, James Knight
Anind K. Dey, Jonna Häkkilä
Bent Schmidt-Nielsen, Bret Harsham, Bhiksha Raj, Clifton Forlines
Nikolaos Tselios, Ioanna Papadimitriou, Dimitrios Raptis, Nikoletta Yiannoutsou, Vassilis Komis, Nikolaos Avouris
Siu Cheung Kong
Hyungsung Park, Young Kyun Baek, David Gibson
Nikola Mitrovic, Eduardo Mena, Jose Alberto Royo
Michael J. O’Grady, Gregory M.P. O’Hare
Yang Li, Scott Klemmer, James A. Landay
Emmanuel Dubois, Wafaa Abou Moussa, Cédric Bach, Nelly de Bonnefoy
Ioannis D. Zaharakis, Achilles D. Kameas
Rafael Ballagas, Michael Rohs, Jennifer G. Sheridan, Jan Borchers
Mark David Dunlop, Michelle Montgomery Masters
Min Lin, Andrew Sears, Steven Herbst, Yanfang Liu
Louise E. Moser, P.M. Melliar-Smith
Dong Yu, Li Deng
Parisa Eslambolchilar, Andrew Crossan, Roderick Murray-Smith, Sara Dalzel-Job, Frank Pollick
Panu Korpipää, Jukka Linjama, Juha Kela, Tapani Rantakokko
Enrico Costanza, Samuel A. Inverso, Rebecca Allen, Pattie Maes
Tolga Capin, Antonio Haro
Andrea Sanna, Fabrizio Lamberti
Rock Leung, Joanna Lumsden
Mark Matthews, Gavin Doherty, David Coyle, John Sharry
Francesco Bellotti, Riccardo Berta, Alessandro De Gloria, Massimiliano Margarone
Shigueo Nomura, Takayuki Shiose, Hiroshi Kawakami, Osamu Katai, Keiji Yamanaka
Florence Gaunet, Xavier Briffault
Julio Abascal, Borja Bonail, Daniel Cagigas, Nestor Garay, Luis Gardeazabal
Regina Bernhaupt, Kristijan Mihalic, Marianna Obrist
Jan Willem Streefkerk, Myra P. van Esch-Bussemakers, Mark A. Neerincx, Rosemarijn Looije
Enrico Bertini, Tiziana Catarci, Alan Dix, Silvia Gabrielli, Stephen Kimani, Giuseppe Santucci
Thomas Alexander, Christopher Schlick, Alexander Sievert, Dieter Leyk
Maria de Fátima Queiroz Vieira Turnell, José Eustáquio Rangel de Queiroz, Danilo de Sousa Ferreira
Jaakko T. Lehikoinen
Dong-Han Ham, Jeongyun Heo, Peter Fossick, William Wong, Sanghyun Park, Chiwon Song, Mike Bradley
Kaikkonen, Kaikkonen, Anne, Anne, Aki Kekäläinen, Mikael Cankar, Titti Kallio
Murray Crease, Robert Longworth
Andrew Crossan, Roderick Murray-Smith, Stephen Brewster, Bojan Musizza
Murray Crease, Joanna Lumsden
Rune T. Høegh, Jesper Kjeldskov, Mikael B. Skov, Jan Stage
Adrian Stoica, Georgios Fiotakis, Dimitrios Raptis, Ioanna Papadimitriou, Vassilis Komis, Nikolaos Avouris
Kater Oakley, Gitte Lindgaard, Peter Kroeger, John Miller, Earl Bryenton, Paul Hébert
Shwetak N. Patel, Khai N. Truong, Gillian R. Hayes, Giovanni Iachello, Julie A. Kientz, Gregory D. Abowd
Saturnino Luz, Masood Masoodian
Jason T. Black, Lois Wright Hawkes
Tiong T. Goh, Kinshuk, Nian-Shing Chen