Experimental Setups for User Evaluation of Mobile Devices and Ubiquitous Systems

Experimental Setups for User Evaluation of Mobile Devices and Ubiquitous Systems

Francis Jambon (Grenoble University, France)
DOI: 10.4018/978-1-60960-499-8.ch009

Abstract

Nowadays, mobile devices features are often linked up to the context of usage. As a consequence, researchers must consider not only the user and the device, but also the surrounding environment when designing effective user study evaluations. Two opposite experimental setups are possible: in-situ and in the laboratory. There is no consensus on their respective benefits, for instance with regard to the number of usability issues detected. In this chapter, the author isolates independent variables that could contribute to evaluation biases by proposing a taxonomy that splits the in-situ and laboratory experimental setups into two new setups. The author describes the concept of the “Uncertainty Principle” to emphasize the dilemma between precise observation and bias minimization and introduce the “Trojan Horse” technique to partially overcome the consequences of the uncertainty principle. As a conclusion, a methodology using the four experimental setups in a complementary way is proposed.
Chapter Preview
Top

Methods: Laboratory Or In-Situ?

First, it is necessary to determine what are the relevant elements of the interactive environment that must be set up for the experiment. We propose to structure this set of elements into four categories: the user (or users if a collaborative environment is under evaluation); the devices in direct interaction with the user; the tasks directly related to the evaluated devices; and the context, which is defined as a complementary set of the three preceding elements, i.e., other users, other devices, and tasks with no direct link with the evaluation.

Complete Chapter List

Search this Book:
Reset