Abstract
Evaluation refines and validates design solutions in order to establish adequate user experiences. For mobile user interfaces in dynamic and critical environments, user experiences can vary enormously, setting high requirements for evaluation. This chapter presents a framework for the selection, combination, and tuning of evaluation methods. It identifies seven evaluation constraints, that is, the development stage, the complexity of the design, the purpose, participants, setting, duration, and cost of evaluation, which influence the appropriateness of the method. Using a combination of methods in different settings (such as Wizard-of-Oz, game-based, and field evaluations) a concise, complete, and coherent set of user experience data can be gathered, such as performance, situation awareness, trust, and acceptance. Applying this framework to a case study on context-aware mobile interfaces for the police resulted in specific guidelines for selecting evaluation methods and succeeded to capture the mobile context and its relation to the user experience.
Key Terms in this Chapter
Ethnography: An evaluation method in the analysis stage during which end-users are observed and interviewed in their work environment
Empirical Evaluation: An evaluation method in which results are derived by observation or experiment instead of theory
User Experience: The set of cognitive, affective, and social responses that are induced by the use of a product or service
Context Aware Technology: Technology that models a set of factors about the use context to adapt the interaction with the user
Wizard of Oz Evaluation: An evaluation method in which a functioning system is simulated by a person controlling a non-functioning prototype
Game-Based Evaluation: An evaluation method in which a simulated environment is used adapted from existing computer games
User-Centered Design: A design methodology in which end-users needs and requirements guide the design choices
Adaptive User Interface: A computer interface that adapts the interaction guided by a model of a specific set of factors
Requirements Analysis: An analysis method during the analysis stage in which the user needs and requirements for design solutions are specified