Multiple-User Simultaneous Testing: Experience with Two Methods

Abstract

Web developers and usability specialists face the challenge of dealing with the cost and quality of usability testing that must be implemented in a short period of time. Multiple-User Simultaneous Testing (MUST) can reduce usability testing time by allowing data collection from many users at the same time. This chapter reviews the literature on MUST and related methodologies for group users. It describes on a conceptual level two methods for setting up the MUST testing environment: self-paced and moderated. The authors do not aim to present which method is better from the empirical standpoint. Instead, the chapter describes the authors’ comparative experiences with these methods, with a focus on the laboratory set up, data collection protocols, and data analysis issues. The chapter concludes with suggestions for future research and recommendations for usability specialists on how to conduct well designed MUST studies.