Abstract
This chapter introduces a new usability evaluation method, the axiomatic evaluation method, which is developed based on the axiomatic design theory – a formalized design methodology that can be used to solve a variety of design problems. This new evaluation method examines three domains of a product: customer domain, functional domain, and control domain. This method investigates not only usability problems reported by the users, but also usability problems related to customer requirements and usability problems related to control through checking the mapping matrix between the three domains. To determine how well this new usability evaluation method works, a between-subject experiment was conducted to compare the axiomatic evaluation method with the think aloud method. Sixty participants were randomly assigned to use either method to evaluate three popular consumer electronic devices (music player, digital camera, mobile phone) that represented different levels of complexity. Number of usability problems discovered, completion time, and overall user satisfaction were collected. Results show that the axiomatic evaluation method performed better in finding usability problems for the mobile phone. The axiomatic evaluation method was also better at finding usability problems about user expectation and control than the think aloud method. Benefits and drawbacks of using the axiomatic evaluation method are discussed.This chapter introduces a new usability evaluation method, the axiomatic evaluation method, which is developed based on the axiomatic design theory – a formalized design methodology that can be used to solve a variety of design problems. This new evaluation method examines three domains of a product: customer domain, functional domain, and control domain. This method investigates not only usability problems reported by the users, but also usability problems related to customer requirements and usability problems related to control through checking the mapping matrix between the three domains. To determine how well this new usability evaluation method works, a between-subject experiment was conducted to compare the axiomatic evaluation method with the think aloud method. Sixty participants were randomly assigned to use either method to evaluate three popular consumer electronic devices (music player, digital camera, mobile phone) that represented different levels of complexity. Number of usability problems discovered, completion time, and overall user satisfaction were collected. Results show that the axiomatic evaluation method performed better in finding usability problems for the mobile phone. The axiomatic evaluation method was also better at finding usability problems about user expectation and control than the think aloud method. Benefits and drawbacks of using the axiomatic evaluation method are discussed.
TopIntroduction
Today usability is a ubiquitous term used in human-computer interaction and commonly used without definition (Hertzum, 2010). The document ISO 9241-11 (1998) defines usability as: the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use. Here effectiveness is defined as the accuracy and the completeness with which users achieve goals in particular environments; efficiency refers to the resources expended in relation to the accuracy and completeness of the goals achieved; and satisfaction is defined as the comfort and the acceptability of the system for its users and other people affected by its use. Nielsen (1993) also considered usability as a measurable construct, and identified five attributes: learnability, efficiency, memorability, errors, and satisfaction. Learnability refers to how easily novice users can learn a new system to reach a reasonable level of usage proficiency within a short time; efficiency means the expert user’s steady-state level of performance that can be obtained once the user has learned the system; memorability refers to how easy it is to remember the system without relearning it again when casual users return to the system after not using it for some time; errors are defined as actions that do not accomplish the desired goals when performing the specified tasks; satisfaction refers to how pleasant and satisfying it is for users to use the system. These definitions of usability have been the basis for many usability evaluation studies that try to find usability problems. In recent years, researchers and designers aim at a more holistic understanding and evaluation of user experience, beyond mere cognitive or ergonomic issues, for example enjoyment, pleasure, emotional and cultural aspects (Macaulay et al., 2006; Wright & McCarthy, 2004). Hertzum (2010) introduces 6 images of usability: universal usability (the systems can be used by everybody), situational usability (quality-in-use of a system in a specified situation with its users, tasks, and wider context of use), perceived usability (user’s subjective experience of a system based on his or her interaction with it), hedonic usability (joy of use), organizational usability (groups of people collaborating in an organizational setting), and cultural usability (usability has different meanings depending on the users’ cultural background).
The term usability evaluation method was popularized by Gray and Salzman (1998). It refers to any method or technique that performs formative usability evaluation of a human-computer interaction design at any stage of the design’s development. Hilbert and Redmiles (2000, pp. 388) defined usability evaluation as “the act of measuring (or identifying potential issues affecting) usability attributes of a system or device with respect to particular users, performing particular tasks, in particular contexts.” The essential characteristic of usability evaluation methods is that every method, when applied to an interaction design, produces a list of potential usability problems as its output. Several methods, including cognitive walkthrough, heuristic evaluation, think aloud, interview, and experience prototyping have been developed and used broadly to discover usability problems (Andre et al., 2003; Buchenau & Fulton, 2000; Gray & Salzman, 1998).