Guidelines for Error Message Design

Introduction

The body of knowledge regarding warning design theory is extensive and includes investigations on what constitutes an effective warning, how to design an effective warning, and when and why warnings work (Laughery & Wogalter, 2006). Traditionally the research in this field is concerned with warnings in the physical world, but more recent research refers to warnings in digital systems. Factors found in traditional warning design are also applicable to digital systems (Bravo-Lillo et al., 2013), which is expected because the principles of warning design are independent of the specific product or environment (Laughery, 2006).

Error messages have received less attention, where the body of knowledge has limited guidance on the content of error messages (Amer & Jo-Mae, 2010). Error messages in this case refer to messages that appear in digital systems to indicate that an error has occurred. Such errors are usually a result of an invalid user action or problems encountered while processing data.

To address the lack of research on the design of error messages, warning design theory could be applied to error messages. The assumption is that the effectiveness of an error message can be determined on the same grounds as for warnings, that is, whether the warning draws and holds attention, whether information provided to the user is understood, and whether it results in the intended behavioral change (Laughery, 2006). This is expected to be true because error messages are similar to compliance and warning messages (Amer & Jo-Mae, 2010).

If this assumption holds, then it should be possible to combine warning design theory with error message design research into a set of design guidelines for error messages. The objective of this study was, therefore, to create this set of guidelines and (at least partially) test the assumption. Such a set of guidelines could be used to inform the design of error messages and for early evaluation of user interfaces through a heuristic evaluation (Hollingsed & Novick, 2007). A heuristic evaluation entails multiple experts evaluating a user interface based on set of usability principles, referred to as heuristics (Nielsen, 1992). A heuristic evaluation focusing exclusively on error messages can be valuable because usability problems related to error messages may prove costly (Nielsen & Molich, 1990).

If the guidelines are successful in informing the design and development process and assisting in early evaluation of interface prototypes, the amount of user testing required to evaluate the error scenarios in a system will be reduced. This will be especially beneficial on smaller projects where the cost of user testing is comparably high (Nielsen & Molich, 1990).

If guidelines for error message design are based in part on warning design theory, it is necessary to validate the guidelines to prove that warning design theory is applicable to error messages. Additionally, the guidelines have to be evaluated to determine whether they are easy to understand and apply. In this study, the evaluation was done by completing two usability studies. The first was a heuristic evaluation of a specific set of error messages using the guidelines as heuristics, and the second entailed individual interviews with users to evaluate the same set of error messages. The outcomes of the two evaluations were compared to assess how successfully the heuristic evaluation could identify the errors that emerged from the user evaluation.

In summary, the purpose of the research was to develop a set of guidelines for error message design to address the lack of such guidelines reported in the literature. Also, to provide a usable tool for informing the design and evaluation of error messages. To ensure that the proposed guidelines are valid and effective they were firstly informed by existing literature and theory, and secondly validated through two usability studies.

We next provide a brief review of the literature on warning and error message design. Following that, we explain in detail the methods used to develop, evaluate and refine a set of guidelines for error message design. In the Results and Discussion sections we present the initial guidelines that emerged from the literature, the outcomes of the evaluations and, finally, the refined set of guidelines.