Methodology for Developing a Usability Assessment Questionnaire in Spanish as a Bilingual Strategy for Software Improvement: A Validation Approach

Background

Usability is a quality attribute in software engineering (Pressman & Maxim, 2015), and its definition varies across sources. For example, according to the international standards ISO/IEC 25010:2011 (ISO/IEC, 2011) and ISO 9241-11:1998 1998 (International Organization for Standardization, 1998), usability is a characteristic that allows a product to be used effectively, efficiently, and satisfactorily. Similarly, to Jakob Nielsen (Nielsen, 1993), usability refers to characteristics in a product, such as learnability, efficiency, memorability, low error levels, and satisfaction. However, even though organizations such as the International Organization for Standardization and the European Telecommunications Standards Institute (International Organization for Standardization, 1998; ISO/IEC, 2011) recognize the importance of usability in software development, none of them have proposed a usability assessment instrument. Developers thus have had to select among a series of available instruments, such as field observations, heuristic evaluations, focus groups, surveys, and questionnaires.

Questionnaires are perhaps the most popular usability assessment instruments. Examples of well-known usability assessment questionnaires are the Questionnaire for User Interface Satisfaction (QUIS), developed by Ben Shneiderman in 1987 (Shneiderman, 1987), the Perceived Usefulness and Ease of Use (PUEU) instrument, proposed by Fred Davis in 1989, the Software Usability Measurement Inventory (SUMI), developed by Jurek Kirakowski in 1993 (Kirakowski, 1999), the Computer System Usability Questionnaire (CSUQ), employed by IBM and tested by James R. Lewis in 1995 (Lewis, 1995), the Purdue Usability Testing Questionnaire (PUTQ), developed by Han X. Lin, et al, in 1997 (Lin, Choong, & Salvendy, 1997), and the System Usability Scales, used at DEC and tested in 1991 by N. M. Lucey (Kirakowski, 1999). All these questionnaires are available in English, but their versions in other languages are rare or restricted.

With regards to the usability assessment questionnaires, they consist of different types of questions. General questions, usually known as demographic or sociodemographic questions, allow researchers to make inferences in terms of user age, gender, and experience, among others. However, the level of flexibility in general questions can compromise the data analysis. Next, open ended questions are generally less reliable than closed-ended questions, which is why they should be used only when the scope of the responses cannot be covered through pre-established options. On the other hand, close ended questions can be answered with “yes” or “no” or have a restricted set of possible answers (i.e., A, B, C, All of the above).

Perhaps the main disadvantage of close ended questions is that they might force respondents to make choices they would not make in the real world; nonetheless, they are time-efficient and statistically easy to analyze (European Telecommunications Standards Institute, 2000). In addition, every questionnaire must include instructions and an explanation of its goal, so that researchers can obtain clearer results. Similarly, before being administered, the instrument must be validated from a series of perspectives such as content validity, internal consistency, and construct validity (Aliaga Tovar, 2006; Faan, Polit, & Beck, 2013; Hernández, Fernández, & Baptista, 2010).