Methodology for Developing a Usability Assessment Questionnaire in Spanish as a Bilingual Strategy for Software Improvement: A Validation Approach

Methodology for Developing a Usability Assessment Questionnaire in Spanish as a Bilingual Strategy for Software Improvement: A Validation Approach

Manuel Alejandro Barajas Bustillos (Autonomous University of Ciudad Juárez, Mexico), Aide Aracely Maldonado-Macías (Autonomous University of Ciudad Juarez, Mexico), Juan Luis Hernández Arellano (Autonomous University of Ciudad Juárez, Mexico), Liliana Avelar Sosa (Autonomous University of Ciudad Juárez, Mexico) and Rosa María Reyes Martínez (Technological Institute of Ciudad Juárez, Mexico)
DOI: 10.4018/978-1-5225-6992-3.ch007


Usability is the characteristic of a software product of being effective and efficient and producing satisfaction for users and traditionally is assessed through questionnaires but most of them are only available in English. A software usability assessment questionnaire (SUAQ) is proposed in two languages: Spanish and English. The methodology comprises four stages: 1) questionnaire development, 2) administration, 3) statistical validation, and 4) sample size determination. Twenty items were evaluated in terms of clarity, consistency, and relevancy. Then, the SUAQ was administered to 95 respondents. Overall, reliability values were acceptable in Spanish and English version, respectively. The factor analysis was feasible since the KMO index, and the Bartlett sphericity test was statistically significant. Both versions of SUAQ were tested to determine their validity. The findings show that the proposed methodology is an effective usability assessment instrument and thus an effective software improvement tool from a bilingual approach.
Chapter Preview


Usability is a quality attribute in software engineering (Pressman & Maxim, 2015), and its definition varies across sources. For example, according to the international standards ISO/IEC 25010:2011 (ISO/IEC, 2011) and ISO 9241-11:1998 1998 (International Organization for Standardization, 1998), usability is a characteristic that allows a product to be used effectively, efficiently, and satisfactorily. Similarly, to Jakob Nielsen (Nielsen, 1993), usability refers to characteristics in a product, such as learnability, efficiency, memorability, low error levels, and satisfaction. However, even though organizations such as the International Organization for Standardization and the European Telecommunications Standards Institute (International Organization for Standardization, 1998; ISO/IEC, 2011) recognize the importance of usability in software development, none of them have proposed a usability assessment instrument. Developers thus have had to select among a series of available instruments, such as field observations, heuristic evaluations, focus groups, surveys, and questionnaires.

Questionnaires are perhaps the most popular usability assessment instruments. Examples of well-known usability assessment questionnaires are the Questionnaire for User Interface Satisfaction (QUIS), developed by Ben Shneiderman in 1987 (Shneiderman, 1987), the Perceived Usefulness and Ease of Use (PUEU) instrument, proposed by Fred Davis in 1989, the Software Usability Measurement Inventory (SUMI), developed by Jurek Kirakowski in 1993 (Kirakowski, 1999), the Computer System Usability Questionnaire (CSUQ), employed by IBM and tested by James R. Lewis in 1995 (Lewis, 1995), the Purdue Usability Testing Questionnaire (PUTQ), developed by Han X. Lin, et al, in 1997 (Lin, Choong, & Salvendy, 1997), and the System Usability Scales, used at DEC and tested in 1991 by N. M. Lucey (Kirakowski, 1999). All these questionnaires are available in English, but their versions in other languages are rare or restricted.

With regards to the usability assessment questionnaires, they consist of different types of questions. General questions, usually known as demographic or sociodemographic questions, allow researchers to make inferences in terms of user age, gender, and experience, among others. However, the level of flexibility in general questions can compromise the data analysis. Next, open ended questions are generally less reliable than closed-ended questions, which is why they should be used only when the scope of the responses cannot be covered through pre-established options. On the other hand, close ended questions can be answered with “yes” or “no” or have a restricted set of possible answers (i.e., A, B, C, All of the above).

Perhaps the main disadvantage of close ended questions is that they might force respondents to make choices they would not make in the real world; nonetheless, they are time-efficient and statistically easy to analyze (European Telecommunications Standards Institute, 2000). In addition, every questionnaire must include instructions and an explanation of its goal, so that researchers can obtain clearer results. Similarly, before being administered, the instrument must be validated from a series of perspectives such as content validity, internal consistency, and construct validity (Aliaga Tovar, 2006; Faan, Polit, & Beck, 2013; Hernández, Fernández, & Baptista, 2010).

Key Terms in this Chapter

Usability: Relates to the product or software characteristic that deals with the easiness of use in order to achieve a certain objective.

Assessment: The process of making a judgement after appreciating or calculating the importance of a given thing or matter.

Validation: The action and effect of turning something into something valid, giving it strength or determination.

Bilingualism: A person's ability to use two languages indistinctly, whether native or acquired.

Software Engineering: A discipline formed by a set of methods, tools, and techniques that are used in the development of software.

Software: A set of computer programs, instructions and rules that allow you to perform different tasks on a computer.

Questionnaire: A set of questions, or items, designed to obtain information for a specific purpose. There are numerous of styles and formats of questionnaires, according to a specific purpose.

Computer Programs: Normally includes several elements. For the purposes of this work, only the elements that interact with the user are considered, mainly the user interface where the information is displayed.

Translating: The action and effect of expressing in one language something that has been previously expressed or written in another language.

Cognitive: Related to knowledge acquisition, processing, and storage of information, available thanks to a learning process or experience.

Complete Chapter List

Search this Book: