Evaluating the Quality of Mobile Medical Speech Translators Based on ISO/IEC 9126 Series: Definition, Weighted Quality Model and Metrics

Evaluating the Quality of Mobile Medical Speech Translators Based on ISO/IEC 9126 Series: Definition, Weighted Quality Model and Metrics

Nikos Tsourakis (ISSCO/TIM/FTI, University of Geneva, Geneva, Switzerland) and Paula Estrella (FaMAF, Universidad Nacional de Córdoba, Córdoba, Argentina)
Copyright: © 2013 |Pages: 20
DOI: 10.4018/ijrqeh.2013040101
OnDemand PDF Download:


Medical translation systems present an intriguing research area as language barriers can become life-threatening when health issues come into place. There is however a lack of common evaluation methodologies, making the fair comparison of such systems a difficult task. In this work the authors try to remedy this deficiency by proposing a quality model based on the ISO/IEC 9126 standard that could serve as a comparison basis among homologous systems. Their work involves the definition of the internal and external quality characteristics of the model along with the quantification of their relative importance based on the results of polling two groups of users (12 doctors and 12 potential patients) that demonstrate different needs and goals towards the system. Additionally, for the internal characteristics that reflect mainly a technical viewpoint the authors polled 6 developers. Finally, as the model contains attributes that are measurable the authors proposed metrics for each characteristic.
Article Preview


The world's current population of about 7 billion speaks almost 7,000 languages (Lewis, 2009). Language barriers often cause inconvenience but when medical issues are involved they cease to be mere inconvenience and can become life-threatening. Anyone who has had the misfortune to fall ill in a country where they do not speak the local language will be painfully aware of this from their own experience. Moreover, quantitative studies have shown that limited English proficiency correlates with a vastly increased probability of negative outcomes across a variety of objective indicators: poor access to medical care (Brach & Chevarley, 2008; DuBard & Gizlice, 2008; Pippins, Alegría, & Haas, 2007), excess hospitalizations, medical errors, and drug complications (Bard et al., 2004; Flores, 2005; Hampers, Cha, Gutglass, Binns, & Krug, 1999; Waxman & Levitt, 2000), and poor satisfaction with the provided care (Flores, 2005).

Unfortunately, trained medical translators are both scarce and expensive. Even if an interpreter is available, health-care practitioner expressed dissatisfaction with both their own methods of working with them and with the interpreter's qualifications (Kale, 2010). Although a universal speech-to-speech translator still seems an insurmountable problem, the substantial gap between the need for and availability of language services in health care could be bridged through effective medical speech translation systems (Bouillon et al., 2007; Fluential, 2012); such a system would be far more useful to users if it was available on a hand-held device, as exemplified in (Figure 1). Indeed, different systems already moved towards the deployment of mobile speech-to-speech translation applications in academia (Tsourakis, Bouillon, & Rayner, 2009; Zhang & Vogel, 2008), and commercially (Google, 2012; Prasad et al., 2011; Voxtec, 2012).

Figure 1.

Doctor interacting through the mobile device

During the development lifecycle of these systems authors have provided evaluation results leveraging various computer and human centered metrics. Despite some early efforts towards a common evaluation framework (Rayner et al., 2008) we argue that there is a lack of such a methodology that would provide a fair comparison framework for different mobile medical translation systems. Additionally, the lack of appropriate quality assessment techniques can deteriorate user satisfaction. In the US for example, user dissatisfaction even caused legislative actions governing software quality and suppliers' responsibilities (Kaner, 2000). The purpose of this paper is to provide a common evaluation methodology in order to compare mobile medical speech translators and also to help developers focus on those aspects of quality that users deem important.

Complete Article List

Search this Journal:
Open Access Articles: Forthcoming
Volume 6: 4 Issues (2017)
Volume 5: 4 Issues (2016)
Volume 4: 4 Issues (2015)
Volume 3: 4 Issues (2014)
Volume 2: 4 Issues (2013)
Volume 1: 4 Issues (2012)
View Complete Journal Contents Listing