Quality in Use Analysis to Evaluate User Experience of Open Source Software Compatible with MATLAB

Quality in Use Analysis to Evaluate User Experience of Open Source Software Compatible with MATLAB

Manar Abu Talib
Copyright: © 2016 |Pages: 19
DOI: 10.4018/IJOSSP.2016070101
(Individual Articles)
No Current Special Offers


Mathematical and statistical open source software (OSS) has played a vital and positive role in computing statistics, performing numerical computations, and solving for real and complex scalars. There is an enormous need to apply quality models to this type of software. In this paper, we present an inclusive set of current quality models and demonstrate their application to OSS. As very little research has been carried out on the quality in use assessment of mathematical software, this work investigates the application of the quality in use model inspired by ISO/IEC 25010 on three OSS products: Scilab, R-Language, and GNU Octave. We can confidently state that Scilab, R-Language, and GNU Octave perform similarly in terms of effectiveness, efficiency, productivity, and cognitive load. However, Scilab outperforms GNU Octave slightly when it comes to error safety.
Article Preview


Quality in use measures the “effectiveness, efficiency and satisfaction with which specified users can achieve specified goals in specified environments” (Bevan, N., 1997).

ISO/IEC 9126-4 (Quality-in-use) is part of ISO/IEC 9126 (Product Quality), which is a whole package for Product Quality. It is divided into four parts, as follows (ISO, 2001):

  • Quality model (ISO/IEC 9126-1);

  • External metrics (ISO/IEC 9126-2);

  • Internal metrics (ISO/IEC 9126-3);

  • Quality-in-use metrics (ISO/IEC 9126-4).

As of March 1, 2011, ISO 9126 has been replaced by ISO 25010 - Systems and Software Quality Requirements and Evaluation - also known as SQuaRE (ISO, 2011). It has eight product quality characteristics, unlike Product Quality (ISO 9126), which only has six characteristics (ISO, 2001).

Quality-in-use is a more comprehensive concept than usability. Usability refers to the software itself. However, quality-in-use embodies the effects of using the software in a specified context such as forensics. According to Alnanih (2015), quality-in-use based on ISO/IEC 25010:2011(E) and ISO/IEC 25022:2014 is “the degree to which a product or system can be used by specific users to meet their needs to achieve specific goals with effectiveness, efficiency, freedom from risk, and satisfaction in specific contexts of use.” ISO/IEC 25010 includes two types of quality characteristics: product quality and quality in use.

In this paper, the quality in use characteristics are discussed as assessment of this type of software has received little attention in the literature. Moreover, Alnanih (2015) quality-in-use model is measured based on the software, system or tool’s usage results, rather than the software properties themselves.

The ongoing challenge to evaluate the quality of MATLAB tools become a critical part since the number of tools that exist is a lot and the users of using these tools are becoming different in their background and technology interests. In this paper, we aim to evaluate the Open Source Software compatible with MATLAB by applying the quality-in-use characteristics model (Alnanih, 2015) to obtain feedback from expert users using these tools as part of their regularly assigned duties.

The main contribution of this research paper is to validate the quality-in-use characteristics for the selected open source tools used by both non-expert and professional people in order to easily produce the best tool by evidence without compromising the data. In addition, our objective is to conduct a comparative study by measuring effectiveness, productivity, efficiency, error safety, and cognitive load for the tools mentioned above. In this paper, we apply Alnanih (2015) quality-in-use model, inspired by the ISO/IEC 25010 (2011) and adapted to the context of the open source software paradigm instead to healthcare applications, to assess the selected open source tools (see methodology section).

In this paper, we first present an inclusive set of quality models for assessing OSS in the literature review section. As discussed below, very little research has been carried out on quality in use for OSS. In section 3, we explain the methodology we have adopted for the application of the quality in use model on three mathematical and statistical OSS products. The experiments are detailed in section 4, and a discussion is presented in section 5. Conclusions and directions for future work are provided in section 6.

Complete Article List

Search this Journal:
Volume 15: 1 Issue (2024): Forthcoming, Available for Pre-Order
Volume 14: 1 Issue (2023)
Volume 13: 4 Issues (2022): 1 Released, 3 Forthcoming
Volume 12: 4 Issues (2021)
Volume 11: 4 Issues (2020)
Volume 10: 4 Issues (2019)
Volume 9: 4 Issues (2018)
Volume 8: 4 Issues (2017)
Volume 7: 4 Issues (2016)
Volume 6: 1 Issue (2015)
Volume 5: 3 Issues (2014)
Volume 4: 4 Issues (2012)
Volume 3: 4 Issues (2011)
Volume 2: 4 Issues (2010)
Volume 1: 4 Issues (2009)
View Complete Journal Contents Listing