Evaluating Open Source Software through Prototyping

Evaluating Open Source Software through Prototyping

Ralf Carbon (Fraunhofer Institute for Experimental Software Engineering (IESE), Germany) and Marcus Ciolkowski (Software Engineering: Processes and Measurement Research Group, Germany)
DOI: 10.4018/978-1-59140-999-1.ch021
OnDemand PDF Download:


The increasing number of high quality open source software (OSS) components lets industrial organizations seriously consider integrating them into their software solutions for critical business cases. But thorough considerations have to be undertaken to choose the “right” OSS component for a specific business case. OSS components need to fulfill specific functional and non-functional requirements, must fit into a planned architecture, and must comply with context factors in a specific environment. This chapter introduces a prototyping approach to evaluate OSS components. The prototyping approach provides decision makers with context-specific evaluation results and a prototype for demonstration purposes. The approach can be used by industrial organizations to decide on the feasibility of OSS components in their concrete business cases. We present one of the industrial case studies we conducted in a practical course at the University of Kaiserslautern to demonstrate the application of our approach in practice. This case study shows that even inexperienced developers like students can produce valuable evaluation results for an industrial customer that wants to use open source components.

Key Terms in this Chapter

Case Study: An observational empirical study, which is done by observation of an on-going project or activity. A case study typically monitors a project or assignment. Case studies are normally aimed at tracking a specific attribute or establishing relationships between different attributes.

Empirical Evaluation: A study where the research ends are based on evidence and not just theory. This is done to comply with the scientific method that asserts the objective discovery of knowledge based on verifiable facts of evidence. This includes observing a phenomenon under laboratory conditions (e.g., in a controlled experiment) or in the field (e.g., in a case study).

Context (Factor): The context is the environment in which an empirical study is run. Context factors are influences in the context (such as the experience of developers) that may have an impact on the phenomenon under observation.

Goal-Question-Metric Paradigm (GQM): The goal-question-metric (GQM) paradigm has been proposed to support the definition of quantifiable goals and the interpretation of collected measurement data. It is a goal-oriented approach to derive metrics from measurement goals to ensure that collected data is usable and serves a purpose.

Quality Improvement Paradigm (QIP): The quality improvement paradigm is a general improvement scheme tailored for the software business. It is a goal-driven feedback-oriented improvement paradigm for software engineering based on total quality management principles, and on the plan/do/check/act cycle ( Demming, 1986 ).

Open Source Software (OSS): Software whose code is developed collaboratively, and is freely available to the public under specific license conditions.

Iterative Software Development: Denotes a software development process that splits system development into several parts (or iterations). The basic idea behind iterative development is to develop a software system incrementally, allowing the developer to take advantage of what was being learned during the development of earlier, incremental, deliverable versions of the system.

Prototyping: Denotes the process of quickly putting together a working model (a prototype) in order to test various aspects of a design, illustrate ideas or features and gather early user feedback. Prototyping is often treated as an integral part of the system design process, where it is believed to reduce project risk and cost. Its characteristic is that prototypes are typically developed without adhering to software engineering principles, which typically results in products that are not maintainable.

Complete Chapter List

Search this Book: