“Open source software” implies more than just freely distributing source code. A generally accepted definition for open source software relies on 10 criteria (Coar, 2006):
The software must be freely distributed without fee
The source code must be freely accessible
The software license must allow modifications and derived works
The license may prohibit derived works from using the author’s name so there is no confusion of authorship
There must be no discrimination of use against persons or groups
The license cannot limit the use of the software to a particular field or business or industry
The license must be freely distributed along with the software
The license must be contingent on the software being distributed in a particular product or package
The license can make no claims about other software that may be distributed along with licensed software
The license cannot dictate that only a particular technology be used with it
Open source software is widely used by government, businesses, and non-profits alike because of the financial benefits. However, even though the software is “free” it is critical to select the right open source software solution for a given situation.
It can be difficult to evaluate any type of software for fitness of purpose, but open source software presents unique challenges and advantages. One of the unique challenges is the sheer number of open source projects. Anyone can create an open source project on free hosting sites such as SourceForge.net and many of these projects are very immature (Polancic, Horvat, & Rozman, 2004). Another challenge is that open source software rarely provides documentation (Wheeler, 2009). Without the documentation and marketing materials that traditionally accompany commercial software, it can be difficult to determine the features of an open source software product.
Balancing the challenges of evaluating open source software are significant advantages, mainly that the executable and source codes are freely available. Another advantage is that many open source projects provide public read-only access to their problem tracking system, which can give valuable insight into how fast the project is growing whether defects are being found and fixed, and the amount of time it takes to add new features.
There are at least four open source software evaluation models. Capgemini (2003) created the Open Source Maturity Model (OSMM) employing 12 weighted criteria, which can be input into a spreadsheet for evaluation. Navica (2004) created the Open Source Maturity Model (OSMM), which assesses six key software, and contains spreadsheets to assist in the evaluation (Golden, 2004). It is coincidental that both the Capgemini and Navica models use the same abbreviations.
The Qualification and Selection of Open Source software method (QSOS) was created in 2004 and sponsored by the consulting company Atos Origin. QSOS contains a set of evaluation criteria, and provides web-based tools to assist in the evaluation process. The website also contains a database of prior evaluations (QSOS, 2004).
The Business Readiness Rating (BRR) was created in 2005 and sponsored by Carnegie Mellon West, SpikeSource, O’Reilly, and Intel. The BRR provides evaluation criteria and spreadsheets for evaluation as well (BRR, 2007).
Even though it is not a formal evaluation model, another important work is Wheeler’s (2008) “How to Evaluate Open Source Software/Free Software (OSS/FS) Programs.”