Reliability Growth Models for Defect Prediction

Background

Several projects have demonstrated the validity and applicability of applying metrics to identify fault prone software at the code level (Khoshgoftaar & Allen, 1998; Khoshgoftaar, Allen, Halstead, & Trio, 1996a; Khoshgoftaar, Allen, Kalaichelvan, & Goel, 1996b; Schneidewind, 2000). This approach is applied at the requirements level to allow for early detection of reliability and maintainability problems. Once high-risk areas of the software have been identified, they would be subject to detailed tracking throughout the development and maintenance process (Schneidewind, 1999).

Much of the research and literature in software metrics concerns the measurement of code characteristics (Nikora, Schneidewind, & Munson, 1998). This is satisfactory for evaluating product quality and process effectiveness once the code is written. However, if organizations use measurement plans that are limited to measuring code, the plans will be deficient in the following ways: incomplete, lack coverage (e.g., no requirements analysis and design), and start too late in the process. For a measurement plan to be effective, it must start with requirements and continue through to operation and maintenance. Since requirements characteristics directly affect code characteristics and hence reliability, it is important to assess their impact on reliability when requirements are specified. As will be shown, it is feasible to quantify the risks to reliability of requirements changes—either new requirements or changes to existing requirements.

Once requirements attribute that portend high risk for the operational reliability of the software are identified, it is possible to suggest changes in the development process of the organization. To illustrate, a possible recommendation is that any requirements change to mission critical software—either new requirements or changes to existing requirements--would be subjected to a quantitative risk analysis. In addition to stating that a risk analysis would be performed, the policy would specify the risk factors to be analyzed (e.g., number of modifications of a requirement or mod level) and their threshold or critical values. The validity and applicability of identifying critical values of metrics to identify fault prone software at the code level have been demonstrated (Schneidewind, 2000). For example, on the space shuttle, rigorous inspections of requirements, design documentation, and code have contributed more to achieving high reliability than any other process factor. The objective of these policy changes is to prevent the propagation of high-risk requirements through the various phases of software development and maintenance. The payoff to the organization would be to reduce the risk of mission critical software not meeting its reliability goals during operation.