In order to continue to make progress in software measurement as it pertains to reliability, there must be a shift in emphasis from design and code metrics to metrics that characterize the risk of making requirements changes. By doing so, the quality of delivered software can be improved because defects related to problems in requirements specifi- cations will be identified early in the life cycle. An approach is described for identifying requirements change risk factors as predictors of reliability problems. This approach can be generalized to other applications with numerical results that would vary according to application.
Several projects have demonstrated the validity and applicability of applying metrics to identify fault prone software at the code level (Khoshgoftaar & Allen, 1998; Khoshgoftaar, Allen, Halstead, & Trio, 1996a; Khoshgoftaar, Allen, Kalaichelvan, & Goel, 1996b; Schneidewind, 2000). This approach is applied at the requirements level to allow for early detection of reliability and maintainability problems. Once high-risk areas of the software have been identified, they would be subject to detailed tracking throughout the development and maintenance process (Schneidewind, 1999).
Much of the research and literature in software metrics concerns the measurement of code characteristics (Nikora, Schneidewind, & Munson, 1998). This is satisfactory for evaluating product quality and process effectiveness once the code is written. However, if organizations use measurement plans that are limited to measuring code, the plans will be deficient in the following ways: incomplete, lack coverage (e.g., no requirements analysis and design), and start too late in the process. For a measurement plan to be effective, it must start with requirements and continue through to operation and maintenance. Since requirements characteristics directly affect code characteristics and hence reliability, it is important to assess their impact on reliability when requirements are specified. As will be shown, it is feasible to quantify the risks to reliability of requirements changes—either new requirements or changes to existing requirements.
Once requirements attribute that portend high risk for the operational reliability of the software are identified, it is possible to suggest changes in the development process of the organization. To illustrate, a possible recommendation is that any requirements change to mission critical software—either new requirements or changes to existing requirements--would be subjected to a quantitative risk analysis. In addition to stating that a risk analysis would be performed, the policy would specify the risk factors to be analyzed (e.g., number of modifications of a requirement or mod level) and their threshold or critical values. The validity and applicability of identifying critical values of metrics to identify fault prone software at the code level have been demonstrated (Schneidewind, 2000). For example, on the space shuttle, rigorous inspections of requirements, design documentation, and code have contributed more to achieving high reliability than any other process factor. The objective of these policy changes is to prevent the propagation of high-risk requirements through the various phases of software development and maintenance. The payoff to the organization would be to reduce the risk of mission critical software not meeting its reliability goals during operation.
Dir:Cumulative defects (e.g., discrepancy reports: program executes incorrect path due to excessive requirements conflicts), corresponding to risk variable r during operating time interval i.
ri: Requirements risk (e.g., risk of excessive size, excessive conflicting issues)
rmin: Minimum value of ri
rmax: maximum value of ri
Sloc:Cumulative source lines of code (sloc): number of source lines of code written for a Shuttle release.
Issues:Cumulative number of possible conflicts among requirements (e.g., an aggregation of requirements to provide greater Shuttle thrust conflicts with the fact that more thrust requires more engine weight). An aggregation of conflicting requirements issues causes reliability risk.
t: Operating time (e.g., execution, wall clock, calendar time)
tc: Critical value of t = maximum value of t
i: Operating time interval
a and b: Coefficients of Dir obtained through regression analysis
Decision maker: Software quality control manager
Key Terms in this Chapter
Risk: The chance of injury, damage, or loss.
Reliability: The ability of a system or component to perform its required functions under stated conditions for a specified period of time.
Metric: A quantitative measure of the degree to which a system, component, or process possesses a given attribute.
Requirement: A condition or capability needed by a user to solve a problem or achieve an objective.
Quality: The degree to which a system, component, or process meets specified requirements.