Abstract
This chapter describes a case study where Bayesian Networks (BNs) were used to construct an expert-based software effort and risk prediction model for use by a large healthcare organisation in Auckland (New Zealand) to manage healthcare software projects delivered on the Web. This model was solely elicited from expert knowledge, with the participation of seven project managers, and was validated using data from 22 past finished projects. The model led to numerous changes in process and also in business. The company adapted their existing effort and risk management process to be in line with the model that was created, and the use of a mathematically based model also led to an increase in the number of projects being outsourced to this company by other company branches worldwide. Their predictions improved significantly too. The results suggest that the use of a model that allows the representation of uncertainty, inherent in effort estimation, can outperform expert-based estimates.
TopIntroduction And Background
Healthcare projects, similarly to software projects in other domains, need to be managed effectively so to enable the corresponding applications to be delivered on time and within budget. However, for a project to be managed effectively, it is important that a realistic estimate of the amount of effort (in person hours) needed to develop a software application be obtained and used as basis to predict project costs and to allocate resources (e.g. developers). This estimate is generally derived taking into account the characteristics of the new project, and corresponding application, for which an estimate is needed, and also the characteristics of previous ‘similar’ projects, and corresponding applications, for which actual effort is known. The project and application characteristics employed herein are only those assumed to be relevant in determining effort.
The process of deriving an effort estimate is presented in Figure 1. The input to this process comprises the following:
Figure 1. Effort estimation process
- 1.
Data and/or knowledge on past finished projects, for which actual effort is known, represented by project and application characteristics (independent variables) believed to have an effect upon the amount of effort needed to accomplish a task/activity/process. Whenever a company does not have either data or experience on similar projects e their corresponding applications, these tend to be replaced by an “educated guess” based on prior experience with dissimilar projects.
- 2.
Data relating to the new project for which effort is to be estimated (estimated size and other factors), which is estimated based on the new application’s requirements (e.g. what functionality the application should offer to users). Such data also uses the same project characteristics employed in 1 above.
The output of this process is an effort estimate (dependent variable), which is then used to allocate resources, and to estimate project duration and costs.
The most widely used mechanism to derive an effort estimate is still expert judgement. However, despite the expertise of those involved in obtaining the estimate, the means of obtaining it are not explicit as the knowledge employed is only tacit. This means that the factors that are taken into account by the expert(s) are not known, and therefore repeating past successes becomes practically an impossible task. Given this situation, the research in this field started to investigate ways to model the relationship between effort and project & application characteristics. Numerous techniques have been used in order to build such models, such as statistical multivariate regression, case-based reasoning, classification and regression trees, neural networks, and Bayesian networks. Further details on each of these techniques are given in (Mendes & Mosley, 2008). Figure 2 shows this situation, where the estimation process itself includes two sub-processes, detailed below:
Figure 2. Effort estimation process with explicit modeling
- 1.
Effort Model Building: This sub-process represents the use of techniques to help with the construction of a tangible representation of the association between project & application characteristics and effort using data/knowledge from past finished projects for which actual effort is known. Such representation can take several forms, as abovementioned, e.g. an Equation, a binary tree, an acyclic graph. This sub-process is shown using a dashed line because in some instances no concrete model representation exists, as for example, when employing a technique such as case-based reasoning.
- 2.
Deriving an Effort Estimate: This sub-process represents either the use of the estimated characteristics of a new project & application as input to a concrete effort model that provides an effort estimate, or the use of the estimated characteristics of a new project as input to [the] sub-process [from step 1 above] that derives an effort estimate using previous knowledge from past projects.
Key Terms in this Chapter
Knowledge Elicitation: The process employed to elicit an expert’s tacit knowledge (expertise and experience) so to obtain a tangible representation of this knowledge.
Bayesian Networks: A Bayesian Network is a model that represents a domain knowledge using a directed acyclic graph structure (to model variables and their causal relationships), and also enables these causal relationships to be quantified using probabilities. These probabilities represent the uncertainty in the domain being modeled.
Estimated Software Size: A measure that characterizes the size of the problem that is to be solved by developing a software application. In general, in order to obtain this estimate one uses as input a requirements specification document that details what the application is supposed to do.
Knowledge Elicitation: The process employed to elicit an expert’s tacit knowledge (expertise and experience) so to obtain a tangible representation of this knowledge.
Bayesian Networks: A Bayesian Network is a model that represents a domain knowledge using a directed acyclic graph structure (to model variables and their causal relationships), and also enables these causal relationships to be quantified using probabilities. These probabilities represent the uncertainty in the domain being modeled.
Expert-Based Effort Estimation: An effort estimate that has been obtained by subjective means, only based on the tacit knowledge of experts, and optionally also some data from past finished projects.
Software Project Management: Process that is employed to plan and control the development and delivery of software applications.
Expert-Based Effort Estimation: An effort estimate that has been obtained by subjective means, only based on the tacit knowledge of experts, and optionally also some data from past finished projects.
Software Project Management: Process that is employed to plan and control the development and delivery of software applications.
Effort Estimation: The process by which the amount of effort (in person hours) needed to develop a software application is predicted, in order to be used as basis to predict project costs and to allocate resources (e.g. developers). This estimate is generally derived taking into account the characteristics of the new project, and corresponding application, for which an estimate is needed, and also the characteristics of previous ‘similar’ projects, and corresponding applications, for which actual effort is known.
Domain Uncertainty: Characterises the uncertainty that is inherent to certain domains, and that should be taken into account as part of a decision making process. For example, effort estimation is a very complex domain where the relationship between factors is non-deterministic and has an inherently uncertain nature. For example, assuming there is a relationship between development effort and an application’s size (e.g. number of Web pages, functionality), it is not necessarily true that increased effort will lead to larger size. However, as effort increases so does the probability of larger size.
Domain Uncertainty: Characterises the uncertainty that is inherent to certain domains, and that should be taken into account as part of a decision making process. For example, effort estimation is a very complex domain where the relationship between factors is non-deterministic and has an inherently uncertain nature. For example, assuming there is a relationship between development effort and an application’s size (e.g. number of Web pages, functionality), it is not necessarily true that increased effort will lead to larger size. However, as effort increases so does the probability of larger size.
Effort Estimation: The process by which the amount of effort (in person hours) needed to develop a software application is predicted, in order to be used as basis to predict project costs and to allocate resources (e.g. developers). This estimate is generally derived taking into account the characteristics of the new project, and corresponding application, for which an estimate is needed, and also the characteristics of previous ‘similar’ projects, and corresponding applications, for which actual effort is known.
Estimated Software Size: A measure that characterizes the size of the problem that is to be solved by developing a software application. In general, in order to obtain this estimate one uses as input a requirements specification document that details what the application is supposed to do.