Quantifying Unknown Unknowns in an Oil and Gas Capital Project

Introduction

Recent empirical data show the project failure rate remains high despite the well-known 1995 'Chaos' benchmarks (http://www.projectsmart.co.uk/docs/chaos-report.pdf) which warned project sponsors and practitioners about the lack of project success. The Independent Project Analysis Institute (IPAI, 2009) of Virginia USA reported that the project failure rate remains high: a staggering 56% of major projects fail. IPAI determined the major reasons for failure were due to:

• ▪

  Budget overspending for more than 25%, or

• ▪

  Schedule slipping for more than 25%, or

• ▪

  Severe continuing operational problems remaining unresolved for at least one year.

One of the top reasons for the failures is inadequate or inconsistent application of proven project risk management methods. In other words, project scope, cost or schedule development cannot be considered completed or reliable until consistent project risk analysis is carried out. This requires development of adequate project contingencies. Also a major problem with projects is the lack of risk estimation for the known unknowns and the unknown unknowns.

The terms 'known unknowns' and 'unknown unknowns' sound mysterious. As explained above, known unknowns are uncertainties (risks) that may have occurred in past projects - so project managers are aware of their existence - but they probability is completely unknown on a particular project. Known unknown uncertainty is generally anticipated by the probability value and time of occurrence is not apparent.

The unknown unknown uncertainties are those that are not anticipated, such as low chance probabilities, natural disasters. These would not be known even from previous projects (otherwise the project manager would classify them as known unknowns). Bias is a systematic type of uncertainty associated with human error or perception distortion. The Program Evaluation and Review Technique (PERT) developed by the US government is a method to assist in quantifying bias by calculating the standard deviations and probabilities after estimate the duration of project activities as pessimistic times, most likely times, and optimistic times. PERT is beyond the scope of this study but is mentioned because the probabilities could be used.

Several publications point to the importance of taking unknown unknowns into account for risk management (Chapman & Ward, 2003; Hubbard, 2009; Wideman, 1992). Missing or inadequately taken into account unknown unknowns can lead to non-adequate contingencies. Contingencies are buffers that are set aside in the project schedule, or in the project budget, as a reserve in case the unexpected does occur. The literature is clear in that it is the unknown unknown uncertainty which gives rise to the high rate of project failures mentioned, although other relevant risk factors would also be at play across the various industries.

This study builds on the earlier work by the author to develop a methodology for identifying and quantifying the 'unknown unknowns' in O&G industry capital projects (Raydugin, 2011). This paper shows how to quantify unknown-unknown uncertainties using probabilistic cost and schedule models from a case study. A literature review and experience are used to create a taxonomy for classifying the unknown-unknown factors. Brainstorming and heuristics are used to quantify uncertainty, and then the estimates are formulated into radar diagrams to assist in project selection, communication, and project management.

Literature Review

Oil-gas exploration and development projects have different phases and activities as compared to projects in other industries. Nonetheless, project management is a cross-disciplinary science that includes generally accepted techniques, including risk management, intended for any industry. In this industry, risk management is primarily focused on cost and schedule uncertainty.