Do Spreadsheet Errors Lead to Bad Decisions? Perspectives of Executives and Senior Managers

Do Spreadsheet Errors Lead to Bad Decisions? Perspectives of Executives and Senior Managers

Jonathan P. Caulkins (Carnegie Mellon University, USA), Erica Layne Morrison (IBM Global Services, USA) and Timothy Weidemann (Fairweather Consulting, USA)
DOI: 10.4018/978-1-60566-136-0.ch004
OnDemand PDF Download:
$37.50

Abstract

Spreadsheets are commonly used and commonly flawed, but it is not clear how often spreadsheet errors lead to bad decisions. We interviewed 45 executives and senior managers/analysts in the private, public, and non-profit sectors about their experiences with spreadsheet quality control and with errors affecting decision making. Almost all of them said spreadsheet errors are common. Quality control was usually informal and applied to the analysis and/or decision, not just the spreadsheet per se. Most respondents could cite instances of errors directly leading to bad decisions, but opinions differ as to whether the consequences of spreadsheet errors are severe. Some thought any big errors would be so obvious as to be caught by even informal review. Others suggest that spreadsheets inform but do not make decisions, so errors do not necessarily lead one for one to bad decisions. Still, many respondents believed spreadsheet errors were a significant problem and that more formal spreadsheet quality control could be beneficial.
Chapter Preview
Top

Introduction

Spreadsheets are used in diverse domains by decision makers at all levels (Gerson, Chien, & Raval, 1992; Chan and Storey, 1996; Seal, Przasnyski, & Leon, 2000; Croll, 2005); the entire July-August, 2008 issue of the journal Interfaces is devoted to spreadsheet modeling success stories. However, laboratory studies and field audits consistently find that a large proportion of spreadsheets contain errors. In a dozen studies reviewed by Kruck, Maher, & Barkhi (2003), the average proportion of spreadsheets with errors was 46%. Panko’s (2000a, 2005) synthesis of spreadsheet audits published since 1995 suggested a rate of 94%. Powell, Baker, & Lawson (2007a, 2008b) critique past work and greatly advance methods of defining and measuring spreadsheet errors but at the end of the day reach the same overall error rate of 94%.

Hence, one might expect that (1) spreadsheet errors frequently lead to poor decisions and (2) organizations would invest heavily in quality control procedures governing spreadsheet creation and use. We investigated both hypotheses through 45 semi-structured interviews with executives and senior managers / analysts in the public, nonprofit, and private sectors. Field interviews raise fewer concerns about external validity than do laboratory studies, and they focus on the overall decision making process, not just the spreadsheet artifact as in audit studies. However, our approach has two important limitations. First, the respondents are a convenience sample. Second, self-report can be flawed, whether through imperfect memories, self-serving bias, conscious deception, and/or limited self-awareness. Given these limitations, we focus on broad qualitative conclusions. In brief, we found that most respondents could describe instances in which spreadsheet errors contributed to poor decisions, some with substantial consequences, yet few reported that their organization employs quality control procedures specific to spreadsheet analysis.

The literature on spreadsheet errors in general is large (see Panko, 2000b and Powell, Baker, & Lawson, 2008a for reviews), but much less has been written on these specific questions. Regarding the frequency with which spreadsheet errors lead to bad decisions, the European Spreadsheet Research Interest Group (EUSPRIG) maintains a webpage of news stories reporting the consequences of spreadsheet errors (http://www.eusprig.org/stories.htm). However, spreadsheets are used by so many organizations that even if only a small proportion were hurt badly by spreadsheet errors, there could still be scores of examples. We started with a population of individuals and organizations for which we had no a priori reason to think spreadsheet errors were a particular problem. This approach has been taken by others (e.g., Cragg and King, 1993) to explore the prevalence of defective spreadsheets, but like Powell, Baker, & Lawson (2007b), we shift the focus to assessing the impact of those spreadsheet errors.

A considerable corpus on controlling spreadsheet errors concerns what organizations should do. Classic recommendations lean toward application of good software engineering principles (Mather, 1999; Janvrin and Morrison, 2000; Rajalingham, Chadwick, & Knight, 2000; Grossman and Özlük, 2004) or formal theories (Isakowitz, Schocken, & Lucas, 1995). Kruck and Sheetz (2001) combed practitioner literature for practical axioms validated by empirical results, supporting aspects of the spreadsheet lifecycle theory (e.g., include planning / design and testing/debugging stages) and recommendations to decrease formula complexity.

There is also some literature describing what organizations actually do. Notably, Finlay and Wilson (2000) surveyed 10 academics and 10 practitioners on the factors influencing spreadsheet validation. Those most commonly mentioned were (a) aspects of the decision and (b) aspects of the spreadsheet underlying the decision context. However, Grossman (2002) argues that it would be valuable to have greater knowledge of what error control methods are currently used.

Complete Chapter List

Search this Book:
Reset
Table of Contents
Preface
Steve Clarke
Chapter 1
Jeremy Fowler
Although the discipline of information systems (IS) development is well established, IS failure and abandonment remains widespread. As a result, a... Sample PDF
Information Systems Success and Failure—Two Sides of One Coin, or Different in Nature? An Exploratory Study
$37.50
Chapter 2
Jeanette Eriksson, Yvonne Dittrich
This chapter reports on a case study performed in cooperation with a telecommunication provider. The telecom business changes rapidly as new... Sample PDF
Achieving Sustainable Tailorable Software Systems by Collaboration Between End-Users and Developers
$37.50
Chapter 3
Marvin D. Troutt, Douglas A. Druckenmiller, William Acar
This chapter uses some special usability and ethical issues that arise from experience with what can be called captive end-user systems (CEUS).... Sample PDF
Usability, Testing, and Ethical Issues in Captive End-User Systems
$37.50
Chapter 4
Jonathan P. Caulkins, Erica Layne Morrison, Timothy Weidemann
Spreadsheets are commonly used and commonly flawed, but it is not clear how often spreadsheet errors lead to bad decisions. We interviewed 45... Sample PDF
Do Spreadsheet Errors Lead to Bad Decisions? Perspectives of Executives and Senior Managers
$37.50
Chapter 5
Lixuan Zhang, Randall Young, Victor Prybutok
The means by which the U.S. justice system attempts to control illegal hacking are practiced under the assumption that hacking is like any other... Sample PDF
A Comparison of the Inhibitors of Hacking vs. Shoplifting
$37.50
Chapter 6
Dewi Rooslani Tojib
he last decade has seen the proliferation of business-to-employee (B2E) portals as integrated, efficient, and user-friendly technology platform to... Sample PDF
Developing Success Measure for Staff Portal Implementation
$37.50
Chapter 7
Peter Baloh
Improving how knowledge is leveraged in organizations for improved business performance is currently considered as a major organizational change.... Sample PDF
Contingencies in the KMS Design: A Tentative Design Model
$37.50
Chapter 8
Beryl Burns
We report the findings of a field study of the enactment of ICT supported knowledge work in a Human Resources contact centre, illustrating the... Sample PDF
Users as Developers: A Field Study of Call Centre Knowledge Work
$37.50
Chapter 9
Raymond R. Panko
This chapter describes two experiments that examined overconfidence in spreadsheet development. Overconfidence has been seen widely in spreadsheet... Sample PDF
Two Experiments in Reducing Overconfidence in Spreadsheet Development
$37.50
Chapter 10
Steven John Simon, David Paper
Voice recognition technology-enabled devices possess extraordinary growth potential, yet some research indicates that organizations and consumers... Sample PDF
User Acceptance of Voice Recognition Technology: An Empirical Extension of the Technology Acceptance Model
$37.50
Chapter 11
Peter P. Mykytyn
Colleges of business have dealt with teaching computer literacy and advanced computer application concepts for many years, often with much... Sample PDF
Educating Our Students in Computer Application Concepts: A Case for Problem-Based Learning
$37.50
Chapter 12
Elaine H. Ferneley
End user development (EUD) of system applications is typically undertaken by end users for their own, or closely aligned colleagues, business needs.... Sample PDF
Covert End User Development: A Study of Success
$37.50
Chapter 13
Steven Hornik, Richard D. Johnson, Yu Wu
Central to the design of successful virtual learning initiatives is the matching of technology to the needs of the training environment. The... Sample PDF
When Technology Does Not Support Learning: Conflicts Between Epistemological Beliefs and Technology Support in Virtual Learning Environments
$37.50
Chapter 14
Tom Butler
The study’s objective is to arrive at a theoretical model and framework to guide research into the implementation of KMS, while also seeking to... Sample PDF
A Theoretical Model and Framework for Understanding Knowledge Management System Implementation
$37.50
Chapter 15
Jun Xu, Mohammed Quaddus
This chapter develops a model of adoption and continued use of knowledge management systems (KMSs), which is primarily built on Rogers’ (1995)... Sample PDF
Exploring the Factors Influencing End Users' Acceptance of Knowledge Management Systems: Development of a Research Model of Adoption and Continued Use
$37.50
Chapter 16
Wei-Na Lee
In today’s global environment, a myriad of communication mechanisms enable cultures around the world to interact with one another and form complex... Sample PDF
Classifying Web Users: A Cultural Value-Based Approach
$37.50
Chapter 17
Annette Hallin, Kristina Lundevall
This chapter presents the mCity Project, a project owned by the City of Stockholm, aiming at creating user-friendly mobile services in collaboration... Sample PDF
mCity: User Focused Development of Mobile Services Within the City of Stockholm
$37.50
Chapter 18
Cristina Hava Muntean, Gabriel-Miro Muntean
Lately, user quality of experience (QoE) during their interaction with a system is a significant factor in the assessment of most systems. However... Sample PDF
End-User Quality of Experience-Aware Personalized E-Learning
$37.50
Chapter 19
High-Tech Meets End-User  (pages 302-320)
Marc Steen
One challenge within the high-tech sector is to develop products that end users will actually need and will be able to use. One way of trying to... Sample PDF
High-Tech Meets End-User
$37.50
About the Contributors