Two Experiments in Reducing Overconfidence in Spreadsheet Development

Two Experiments in Reducing Overconfidence in Spreadsheet Development

Raymond R. Panko (University of Hawai`i, USA)
DOI: 10.4018/978-1-60566-136-0.ch009
OnDemand PDF Download:


This chapter describes two experiments that examined overconfidence in spreadsheet development. Overconfidence has been seen widely in spreadsheet development and could account for the rarity of testing by end-user spreadsheet developers. The first experiment studied a new way of measuring overconfidence. It demonstrated that overconfidence really is strong among spreadsheet developers. The second experiment attempted to reduce overconfidence by telling subjects in the treatment group the percentage of students who made errors on the task in the past. This warning did reduce overconfidence, and it reduced errors somewhat, although not enough to make spreadsheet development safe.
Chapter Preview


Spreadsheet development was one of the earliest end-user applications, along with word processing. Spreadsheet development continues to be among the most widely used computer applications in organizations (United States Bureau of the Census, 2003). Although many spreadsheets are small and simple throwaway calculations, surveys have shown that many spreadsheets are quite large (Cale, 1994; Cragg & King, 1993; Floyd, Walls, & Marr, 1995; Hall, 1996), complex (Hall, 1996), and very important to the firm (Chan & Storey, 1996; Gable, Yap, & Eng, 1991).

Unfortunately, there is growing evidence that inaccurate spreadsheets are commonplace. For instance, Table 1 shows that recent audits of 88 real-world spreadsheets have found errors in 94%; yet several studies only reported spreadsheets with serious errors. The implications of this ubiquity of errors are sobering.

Table 1.
Studies of spreadsheet errors
StudyYearNumber of SpreadsheetsPercent of Spreadsheets Containing at Least One ErrorFormula Error Rate (FER): Percent of Cells Containing Errors
Field Audits
Coopers & Lybrand (c)19972391%
KPMG (b)19982291%
Lukasic19982100%2.2%, 2.5%
Clermont, Hanin, & Mittermeier (a)20023100%1.3%, 6.7%, 0.1%
Lawrence & Lee200430100%Average of 6.9%
Total/Per Spreadsheet8894%5.2%
Development Experiments
Brown & Gould19872763%
Olson & Nilsen (f,g)1987-19881421%
Lerch (f,g)1988219.3%
Hassinen (g)19889255%4.3%
Panko & Halverson19974279%5.6%
Panko & Halverson19973586%4.6%
Teo & Tan199716842%2.1%
Panko & Sprague (i)19982635%2.1%
Panko & Sprague (j)19981724%1.1%
Janvrin & Morrison (h)2000616.6%-9.6%
Janvrin & Morrison (h)20008.4%-16.8%
Kreie (posttest)20007342%2.5%

(a) Computed on basis of all non-empty cells instead of on the basis of formula cells. (b) Only spreadsheets with major errors were counted. (c) A dependent variable value was off by at least 5%. (d) Only errors large enough to demand additional tax payments were counted. (e) Only serious errors were counted. (f) Counted errors even if they were corrected by the developer. (g) CER is based only on formula cells. (h) CER was based only on high-risk formula cells. (i) MBA students with little or no development experience. (j) MBA students with at least 250 hours of spreadsheet development experience.

Source: Panko ( References to studies are given at the Web site.

Complete Chapter List

Search this Book:
Table of Contents
Steve Clarke
Chapter 1
Jeremy Fowler
Although the discipline of information systems (IS) development is well established, IS failure and abandonment remains widespread. As a result, a... Sample PDF
Information Systems Success and Failure—Two Sides of One Coin, or Different in Nature? An Exploratory Study
Chapter 2
Jeanette Eriksson, Yvonne Dittrich
This chapter reports on a case study performed in cooperation with a telecommunication provider. The telecom business changes rapidly as new... Sample PDF
Achieving Sustainable Tailorable Software Systems by Collaboration Between End-Users and Developers
Chapter 3
Marvin D. Troutt, Douglas A. Druckenmiller, William Acar
This chapter uses some special usability and ethical issues that arise from experience with what can be called captive end-user systems (CEUS).... Sample PDF
Usability, Testing, and Ethical Issues in Captive End-User Systems
Chapter 4
Jonathan P. Caulkins, Erica Layne Morrison, Timothy Weidemann
Spreadsheets are commonly used and commonly flawed, but it is not clear how often spreadsheet errors lead to bad decisions. We interviewed 45... Sample PDF
Do Spreadsheet Errors Lead to Bad Decisions? Perspectives of Executives and Senior Managers
Chapter 5
Lixuan Zhang, Randall Young, Victor Prybutok
The means by which the U.S. justice system attempts to control illegal hacking are practiced under the assumption that hacking is like any other... Sample PDF
A Comparison of the Inhibitors of Hacking vs. Shoplifting
Chapter 6
Dewi Rooslani Tojib
he last decade has seen the proliferation of business-to-employee (B2E) portals as integrated, efficient, and user-friendly technology platform to... Sample PDF
Developing Success Measure for Staff Portal Implementation
Chapter 7
Peter Baloh
Improving how knowledge is leveraged in organizations for improved business performance is currently considered as a major organizational change.... Sample PDF
Contingencies in the KMS Design: A Tentative Design Model
Chapter 8
Beryl Burns
We report the findings of a field study of the enactment of ICT supported knowledge work in a Human Resources contact centre, illustrating the... Sample PDF
Users as Developers: A Field Study of Call Centre Knowledge Work
Chapter 9
Raymond R. Panko
This chapter describes two experiments that examined overconfidence in spreadsheet development. Overconfidence has been seen widely in spreadsheet... Sample PDF
Two Experiments in Reducing Overconfidence in Spreadsheet Development
Chapter 10
Steven John Simon, David Paper
Voice recognition technology-enabled devices possess extraordinary growth potential, yet some research indicates that organizations and consumers... Sample PDF
User Acceptance of Voice Recognition Technology: An Empirical Extension of the Technology Acceptance Model
Chapter 11
Peter P. Mykytyn
Colleges of business have dealt with teaching computer literacy and advanced computer application concepts for many years, often with much... Sample PDF
Educating Our Students in Computer Application Concepts: A Case for Problem-Based Learning
Chapter 12
Elaine H. Ferneley
End user development (EUD) of system applications is typically undertaken by end users for their own, or closely aligned colleagues, business needs.... Sample PDF
Covert End User Development: A Study of Success
Chapter 13
Steven Hornik, Richard D. Johnson, Yu Wu
Central to the design of successful virtual learning initiatives is the matching of technology to the needs of the training environment. The... Sample PDF
When Technology Does Not Support Learning: Conflicts Between Epistemological Beliefs and Technology Support in Virtual Learning Environments
Chapter 14
Tom Butler
The study’s objective is to arrive at a theoretical model and framework to guide research into the implementation of KMS, while also seeking to... Sample PDF
A Theoretical Model and Framework for Understanding Knowledge Management System Implementation
Chapter 15
Jun Xu, Mohammed Quaddus
This chapter develops a model of adoption and continued use of knowledge management systems (KMSs), which is primarily built on Rogers’ (1995)... Sample PDF
Exploring the Factors Influencing End Users' Acceptance of Knowledge Management Systems: Development of a Research Model of Adoption and Continued Use
Chapter 16
Wei-Na Lee
In today’s global environment, a myriad of communication mechanisms enable cultures around the world to interact with one another and form complex... Sample PDF
Classifying Web Users: A Cultural Value-Based Approach
Chapter 17
Annette Hallin, Kristina Lundevall
This chapter presents the mCity Project, a project owned by the City of Stockholm, aiming at creating user-friendly mobile services in collaboration... Sample PDF
mCity: User Focused Development of Mobile Services Within the City of Stockholm
Chapter 18
Cristina Hava Muntean, Gabriel-Miro Muntean
Lately, user quality of experience (QoE) during their interaction with a system is a significant factor in the assessment of most systems. However... Sample PDF
End-User Quality of Experience-Aware Personalized E-Learning
Chapter 19
High-Tech Meets End-User  (pages 302-320)
Marc Steen
One challenge within the high-tech sector is to develop products that end users will actually need and will be able to use. One way of trying to... Sample PDF
High-Tech Meets End-User
About the Contributors