Knowledge Calibration and Knowledge Management

Background

A variety of scientists, including meteorologists, statisticians, and psychologists, have been interested in measuring and in explaining judgments of confidence and their relation to accuracy (e.g., Harvey, 1997; Yates, 1990). Most of these studies report that people are systematically overconfident about the accuracy of their knowledge and judgment. In fact, scholars have even considered overconfidence as a stylized fact of human cognition.

The construct “calibration of knowledge” refers to the correspondence between accuracy of knowledge and confidence in knowledge (refer to Table 1). High accuracy and high confidence in knowledge promote high calibration; confidence in these decisions is justified. Low accuracy and low confidence also promote high calibration. In this case, decision makers are aware of their ignorance and are unlikely to overreach. A lack of correspondence between accuracy and confidence means miscalibration. Miscalibrated individuals are either overconfident or underconfident: situations that can result in costly mistakes in decision making.

For example, a description of the difficulties XEROX had in successfully bringing their new inventions to market (Carayannis, Gonzalez, & Wetter, 2003) reveals that, among other problems, managers placed great faith in their knowledge of the market, technology, and future trends that was subsequently proved to be misplaced. One could argue that the Bush administration's decision to go to war with Iraq in order to destroy weapons of mass destruction that did not exist, but were claimed to exist on the basis of high confidence in flimsy evidence, is also an example of miscalibration and its influence on decision making.