Telling the Quants from the Quacks: Evaluating Statistical Arguments in Debates Online

Introduction

Numbers have been described many times as a universal language. With their ubiquity and precision, the use of numbers allows information to travel quickly through different communities. However, numerical communications rely on taken-for-granted structures of practice that give numbers their credibility and legitimacy. Traditional validity strategies, the strategies that demonstrate research is credible and empirically grounded, include triangulation (the use of multiple types of evidence), a basis in previous research, and peer-review (Creswell, 2009, p. 191). For research shared online, however, there are frequently no guarantees for the provenance of information being shared, and the speed at which information is generated has increased ten-fold, leaving less time for review and correction (Terras & Ramsay, 2012). As a result, questionable information is shared by readers when it is expressed numerically and therefore appears credible. There are also researchers who have done the hard work, performing time-consuming research, who run the risk of their work being ignored if they do not share it online in a way that attracts the attention of readers. Telling the “quants,” those with legitimate claims to expertise and knowledge in data, from the “quacks,” those who make claims that are not supported by research or experience, has become incredibly difficult with computer-mediated communication.

This chapter addresses new challenges for authors in establishing authority—and for readers in determining credibility—when quantification and statistical methods are used in online debates. Whether the statistical arguments are in the form of original results or paraphrased from other primary sources, the stakes are high. With the Internet being a facilitator for education, communication, and political argument, it is crucial that authors have the tools to share their work persuasively and for readers to have the literacy necessary to evaluate research critically. The online debate over the credibility of statistical methods in climate change is one crucial case: Each side appears to use reliable rhetoric and evidence to justify their position. How are lay audiences to know which side is correct? How should scientists explain their work online so that it is perceived as authoritative? To answer these questions, this chapter introduces a heuristic developed from the controversial debate centered on The Bell Curve, Herrnstein and Murray’s (1994)New York Times bestseller. This heuristic is based on the critical responses to Herrnstein and Murray’s research and is adapted to the exigencies of online communication. The method is demonstrated using the popular climate change websites Watts Up With That? The World’s Most Viewed Site on Global Warming and Climate Change and Skeptical Science: Getting Skeptical about Global Warming Skepticism. The proposed heuristic helps readers sort the “quants” from the “quacks” while simultaneously providing a guide to authors for gaining the attention and respect of their online audience.