Paper Rejected (p>0.05): An Introduction to the Debate on Appropriateness of Null-Hypothesis Testing

Paper Rejected (p>0.05): An Introduction to the Debate on Appropriateness of Null-Hypothesis Testing

Mark. D. Dunlop (University of Strathclyde, UK) and Mark Baillie (University of Strathclyde, UK)
DOI: 10.4018/978-1-60960-499-8.ch022

Abstract

Null-hypothesis statistical testing has been seriously criticised in other domains, to the extent of some advocating a complete ban on publishing p-values. This short position paper aims to introduce the argument to the mobile-HCI research community, who make extensive use of the controversial testing methods.
Chapter Preview
Top

Key Problems With P-Based Statistics

The debate on null-hypothesis testing has identified many “sins” of null-hypothesis significance testing (NHST) and the way that it is normally used in scientific work. Here we look at them as we perceive the severity of the problem in mobile-HCI:

  • 1.

    Treating NHT as a binary approval of result validity;

  • 2.

    Confusing strength of p-value results with effect size;

  • 3.

    Abusing the statistical tests themselves;

  • 4.

    Making conclusions from non-significant results;

  • 5.

    Making illogical arguments based on results.

Reviewing recent proceedings of MobileHCI, we are not as guilty as other domains in which null-hypothesis testing has been criticised. However, we tend to be guilty of the first three sins quite widely and we perceive a risk that as publication becomes more competitive, reviewers might push us further along the route of inappropriate statistics.

Complete Chapter List

Search this Book:
Reset