"This behavior strikes us as ideal": assessment and anticipations of Huisman (2022)

Psychon Bull Rev. 2024 Feb;31(1):242-248. doi: 10.3758/s13423-023-02299-x. Epub 2023 Aug 1.

Abstract

Huisman (Psychonomic Bulletin & Review, 1-10. 2022) argued that a valid measure of evidence should indicate more support in favor of a true alternative hypothesis when sample size is large than when it is small. Bayes factors may violate this pattern and hence Huisman concluded that Bayes factors are invalid as a measure of evidence. In this brief comment we call attention to the following: (1) Huisman's purported anomaly is in fact dictated by probability theory; (2) Huisman's anomaly has been discussed and explained in the statistical literature since 1939; the anomaly was also highlighted in the Psychonomic Bulletin & Review article by Rouder et al. (2009), who interpreted the anomaly as "ideal": an interpretation diametrically opposed to that of Huisman. We conclude that when intuition clashes with probability theory, chances are that it is intuition that needs schooling.

Keywords: Bayes factors; Evidential strength; Hypothesis testing; Jeffreys-Lindley paradox; P-values.

Publication types

  • Review

MeSH terms

  • Bayes Theorem*
  • Humans
  • Probability
  • Sample Size