Bayesian Estimation of Single-Test Reliability Coefficients

Multivariate Behav Res. 2022 Jul-Aug;57(4):620-641. doi: 10.1080/00273171.2021.1891855. Epub 2021 Mar 24.

Abstract

Popular measures of reliability for a single-test administration include coefficient α, coefficient λ2, the greatest lower bound (glb), and coefficient ω. First, we show how these measures can be easily estimated within a Bayesian framework. Specifically, the posterior distribution for these measures can be obtained through Gibbs sampling - for coefficients α, λ2, and the glb one can sample the covariance matrix from an inverse Wishart distribution; for coefficient ω one samples the conditional posterior distributions from a single-factor CFA-model. Simulations show that - under relatively uninformative priors - the 95% Bayesian credible intervals are highly similar to the 95% frequentist bootstrap confidence intervals. In addition, the posterior distribution can be used to address practically relevant questions, such as "what is the probability that the reliability of this test is between .70 and .90?", or, "how likely is it that the reliability of this test is higher than .80?" In general, the use of a posterior distribution highlights the inherent uncertainty with respect to the estimation of reliability measures.

Keywords: Bayesian reliability estimation; Cronbach’s alpha; Guttman’s lambda-2; McDonald’s omega; greatest lower bound; inverse Wishart distribution.

MeSH terms

  • Bayes Theorem*
  • Probability
  • Reproducibility of Results
  • Uncertainty