False alarm? A comprehensive reanalysis of "Evidence that psychopathology symptom networks have limited replicability" by Forbes, Wright, Markon, and Krueger (2017)

Abstract

Forbes, Wright, Markon, and Krueger (2017) stated that "psychopathology networks have limited replicability" (p. 1011) and that "popular network analysis methods produce unreliable results" (p. 1011). These conclusions are based on an assessment of the replicability of four different network models for symptoms of major depression and generalized anxiety across two samples; in addition, Forbes et al. analyzed the stability of the network models within the samples using split-halves. Our reanalysis of the same data with the same methods led to results directly opposed to theirs: All network models replicated very well across the two data sets and across the split-halves. We trace the differences between Forbes et al.'s results and our own to the fact that they did not appear to accurately implement all network models and used debatable metrics to assess replicability. In particular, they deviated from existing estimation routines for relative importance networks, did not acknowledge the fact that the skip structure used in the interviews strongly distorted correlations between symptoms, and incorrectly assumed that network structures and metrics should be the same not only across the different samples but also across the different network models used. In addition to a comprehensive reanalysis of the data, we end with a discussion of best practices concerning future research into the replicability of psychometric networks. (PsycINFO Database Record