Validation of an ICD-Code-Based Case Definition for Psychotic Illness Across Three Health Systems

Anthony J Deo; Victor M Castro; Ashley Baker; Devon Carroll; Joseph Gonzalez-Heydrich; David C Henderson; Daphne J Holt; Kimberly Hook; Rakesh Karmacharya; Joshua L Roffman; Emily M Madsen; Eugene Song; William G Adams; Luisa Camacho; Sarah Gasman; Jada S Gibbs; Rebecca G Fortgang; Chris J Kennedy; Galina Lozinski; Daisy C Perez; Marina Wilson; Ben Y Reis; Jordan W Smoller

doi:10.1093/schbul/sbae064

Validation of an ICD-Code-Based Case Definition for Psychotic Illness Across Three Health Systems

Schizophr Bull. 2024 May 10:sbae064. doi: 10.1093/schbul/sbae064. Online ahead of print.

Authors

Anthony J Deo^{1

2

3

4}, Victor M Castro⁵, Ashley Baker⁶, Devon Carroll^{1

7}, Joseph Gonzalez-Heydrich^{1

2

8

9}, David C Henderson^{10

11}, Daphne J Holt^{2

12}, Kimberly Hook¹³, Rakesh Karmacharya^{2

14

15

16}, Joshua L Roffman^{2

12}, Emily M Madsen^{17

18}, Eugene Song¹⁷, William G Adams¹⁰, Luisa Camacho¹⁰, Sarah Gasman¹⁰, Jada S Gibbs¹⁹, Rebecca G Fortgang^{18

20}, Chris J Kennedy^{2

18}, Galina Lozinski¹⁰, Daisy C Perez^{10

11}, Marina Wilson^{17

18}, Ben Y Reis^{21

22}, Jordan W Smoller^{2

17

18

23}

Affiliations

¹ Department of Psychiatry and Behavioral Sciences, Boston Children's Hospital, Harvard Medical School, Boston, MA, USA.
² Department of Psychiatry, Harvard Medical School, Boston, MA, USA.
³ Department of Psychiatry, Rutgers-Robert Wood Johnson Medical School, Piscataway, NJ, USA.
⁴ Psychiatric Evaluation of Adolescent and Child Experiences (P.E.A.C.E.) Program, Rutgers University Behavioral Health Care, Piscataway, NJ, USA.
⁵ Research Information Science and Computing, Mass General Brigham, Somerville, MA, USA.
⁶ Ascend Integrative Medicine LLC, MA, USA.
⁷ College of Nursing, University of Rhode Island, Providence, RI, USA.
⁸ Tommy Fuss Center for Neuropsychiatric Disease Research, Boston Children's Hospital, Harvard Medical School, Boston, MA, USA.
⁹ Early Psychosis Investigation Center, Boston Children's Hospital, Harvard Medical School, Boston, MA, USA.
¹⁰ Boston Medical Center, Boston, MA, USA.
¹¹ Boston University Chobanian & Avedisian School of Medicine, Boston, MA, USA.
¹² Department of Psychiatry, Massachusetts General Hospital, Boston MA, USA.
¹³ Harvard T.H. Chan School of Public Health, Harvard University, Boston, MA, USA.
¹⁴ Center for Genomic Medicine, Massachusetts General Hospital, Boston, MA, USA.
¹⁵ Chemical Biology and Therapeutic Science Program, Broad Institute of MIT and Harvard, Cambridge, MA, USA.
¹⁶ Schizophrenia and Bipolar Disorder Program, McLean Hospital, Belmont, MA, USA.
¹⁷ Psychiatric & Neurodevelopmental Genetics Unit, Center for Genomic Medicine, Massachusetts General Hospital, Boston, MA, USA.
¹⁸ Center for Precision Psychiatry, Department of Psychiatry, Massachusetts General Hospital, Boston, MA, USA.
¹⁹ Rutgers New Jersey Medical School, Newark, NJ, USA.
²⁰ Department of Psychology, Harvard University, Cambridge, MA, USA.
²¹ Predictive Medicine Group, Harvard Medical School, Boston, MA, USA.
²² Computational Health Informatics Program, Boston Children's Hospital, Boston, MA, USA.
²³ Stanley Center for Psychiatric Research, Broad Institute, Cambridge, MA, USA.

PMID: 38728421
DOI: 10.1093/schbul/sbae064

Abstract

Background and hypothesis: Psychosis-associated diagnostic codes are increasingly being utilized as case definitions for electronic health record (EHR)-based algorithms to predict and detect psychosis. However, data on the validity of psychosis-related diagnostic codes is limited. We evaluated the positive predictive value (PPV) of International Classification of Diseases (ICD) codes for psychosis.

Study design: Using EHRs at 3 health systems, ICD codes comprising primary psychotic disorders and mood disorders with psychosis were grouped into 5 higher-order groups. 1133 records were sampled for chart review using the full EHR. PPVs (the probability of chart-confirmed psychosis given ICD psychosis codes) were calculated across multiple treatment settings.

Study results: PPVs across all diagnostic groups and hospital systems exceeded 70%: Mass General Brigham 0.72 [95% CI 0.68-0.77], Boston Children's Hospital 0.80 [0.75-0.84], and Boston Medical Center 0.83 [0.79-0.86]. Schizoaffective disorder PPVs were consistently the highest across sites (0.80-0.92) and major depressive disorder with psychosis were the most variable (0.57-0.79). To determine if the first documented code captured first-episode psychosis (FEP), we excluded cases with prior chart evidence of a diagnosis of or treatment for a psychotic illness, yielding substantially lower PPVs (0.08-0.62).

Conclusions: We found that the first documented psychosis diagnostic code accurately captured true episodes of psychosis but was a poor index of FEP. These data have important implications for the case definitions used in the development of risk prediction models designed to predict or detect undiagnosed psychosis.

Keywords: bipolar; detection; prediction; schizoaffective; schizophrenia.

© The Author(s) 2024. Published by Oxford University Press on behalf of the Maryland Psychiatric Research Center. All rights reserved. For commercial re-use, please contact reprints@oup.com for reprints and translation rights for reprints. All other permissions can be obtained through our RightsLink service via the Permissions link on the article page on our site—for further information please contact journals.permissions@oup.com.

Abstract

Grants and funding