Risk prediction using natural language processing of electronic mental health records in an inpatient forensic psychiatry setting

Abstract

Objective: Instruments rating risk of harm to self and others are widely used in inpatient forensic psychiatry settings. A potential alternate or supplementary means of risk prediction is from the automated analysis of case notes in Electronic Health Records (EHRs) using Natural Language Processing (NLP). This exploratory study rated presence or absence and frequency of words in a forensic EHR dataset, comparing four reference dictionaries. Seven machine learning algorithms and different time periods of EHR analysis were used to probe which dictionary and which time period were most predictive of risk assessment scores on validated instruments.

Materials and methods: The EHR dataset comprised de-identified forensic inpatient notes from the Wilfred Lopes Centre in Tasmania. The data comprised unstructured free-text case note entries and serial ratings of three risk assessment scales: Historical Clinical Risk Management-20 (HCR-20), Short-Term Assessment of Risk and Treatability (START) and Dynamic Appraisal of Situational Aggression (DASA). Four NLP dictionary word lists were selected: 6865 mental health symptom words from the Unified Medical Language System (UMLS), 455 DSM-IV diagnoses from UMLS repository, 6790 English positive and negative sentiment words, and 1837 high frequency words from the Corpus of Contemporary American English (COCA). Seven machine learning methods Bagging, J48, Jrip, Logistic Model Trees (LMT), Logistic Regression, Linear Regression and Support Vector Machine (SVM) were used to identify the combination of dictionaries and algorithms that best predicted risk assessment scores.

Results: The most accurate prediction was attained on the DASA dataset using the sentiment dictionary and the LMT and SVM algorithms.

Conclusions: NLP, used in conjunction with NLP dictionaries and machine learning, predicted risk ratings on the HCR-20, START, and DASA, based on EHR content. Further research is required to ascertain the utility of NLP approaches in predicting endpoints of actual self-harm, harm to others or victimisation.

Keywords: Electronic health record; Mental health; Natural language processing; Psychiatry; Text mining.