The concurrent validity of a new eDiagnostic system for mental disorders in primary care

Abstract

Background: An eDiagnostic system was implemented to classify mental disorders, to support general practitioners.

Objective: Assessing the validity of the system, compared to the psychologists' judgment.

Methods: Concurrent validity, using routinely collected data of 675 primary care patients with a suspicion of a mental disorder in the Netherlands. Four psychologists classified the patients according to the DSM-IV. Hundred records were randomly selected to investigate the inter-rater reliability among psychologists. To investigate the concurrent validity of the system the sensitivity, specificity, positive predictive values (PPVs), negative predictive values (NPVs) and Cohen's κ-values (κ-values) were calculated.

Results: Inter-rater agreement between psychologists were fair to good or excellent. The system could correctly estimate the echelon (sensitivity range: 0.85-0.95, specificity range: 0.88-0.98) and correctly identify most Axis I classifications (sensitivity: 0.46-1.00, specificity: 0.75-0.99), except for Asperger's, sexual and adjustment disorders (sensitivity: 0.10-0.24, specificity: 0.97-0.99). It could determine the absence of a personality disorder (sensitivity: 0.81, specificity: 0.84, PPV: 0.77, NPV: 0.87 and κ-value: 0.65). The sensitivities and specificities for most specific personality disorders were good, but the PPVs for several specific Axis II classifications were low (PPV range: 0.06-0.77). The system was inaccurate in identifying the global assessment of functioning of patients (e.g. κ-values varied from 0.17-0.46).

Conclusions: Generally, the system can be seen as a valid instrument for most DSM-IV classifications in primary care patients. It could assist healthcare professionals in the assessment and classification of mental disorders. Future research should include comparison to an independently administered structured clinical interview.

Keywords: Diagnostic and Statistical Manual of Mental Disorders; general practitioners; mental health; primary health care; referral and consultation.; sensitivity and specificity.