How nonrecidivism affects predictive accuracy: evidence from a cross-validation of the Ontario Domestic Assault Risk Assessment (ODARA)

J Interpers Violence. 2009 Feb;24(2):326-37. doi: 10.1177/0886260508316478. Epub 2008 Apr 7.

Abstract

Prediction effect sizes such as ROC area are important for demonstrating a risk assessment's generalizability and utility. How a study defines recidivism might affect predictive accuracy. Nonrecidivism is problematic when predicting specialized violence (e.g., domestic violence). The present study cross-validates the ability of the Ontario Domestic Assault Risk Assessment (ODARA) to distinguish subsequent recidivists and nonrecidivists among 391 new cases with less extensive criminal records than previous cross-validation samples, base rate=27%, ROC area=.67. Excluding ambiguous nonrecidivists increases the base rate to 33%, ROC area=.74. Random samples of 50 recidivists and 50 unambiguous nonrecidivists yield ROC areas from .71 to .80. Published norms significantly underestimate official recidivism. Ambiguous nonrecidivism is prevalent and leads to underestimating base rates and predictive accuracy.

Publication types

  • Research Support, Non-U.S. Gov't
  • Validation Study

MeSH terms

  • Adult
  • Cross-Sectional Studies
  • Humans
  • Male
  • Middle Aged
  • Ontario
  • Personality Assessment / statistics & numerical data*
  • Psychometrics / statistics & numerical data
  • ROC Curve
  • Recurrence
  • Reproducibility of Results
  • Risk Assessment / statistics & numerical data*
  • Spouse Abuse / prevention & control
  • Spouse Abuse / statistics & numerical data*
  • Surveys and Questionnaires*
  • Violence / statistics & numerical data