Reliability of a point-based VTE risk assessment tool in the hands of medical residents

Abstract

Background: Venous thromboembolic events (VTE) are a significant cause of mortality in hospitalized medical and surgical patients. Despite recommendations and guidelines, current evidence demonstrates that VTE prophylaxis remains underutilized in at-risk patients. The process of providing VTE prophylaxis begins with assessing each patient's VTE risk. Using an individualized, point-based protocol in the assessment process is a complex task, and might contribute to variability in VTE prescribing behavior. There are no published data on how reliably residents can perform risk assessment and prophylaxis using a point-based VTE risk assessment tool.

Objective: Our aim was to determine inter-rater reliability of a point-based risk assessment tool by residents early in the academic year.

Design: The design was a cross-sectional-cohort observational study.

Setting: The site was an academic medical center.

Patients: Case-based clinical vignettes were used.

Interventions: Verbal instructions were given to medical residents about how to apply our hospital's point-based VTE risk assessment tool.

Measurements: Interobserver agreement was measured of: 1) risk score, 2) risk-stratification, 3) identification of contraindications, 4) VTE prophylaxis plan, and 5) resident adherence to the protocol.

Results: The intra-class correlation (ICC) for the total risk score was 0.66 and the kappa coefficient for risk stratification was 0.51. The kappa scores for absolute and relative contraindications were 0.29 and 0.23, respectively. The kappa score for the VTE plan was 0.28.

Conclusions: We determined that, following brief instructions early in the academic year, a point-based VTE risk assessment tool has only fair to moderate inter-rater reliability, with suboptimal adherence to the protocol. Both might lead to underutilization of VTE prevention strategies.