Analysis of Orthopedic Resident Ability to Apply Levels of Evidence Criteria to Scientific Articles

Abstract

Objective: In the era of evidence-based medicine, understanding study design and levels of evidence (LoE) criteria is an important component of resident education and aids practicing surgeons in making informed clinical decisions. The purpose of this study is to analyze the ability of orthopedic residents to accurately determine LoE criteria for published articles compared with medical students.

Design: Basic science article.

Setting: Geisinger Medical Center (Danville, PA), tertiary referral center.

Participants: Overall, 25 U.S. orthopedic residents and 15 4th year medical students interviewing for a residency position in orthopedic surgery voluntarily participated and provided baseline demographic information. A total of 15 articles from the American Volume of Journal of Bone and Joint Surgery were identified. Study participants were provided with the article title, the abstract, and the complete methods section. The assigned LoE designation was withheld and access to the LoE criteria used by Journal of Bone and Joint Surgery was provided. Each participant was assigned a study type and LoE designation for each article.

Results: There were more correct responses regarding the article type (67%) than for LoE designation (39%). For LoE, the intraclass correlation coefficient was 0.30. The percentage of correct responses for article type and LoE increased with more years of training (p = 0.005 and p = 0.002). Although residents had a higher proportion of correct LoE responses overall than medical students, this difference did not reach statistical significance (42% vs. 35%, p = 0.07).

Conclusions: Although improvements in accurately determining both article type and LoE were seen among residents with increasing years of training, residents were unable to demonstrate a statistically significant improvement for determining LoE or article type when compared with medical students. Strategies to improve resident understanding of LoE guidelines need to be incorporated into orthopedic residencies, especially when considering the increased emphasis on evidence-based medicine.

Keywords: Medical Knowledge; Practice-Based Learning and Improvement; Systems-Based Practice; levels of evidence; orthopedics; research; resident education.