Background: Standardized letters of recommendation (SLORs) were introduced to provide a more objective method of evaluating applicants for orthopaedic surgery residency positions. We sought to establish whether there exists an association between the SLOR summative rank statement (SRS), which is a question that asks the letter-writing authors where they would rank a student relative to other applicants, and success in matching into orthopaedic surgery residency.
Methods: We reviewed 858 applications to an orthopaedic surgery residency program from 2017 to 2018. Data on 9 assessment categories, SRSs, and written comments in the SLORs were extracted. The match success of applicants was determined by an internet search algorithm. Logistic regression was used to evaluate the association between the SRSs and match success. The Spearman correlation was performed between the SRSs and other variables.
Results: Only 60% of all LORs were SLORs. With 24% of the SLORs, a supplemental letter was used. Median percentile rank ranged from 90% to 100% for the 9 categories in the SLORs. Recommendations of "high rank" or higher were found in 88% of the SRSs. The mean of the SLOR SRSs was associated with match success.
Conclusions: The mean of the SLOR SRSs was associated with match success. However, the SLOR is not uniformly used. Future efforts should be devoted to improving question design and validity in order to better discriminate among applicants, increase adherence to the rating scale, and quantify the strength of the written comments in the SLOR.