Interrater and Intrarater Reliability of the Vertebral Bone Quality Score

Abstract

Background: Vertebral bone quality had a significant impact on postoperative outcomes in spinal fusion surgery. New magnetic resonance imaging-based measures, such as the Vertebral Bone Quality (VBQ) score, may allow for bone quality assessment without the radiation associated with conventional testing. In the present study, we sought to assess the intrarater and interrater reliability of VBQ scores calculated by medical professionals and trainees.

Methods: Thirteen reviewers of various specialties and levels of training were recruited and asked to calculate VBQ scores for 30 patients at 2 time points separated by 2 months. Scored volumes were acquired from patients treated for both degenerative and oncologic indications. Intrarater and interrater agreement, quantified by intraclass correlation coefficient (ICC), was assessed using 2-way random effects modeling. Square-weight Cohen κ and Kendall Tau-b were used to determine whether raters assigned similar scores during both evaluations.

Results: All raters showed moderate to excellent reliability for VBQ score (ICC 0.667-0.957; κ0.648-0.921) and excellent reliability for all constituent components used to calculate VBQ score (ICC all ≥0.97). Interrater reliability was also found to be good for VBQ score on both the first (ICC = 0.818) and second (ICC = 0.800) rounds of assessment; scores for the constituent component all had ICC values ≥0.97 for the constituent components.

Conclusions: The VBQ score appears to have both good intrarater and interrater reliability. In addition, there appeared to be no correlation between score reliability and level of training. External validation and further investigations of its ability to accurately model bone biomechanical properties are necessary.

Keywords: Cohen κ; Interrater reliability; Osteoporosis; Score validation; Vertebral bone quality.