Background and purpose: Leptomeningeal metastases (LMs) carry a poor prognosis. Existing LM scoring systems show limited reproducibility. We assessed the contribution of education level on the reproducibility of LM scoring using structured planning and implementation of new experiments (SPINE), a novel web-based platform.
Methods: Stringent radiological definitions of LM and a customized interactive scoring system were implemented in SPINE. Five patients with brain LM and 3 patients with spine, but no brain LM, were selected. Each patient's baseline post-contrast T1-weighted brain MRI was analyzed by three attending neuroradiologists, two neuroradiology fellows, and two radiology residents. Raters identified and characterized all LMs based on: (1) location (cerebrum, cerebellum, brainstem, ventricle, and/or cranial nerves); (2) shape (nodular and/or linear/curvilinear); (3) size (≥ or <5mm in two orthogonal diameters); (4) spatial extension (focal or diffuse). Inter-rater agreement and association of LM with patient survival were investigated.
Results: On average, 6.5 LMs per case were detected. Forty-nine percent of LMs were cerebral, 77.7% were nodular, 86.6% were focal, and 66% were <5 × 5 mm. Agreement on the total number of LMs and the above-mentioned common LM characteristics was higher between attendings (intra-class correlation [ICC] = 0.8-0.94) than fellows (ICC = 0.6-0.82) or residents (ICC = 0.43-0.73). Agreement on ventricular, cranial nerve, and nodular + linear LM was low even between attendings. The number of brainstem LMs showed significant correlation with survival.
Conclusion: Structured education using SPINE may improve consistency in LM reporting. Future work should address the impact of the presented approach on the reproducibility of longitudinal analyses directly relevant to the assessment of treatment-response.
Keywords: Brain MRI; inter-rater agreement; interactive web-based scoring; leptomeningeal metastasis; patient survival.
© 2020 American Society of Neuroimaging.