Developing an Improved Statistical Approach for Survival Estimation in Bone Metastases Management: The Bone Metastases Ensemble Trees for Survival (BMETS) Model

Abstract

Purpose: To determine whether a machine learning approach optimizes survival estimation for patients with symptomatic bone metastases (SBM), we developed the Bone Metastases Ensemble Trees for Survival (BMETS) to predict survival using 27 prognostic covariates. To establish its relative clinical utility, we compared BMETS with 2 simpler Cox regression models used in this setting.

Methods and materials: For 492 bone sites in 397 patients evaluated for palliative radiation therapy (RT) for SBM from January 2007 to January 2013, data for 27 clinical variables were collected. These covariates and the primary outcome of time from consultation to death were used to build BMETS using random survival forests. We then performed Cox regressions as per 2 validated models: Chow's 3-item (C-3) and Westhoff's 2-item (W-2) tools. Model performance was assessed using cross-validation procedures and measured by time-dependent area under the curve (tAUC) for all 3 models. For temporal validation, a separate data set comprised of 104 bone sites treated in 85 patients in 2018 was used to estimate tAUC from BMETS.

Results: Median survival was 6.4 months. Variable importance was greatest for performance status, blood cell counts, recent systemic therapy type, and receipt of concurrent nonbone palliative RT. tAUC at 3, 6, and 12 months was 0.83, 0.81, and 0.81, respectively, suggesting excellent discrimination of BMETS across postconsultation time points. BMETS outperformed simpler models at each time, with respective tAUC at each time of 0.78, 0.76, and 0.74 for the C-3 model and 0.80, 0.78, and 0.77 for the W-2 model. For the temporal validation set, respective tAUC was similarly high at 0.86, 0.82, and 0.78.

Conclusions: For patients with SBM, BMETS improved survival predictions versus simpler traditional models. Model performance was maintained when applied to a temporal validation set. To facilitate clinical use, we developed a web platform for data entry and display of BMETS-predicted survival probabilities.