Validation of the Surprise Question and the Development of a Multivariable Model

Abstract

Context: The Surprise Question (SQ) (would you be surprised if this patient died within a year?) is a prognostic variable explored in chronic illnesses. Validation is limited to sensitivity, specificity, and predictive values.

Objectives: Our objective is to validate the SQ in cancer patients and develop a predictive model with additional variables.

Methods: A prospective cohort study of adult (age>18) cancer patients seen between October 1, 2019, through March 31, 2021, undergoing systemic therapies had the SQ completed by oncologists prior to each change in systemic therapy. The primary outcome was survival for one year. Secondary outcomes were predictions of survival at three, six, and nine months. Patients were grouped into negative SQ (not surprised) and positive SQ (surprised). Sensitivity, specificity, predictive values, and likelihood ratios (LR) were calculated for the SQ. Additional prognostic variables were age, gender, cancer stage, line of therapy, Charleson Comorbid Index (CCI), palliative care consultation (prior to, after the SQ, or not at all), and healthcare utilization (outpatient, inpatient, and emergency department (ED). Logistic regression and receiver operating characteristics (ROC) were used for discrimination and modeling. Akaike information criterion (AIC) was used to compare the model fit as each predictor.

Results: 1366 patients had 1 SQ; 784 died within a year. The SQ predicted survival at one year (P = 0.008), with a positive LR of 1.459 (95%CI 1.316-1.602) and a c-statistic of 0.565 (95%CI 0.530-0.600). Additional variables increased the c-statistic to 0.648 (95% CI 0.608-0.686). The total model best predicted survival at three months, c-statistic of 0.663 (95% CI 0.616-0.706). However, the total model c-statistic remained <0.70.

Conclusions: The SQ, as a single factor, poorly predicts survival and should not be used to alter therapies. Adding additional objective variables improved prognostication, but further refinement and external validation are needed.

Keywords: Prediction; outcome; question; surprise; survival.