Evaluating the Inaccuracy of the National Surgical Quality Improvement Project Surgical Risk Calculator in Plastic Surgery: A Meta-analysis of Short-Term Predicted Complications

Abstract

Background: Preoperative surgical risk assessment is a major component of clinical decision making. The ability to provide accurate, individualized risk estimates has become critical because of growing emphasis on quality metrics benchmarks. The American College of Surgeons National Surgical Quality Improvement Project (NSQIP) Surgical Risk Calculator (SRC) was designed to quantify patient-specific risk across various surgeries. Its applicability to plastic surgery is unclear, however, with multiple studies reporting inaccuracies among certain patient populations. This study uses meta-analysis to evaluate the NSQIP SRC's ability to predict complications among patients having plastic surgery.

Methods: OVID MEDLINE and PubMed were searched for all studies evaluating the predictive accuracy of the NSQIP SRC in plastic surgery, including oncologic reconstruction, ventral hernia repair, and body contouring. Only studies directly comparing SCR predicted to observed complication rates were included. The primary measure of SRC prediction accuracy, area under the curve (AUC), was assessed for each complication via DerSimonian and Laird random-effects analytic model. The I2 statistic, indicating heterogeneity, was judged low (I2 < 50%) or borderline/unacceptably high (I2 > 50%). All analyses were conducted in StataSE 16.1 (StataCorp LP, College Station, Tex).

Results: Ten of the 296 studies screened met criteria for inclusion (2416 patients). Studies were classified as follows: (head and neck: n = 5, breast: n = 1, extremity: n = 1), open ventral hernia repair (n = 2), and panniculectomy (n = 1). Predictive accuracy was poor for medical and surgical complications (medical: pulmonary AUC = 0.67 [0.48-0.87], cardiac AUC = 0.66 [0.20-0.99], venous thromboembolism AUC = 0.55 [0.47-0.63]), (surgical: surgical site infection AUC = 0.55 [0.46-0.63], reoperation AUC = 0.54 [0.49-0.58], serious complication AUC = 0.58 [0.43-0.73], and any complication AUC = 0.60 [0.57-0.64]). Although mortality was accurately predicted in 2 studies (AUC = 0.87 [0.54-0.99]), heterogeneity was high with I2 = 68%. Otherwise, heterogeneity was minimal (I2 = 0%) or acceptably low (I2 < 50%) for all other outcomes.

Conclusions: The NSQIP Universal SRC, aimed at offering individualized quantifiable risk estimates for surgical complications, consistently demonstrated poor risk discrimination in this plastic surgery-focused meta-analysis. The limitations of the SRC are perhaps most pronounced where complex, multidisciplinary reconstructions are needed. Future efforts should identify targets for improving SRC reliability to better counsel patients in the perioperative setting and guide appropriate healthcare resource allocation.