Comparative analysis of three risk assessment tools in Australian patients with prostate cancer

Abstract

What's known on the subject? and What does the study add? Prognostic tools, such as the Cancer of the Prostate Risk Assessment (CAPRA) score and the 1998 Kattan and 2006 Stephenson nomograms, predicting biochemical recurrence after radical prostatectomy are widely used for treatment decision making and counselling patients. However, tools derived in certain cohorts tend to perform less well when they are applied to populations that are dissimilar in terms of population or disease characteristics, health systems or treatment practices. Some of the loss in accuracy of a prognostic tool is a consequence of unknown factors and hence the performance of a tool when applied to a different population is unknown and largely unpredictable. This study validates these widely used tools in South Australian patients treated at three public hospitals. All three tools discriminated well according to risk of recurrence in these patients. However, when compared against observed rates of recurrence, it was found that predictions of recurrence varied widely between the three tools, suggesting that their use in counselling patients on such risk may not be appropriate. Interestingly, the oldest of the three tools (Kattan 1998) was the best predictor of absolute risk of recurrence. In the paper, this is linked to later adoption of updated Gleason grading, among other factors.

Summary: In many countries, prognostic tools, which draw on the experience of thousands of patients with cancer, are used to predict cancer outcomes, but accuracy varies. This paper compares the accuracy of three widely used tools predicting prostate cancer recurrence after surgery in Australian patients. The results show that all tools were good at predicting which patients were most likely to experience recurrence and which were least. However, prediction of absolute risk varied and the oldest tool was the most accurate.

Objective: • To compare performance of the CAPRA score and two commonly used risk assessment nomograms, the 1998 Kattan and the 2006 Stephenson, in an untested Australian cohort.

Patients and methods: • We present data on 635 men from the South Australian Prostate Cancer Clinical Outcomes Database who underwent radical prostatectomy between January 1996 and May 2009 and had all required variables for predicting biochemical recurrence (BCR). • BCR was defined as prostate-specific antigen ≥ 0.2 ng/mL or secondary treatment for a rising prostate-specific antigen. • Accuracy was evaluated using Harrell's concordance index, plotting calibration curves, and constructing decision analysis curves.

Results: • Concordance indices were high for all three tools: 0.791, 0.787 and 0.744 for the 2006 Stephenson nomogram, CAPRA score and 1998 Kattan nomogram respectively. • At 3 years, calibration of the tools (agreement between predicted and observed BCR-free probability) was close to ideal for the 1998 Kattan nomogram, whereas the 2006 Stephenson model underestimated and the CAPRA model overestimated BCR-free probability. • The 1998 Kattan and 2005 CAPRA tools performed better than the 2006 Stephenson nomogram across a wide range of threshold probabilities using decision curve analysis.

Conclusion: • All three tools discriminate between patients' risk effectively. • Absolute estimates of risk are likely to vary widely between tools, however, suggesting that models should be validated and, if necessary, recalibrated in the population to which they will be applied. • Recent development does not mean a nomogram is more accurate for use in a particular population.