Cohen's kappa for capturing discrimination

Abstract

Background: Identification of cut-off values for key biomarkers of clinical risk is a useful clinical tool. The kappa coefficient is a popular descriptive statistical measure for summarising the cross classification of two nominal variables with identical classes. On the basis of this definition, I propose that the kappa coefficient can also be used to capture discrimination, in the same way that the receiver operating characteristic (ROC) curve is used in preventive epidemiology studies.

Methods: The statistics were determined using Cohen's kappa statistics for a gold standard and a continuous biomarker. The proposed design is compared with the ROC curve by applying it to articles on the metabolic syndrome and a colon cancer clinical trial.

Results: The two methods gave similar results. Moreover, Monte Carlo simulation results confirm that, from a power perspective, the proposed method is to be preferred. In general, the proposed method has higher power than the area under the ROC curve (AUC) for a study of positively correlative design.

Conclusion: Overall, the power performance of the proposed method is better that that of the AUC.

Keywords: Area under the ROC curve; Cohen's kappa; Cut-off value; Power; Receiver operating characteristic curve.