Assessing the suitability of sets-based approaches: estimating the discriminative power of risk models for ordinal outcome treatments

Abstract

Background: In order to evaluate the discrimination performance of an ordinal model for improved disease screening, a new test was proposed where information was obtained across all samples simultaneously.

Methods: The ordinal c-index builds upon the volume under the surface methodology without focusing on the accompanying receiver operating characteristic surfaces. However, it can be simplified to an average of pairwise c-indexes. In this paper, a set-based estimate (information was obtained across all samples simultaneously) was proposed by summing all correctly ordered groups. The asymptotic distribution of this proposed estimate was derived using U-statistics.

Results: A predictive model was applied using the blood urea nitrogen/creatinine ratio to discriminate stroke in evolution in acute ischemic stroke patients, which could potentially be life-saving in emergency departments.

Conclusions: By conducting Monte Carlo simulations, it was concluded that the measure proposed herein is a better choice for clinical use because of the asymmetry of the predicted probabilities of groups.

Keywords: Acute ischemic stroke; Discriminative power; National institutes of health stroke scale; Ordinal outcomes; Stroke in evolution.