As widely known, regular accuracy is a misleading and shallow indicator of the performance of a predictive model, especially in real-life domains like medicine, where decisions affect health or life. In this paper we present and discuss a new accuracy measure, the H-accuracy, as a more conservative alternative to regular accuracy, which we claim is more informative in the medical domain (and others of similar needs) for the elements it encompasses. In particular, the proposed measure takes into account important information such as the complexity of the cases and the case prevalance in the population. We also provide proof that the H-accuracy is a generalization of the balanced accuracy and illustrate the descriptive power of this score.
Keywords: Accuracy; Machine Learning; Medical Artificial Intelligence; Validation.