Combining data discretization and missing value imputation for incomplete medical datasets

PLoS One. 2023 Nov 30;18(11):e0295032. doi: 10.1371/journal.pone.0295032. eCollection 2023.

Abstract

Data discretization aims to transform a set of continuous features into discrete features, thus simplifying the representation of information and making it easier to understand, use, and explain. In practice, users can take advantage of the discretization process to improve knowledge discovery and data analysis on medical domain problem datasets containing continuous features. However, certain feature values were frequently missing. Many data-mining algorithms cannot handle incomplete datasets. In this study, we considered the use of both discretization and missing-value imputation to process incomplete medical datasets, examining how the order of discretization and missing-value imputation combined influenced performance. The experimental results were obtained using seven different medical domain problem datasets: two discretizers, including the minimum description length principle (MDLP) and ChiMerge; three imputation methods, including the mean/mode, classification and regression tree (CART), and k-nearest neighbor (KNN) methods; and two classifiers, including support vector machines (SVM) and the C4.5 decision tree. The results show that a better performance can be obtained by first performing discretization followed by imputation, rather than vice versa. Furthermore, the highest classification accuracy rate was achieved by combining ChiMerge and KNN with SVM.

MeSH terms

  • Algorithms*
  • Cluster Analysis
  • Data Mining
  • Support Vector Machine*

Grants and funding

The work was supported in part by the Ministry of Science and Technology of Taiwan under Grant MOST 108-2410-H-008-063-MY3 and MOST 111-2410-H182-015-MY3 and in part by Chang Gung Memorial Hospital, Linkou, under Grant BMRPH13 and CMRPG3J0732. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.