A novel approach for standardizing clinical laboratory categorical test results using machine learning and string distance similarity

Syed Ahmmed; M Rubaiyat Hossain Mondal; Md Raihan Mia; Mohammad Adibuzzaman; Abu Sayed Md Latiful Hoque; Sheikh Iqbal Ahamed

doi:10.1016/j.heliyon.2023.e21523

A novel approach for standardizing clinical laboratory categorical test results using machine learning and string distance similarity

Heliyon. 2023 Nov 8;9(11):e21523. doi: 10.1016/j.heliyon.2023.e21523. eCollection 2023 Nov.

Authors

Syed Ahmmed¹, M Rubaiyat Hossain Mondal¹, Md Raihan Mia^{2

3}, Mohammad Adibuzzaman⁴, Abu Sayed Md Latiful Hoque², Sheikh Iqbal Ahamed³

Affiliations

¹ Institute of Information and Communication Technology, Bangladesh University of Engineering And Technology, Dhaka, Bangladesh.
² Department of Computer Science And Engineering, Bangladesh University of Engineering And Technology, Dhaka, Bangladesh.
³ Department of Computer Science, Marquette University, Milwaukee, WI, USA.
⁴ Oregon Clinical and Translational Research Institute, Oregon Health and Science University, Portland, OR, USA.

Abstract

Standardizing clinical laboratory test results is critical for conducting clinical data science research and analysis. However, standardized data processing tools and guidelines are inadequate. In this paper, a novel approach for standardizing categorical test results based on supervised machine learning and the Jaro-Winkler similarity algorithm is proposed. A supervised machine learning model is used in this approach for scalable categorization of the test results into predefined groups or clusters, while Jaro-Winkler similarity is used to map text terms into standard clinical terms within these corresponding groups. The proposed method is applied to 75062 test results from two private hospitals in Bangladesh. The Support Vector Classification algorithm with a linear kernel has a classification accuracy of 98%, which is better than the Random Forest algorithm when categorizing test results. The experiment results show that Jaro-Winkler similarity achieves a remarkable 99.93% success rate in the test result standardization for the majority of groups with manual validation. The proposed method outperforms previous studies that concentrated on standardizing test results using rule-based classifiers on a smaller number of groups and distance similarities such as Cosine similarity or Levenshtein distance. Furthermore, when applied to the publicly available MIMIC-III dataset, our approach also performs excellently. All these findings show that the proposed standardization technique can be very beneficial for clinical big data research, particularly for national clinical research data hubs in low- and middle-income countries.

Keywords: Data quality; Data science; Electronic health records; LOINC; Machine learning; SNOMED CT; Standardization; String distance similarity.