Using machine learning to identify key subject categories predicting the pre-clerkship and clerkship performance: 8-year cohort study

Shiau-Shian Huang; Yu-Fan Lin; Anna YuQing Huang; Ji-Yang Lin; Ying-Ying Yang; Sheng-Min Lin; Wen-Yu Lin; Pin-Hsiang Huang; Tzu-Yao Chen; Stephen J H Yang; Jiing-Feng Lirng; Chen-Huan Chen

doi:10.1097/JCMA.0000000000001097

Using machine learning to identify key subject categories predicting the pre-clerkship and clerkship performance: 8-year cohort study

J Chin Med Assoc. 2024 Apr 18. doi: 10.1097/JCMA.0000000000001097. Online ahead of print.

Authors

Shiau-Shian Huang^{1

2}, Yu-Fan Lin¹, Anna YuQing Huang³, Ji-Yang Lin³, Ying-Ying Yang^{1

2}, Sheng-Min Lin¹, Wen-Yu Lin¹, Pin-Hsiang Huang¹, Tzu-Yao Chen³, Stephen J H Yang³, Jiing-Feng Lirng^{1

2}, Chen-Huan Chen^{1

2}

Affiliations

¹ Department of Medical Education, Clinical Innovation Center, Taipei Veterans General Hospital, Taipei, Taiwan, ROC.
² College of Medicine, National Yang Ming Chiao Tung University, Taipei, Taiwan, ROC.
³ Department of Computer Science and Information Engineering, National Central University, Taoyuan, Taiwan, ROC.

PMID: 38648194
DOI: 10.1097/JCMA.0000000000001097

Abstract

Background: Medical students need to build a solid foundation of knowledge to become physicians. Clerkship is often considered the first transition point, and clerkship performance is essential for their development. We hope to identify subjects that could predict the clerkship performance, thus helping medical students learn more efficiently to achieve high clerkship performance.

Methods: This cohort study collected background and academic data from medical students who graduated between 2011 and 2019. Prediction models were developed by machine learning techniques to identify the affecting features in predicting the pre-clerkship performance and clerkship performance. Following serial processes of data collection, data pre-processing before machine learning, and techniques and performance of machine learning, different machine learning models were trained and validated using the 10-fold cross-validation method.

Results: Thirteen subjects from the pre-med stage and ten subjects from the basic medical science stage with an area under the ROC curve (AUC) greater than 0.7 for either pre-clerkship performance or clerkship performance were found. In each subject category, medical humanities and sociology in social science, chemistry and physician scientist-related training in basic science, and pharmacology, immunology-microbiology, and histology in basic medical science have predictive abilities for clerkship performance above the top tertile. Using a machine learning technique based on random forest, the prediction model predicted clerkship performance with 95% accuracy and 88% AUC.

Conclusion: Clerkship performance was predicted by selected subjects or combination of different subject categories in the pre-med and basic medical science stages. The demonstrated predictive ability of subjects or categories in the medical program may facilitate students' understanding of how these subjects or categories of the medical program relate to their performance in the clerkship to enhance their preparedness for the clerkship.