Adaptation of Autoencoder for Sparsity Reduction From Clinical Notes Representation Learning

Thanh-Dung Le; Rita Noumeir; Jerome Rambaud; Guillaume Sans; Philippe Jouvet

doi:10.1109/JTEHM.2023.3241635

Adaptation of Autoencoder for Sparsity Reduction From Clinical Notes Representation Learning

IEEE J Transl Eng Health Med. 2023 Feb 2:11:469-478. doi: 10.1109/JTEHM.2023.3241635. eCollection 2023.

Authors

Thanh-Dung Le^{1

2}, Rita Noumeir¹, Jerome Rambaud², Guillaume Sans², Philippe Jouvet²

Affiliations

¹ Biomedical Information Processing Laboratory, Ecole de Technologie SuperieureUniversity of Quebec Montreal QC H3C 1K3 Canada.
² Research Center at CHU Sainte-JustineUniversity of Montreal Montreal QC H3T 1J4 Canada.

Abstract

When dealing with clinical text classification on a small dataset, recent studies have confirmed that a well-tuned multilayer perceptron outperforms other generative classifiers, including deep learning ones. To increase the performance of the neural network classifier, feature selection for the learning representation can effectively be used. However, most feature selection methods only estimate the degree of linear dependency between variables and select the best features based on univariate statistical tests. Furthermore, the sparsity of the feature space involved in the learning representation is ignored.

Goal: Our aim is, therefore, to access an alternative approach to tackle the sparsity by compressing the clinical representation feature space, where limited French clinical notes can also be dealt with effectively.

Methods: This study proposed an autoencoder learning algorithm to take advantage of sparsity reduction in clinical note representation. The motivation was to determine how to compress sparse, high-dimensional data by reducing the dimension of the clinical note representation feature space. The classification performance of the classifiers was then evaluated in the trained and compressed feature space.

Results: The proposed approach provided overall performance gains of up to 3% for each test set evaluation. Finally, the classifier achieved 92% accuracy, 91% recall, 91% precision, and 91% f1-score in detecting the patient's condition. Furthermore, the compression working mechanism and the autoencoder prediction process were demonstrated by applying the theoretic information bottleneck framework. Clinical and Translational Impact Statement- An autoencoder learning algorithm effectively tackles the problem of sparsity in the representation feature space from a small clinical narrative dataset. Significantly, it can learn the best representation of the training data because of its lossless compression capacity compared to other approaches. Consequently, its downstream classification ability can be significantly improved, which cannot be done using deep learning models.

Keywords: Clinical natural language processing; autoencoder; cardiac failure; sparsity.

Publication types

Research Support, Non-U.S. Gov't

MeSH terms

Algorithms*
Correlation of Data
Data Compression*
Humans
Neural Networks, Computer

Grants and funding

This work was supported in part by the Natural Sciences and Engineering Research Council (NSERC), in part by the Institut de Valorisation des Donnees de l’Universite de Montreal (IVADO), in part by the Fonds de la Recherche en Sante du Quebec (FRQS), and in part by the Fonds de Recherche du Quebec-Nature et Technologies (FRQNT).