Short text topic modelling approaches in the context of big data: taxonomy, survey, and analysis

Belal Abdullah Hezam Murshed; Suresha Mallappa; Jemal Abawajy; Mufeed Ahmed Naji Saif; Hasib Daowd Esmail Al-Ariki; Hudhaifa Mohammed Abdulwahab

doi:10.1007/s10462-022-10254-w

Short text topic modelling approaches in the context of big data: taxonomy, survey, and analysis

Artif Intell Rev. 2023;56(6):5133-5260. doi: 10.1007/s10462-022-10254-w. Epub 2022 Oct 26.

Authors

Belal Abdullah Hezam Murshed^{1

2}, Suresha Mallappa¹, Jemal Abawajy³, Mufeed Ahmed Naji Saif⁴, Hasib Daowd Esmail Al-Ariki^{5

6}, Hudhaifa Mohammed Abdulwahab⁷

Affiliations

¹ Department of Studies in Computer Science, Mysore University, Mysore, 570006 Karnataka India.
² Department of Computer Science, College of Engineering and IT, Amran University, Amran, Yemen.
³ School of Information Technology, Faculty of Science, Engineering and Built Environment, Deakin University, Geelong, VIC 3220 Australia.
⁴ Department of Computer Applications, Sri Jayachamarajendra College of Engineering, VTU, Mysore, Karnataka India.
⁵ Department of Computer Networks and Distributed Systems, Al Saeed Faculty for Engineering and IT, Taiz University, Taiz, Yemen.
⁶ Department of Computer Networks Engineering and Technologies, Sana'a Community College, Sana'a, Yemen.
⁷ Department of Computer Application, Ramaiah Institute of Technology, VTU, Bengaluru, India.

Abstract

Social media platforms such as (Twitter, Facebook, and Weibo) are being increasingly embraced by individuals, groups, and organizations as a valuable source of information. This social media generated information comes in the form of tweets or posts, and normally characterized as short text, huge, sparse, and low density. Since many real-world applications need semantic interpretation of such short texts, research in Short Text Topic Modeling (STTM) has recently gained a lot of interest to reveal unique and cohesive latent topics. This article examines the current state of the art in STTM algorithms. It presents a comprehensive survey and taxonomy of STTM algorithms for short text topic modelling. The article also includes a qualitative and quantitative study of the STTM algorithms, as well as analyses of the various strengths and drawbacks of STTM techniques. Moreover, a comparative analysis of the topic quality and performance of representative STTM models is presented. The performance evaluation is conducted on two real-world Twitter datasets: the Real-World Pandemic Twitter (RW-Pand-Twitter) dataset and Real-world Cyberbullying Twitter (RW-CB-Twitter) dataset in terms of several metrics such as topic coherence, purity, NMI, and accuracy. Finally, the open challenges and future research directions in this promising field are discussed to highlight the trends of research in STTM. The work presented in this paper is useful for researchers interested in learning state-of-the-art short text topic modelling and researchers focusing on developing new algorithms for short text topic modelling.

Keywords: Big data; Coherence; Data streaming; Deep learning topic modeling; Short text topic modeling; Social media; Sparseness.

© The Author(s), under exclusive licence to Springer Nature B.V. 2022, Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.