Performance evaluation of machine learning models on large dataset of android applications reviews

Ali Adil Qureshi; Maqsood Ahmad; Saleem Ullah; Muhammad Naveed Yasir; Furqan Rustam; Imran Ashraf

doi:10.1007/s11042-023-14713-6

Performance evaluation of machine learning models on large dataset of android applications reviews

Multimed Tools Appl. 2023 Mar 18:1-23. doi: 10.1007/s11042-023-14713-6. Online ahead of print.

Authors

Ali Adil Qureshi¹, Maqsood Ahmad², Saleem Ullah¹, Muhammad Naveed Yasir³, Furqan Rustam⁴, Imran Ashraf⁵

Affiliations

¹ Department of Computer Science, Khwaja Fareed University of Engineering and Information Technology, Rahim Yar Khan, 64200 Pakistan.
² Department of Information Security, The Islamia University of Bahawalpur, Bahawalpur, Punjab 63100 Pakistan.
³ Department of Computer Science, University of Narowal, Narowal, 51600 Pakistan.
⁴ School of Computer Science, University College Dublin, Belfield, Dublin 4 Ireland.
⁵ Information and Communication Engineering, Yeungnam University, Gyeongsan, 38541 Korea.

Abstract

With an ever-increasing number of mobile users, the development of mobile applications (apps) has become a potential market during the past decade. Billions of users download mobile apps for divergent use from Google Play Store, fulfill tasks and leave comments about their experience. Such reviews are replete with a variety of feedback that serves as a guide for the improvement of existing apps and intuition for novel mobile apps. However, application reviews are challenging and very broad to approach. Such reviews, when segregated into different classes guide the user in the selection of suitable apps. This study proposes a framework for analyzing the sentiment of reviews for apps of eight different categories like shopping, sports, casual, etc. A large dataset is scrapped comprising 251661 user reviews with the help of 'Regular Expression' and 'Beautiful Soup'. The framework follows the use of different machine learning models along with the term frequency-inverse document frequency (TF-IDF) for feature extraction. Extensive experiments are performed using preprocessing steps, as well as, the stats feature of app reviews to evaluate the performance of the models. Results indicate that combining the stats feature with TF-IDF shows better performance and the support vector machine obtains the highest accuracy. Experimental results can potentially be used by other researchers to select appropriate models for the analysis of app reviews. In addition, the provided dataset is large, diverse, and balanced with eight categories and 59 app reviews and provides the opportunity to analyze reviews using state-of-the-art approaches.

Keywords: Google Play Store; Mobile apps reviews; Opinion mining; Sentiment analysis.

© The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature 2023, Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.