Fine-tuned Sentiment Analysis of COVID-19 Vaccine-Related Social Media Data: Comparative Study

Background: The emergence of the novel coronavirus (COVID-19) and the necessary separation of populations have led to an unprecedented number of new social media users seeking information related to the pandemic. Currently, with an estimated 4.5 billion users worldwide, social media data offer an opportunity for near real-time analysis of large bodies of text related to disease outbreaks and vaccination. These analyses can be used by officials to develop appropriate public health messaging, digital interventions, educational materials, and policies.

Objective: Our study investigated and compared public sentiment related to COVID-19 vaccines expressed on 2 popular social media platforms-Reddit and Twitter-harvested from January 1, 2020, to March 1, 2022.

Methods: To accomplish this task, we created a fine-tuned DistilRoBERTa model to predict the sentiments of approximately 9.5 million tweets and 70 thousand Reddit comments. To fine-tune our model, our team manually labeled the sentiment of 3600 tweets and then augmented our data set through back-translation. Text sentiment for each social media platform was then classified with our fine-tuned model using Python programming language and the Hugging Face sentiment analysis pipeline.

Results: Our results determined that the average sentiment expressed on Twitter was more negative (5,215,830/9,518,270, 54.8%) than positive, and the sentiment expressed on Reddit was more positive (42,316/67,962, 62.3%) than negative. Although the average sentiment was found to vary between these social media platforms, both platforms displayed similar behavior related to the sentiment shared at key vaccine-related developments during the pandemic.

Conclusions: Considering this similar trend in shared sentiment demonstrated across social media platforms, Twitter and Reddit continue to be valuable data sources that public health officials can use to strengthen vaccine confidence and combat misinformation. As the spread of misinformation poses a range of psychological and psychosocial risks (anxiety and fear, etc), there is an urgency in understanding the public perspective and attitude toward shared falsities. Comprehensive educational delivery systems tailored to a population's expressed sentiments that facilitate digital literacy, health information-seeking behavior, and precision health promotion could aid in clarifying such misinformation.