The widespread use of social media provides a large amount of data for public sentimentanalysis. Based on social media data, researchers can study public opinions on humanpapillomavirus (HPV) vaccines on social media using machine learning-based approaches that willhelp us understand the reasons behind the low vaccine coverage. However, social media data isusually unannotated, and data annotation is costly. The lack of an abundant annotated dataset limitsthe application of deep learning methods in effectively training models. To tackle this problem, wepropose three transfer learning approaches to analyze the public sentiment on HPV vaccines onTwitter. One was transferring static embeddings and embeddings from language models (ELMo)and then processing by bidirectional gated recurrent unit with attention (BiGRU-Att), called DWEBiGRU-Att. The others were fine-tuning pre-trained models with limited annotated data, called finetuninggenerative pre-training (GPT) and fine-tuning bidirectional encoder representations fromtransformers (BERT). The fine-tuned GPT model was built on the pre-trained generative pretraining(GPT) model. The fine-tuned BERT model was constructed with BERT model. Theexperimental results on the HPV dataset demonstrated the efficacy of the three methods in thesentiment analysis of the HPV vaccination task. The experimental results on the HPV datasetdemonstrated the efficacy of the methods in the sentiment analysis of the HPV vaccination task. Thefine-tuned BERT model outperforms all other methods. It can help to find strategies to improvevaccine uptake.
Keywords: BERT; ELMo; GPT; HPV vaccines; social media; transfer learning.