Source-linked AI summary
Public sentiment analysis and topic modeling regarding COVID-19 vaccines on the Reddit social media platform: A call to action for strengthening vaccine confidence
Chad A Melton, Olufunto A Olusanya, Nariman Ammar, Arash Shaban-Nejad
TL;DR
Vaccine hesitancy and misinformation complicate public confidence in COVID-19 vaccination. This study analyzed Reddit discussions from 13 vaccine-focused communities using sentiment analysis and LDA topic modeling, finding more positive than negative sentiment that remained stable over time, alongside persistent hesitancy-related keywords.
Problem
The study examines public COVID-19 vaccine sentiment and vaccine hesitancy amid misinformation and reluctance to vaccinate.
Method
The study analyzed 11,641 Reddit items from 13 vaccine-focused communities using TextBlob sentiment analysis and monthly Gensim LDA topic models.
Results
Sentiment was overall more positive than negative and did not meaningfully change since December 2020, while LDA detected vaccine-hesitancy keywords.
Takeaways & Limitations
The findings support further work on messaging, digital interventions, and policies to promote vaccine confidence and address misinformation.
Takeaways & Limitations
TextBlob sentiment analysis can misclassify sarcasm and has reported accuracy of 50–70%, while Reddit provides limited demographic and geographic information.
Abstract
from arXiv · showhide
The COVID-19 pandemic fueled one of the most rapid vaccine developments in history. However, misinformation spread through online social media often leads to negative vaccine sentiment and hesitancy. To investigate COVID-19 vaccine-related discussion in social media, we conducted a sentiment analysis and Latent Dirichlet Allocation topic modeling on textual data collected from 13 Reddit communities focusing on the COVID-19 vaccine from Dec 1, 2020, to May 15, 2021. Data were aggregated and analyzed by month to detect changes in any sentiment and latent topics. ty analysis suggested these communities expressed more positive sentiment than negative regarding the vaccine-related discussions and has remained static over time. Topic modeling revealed community members mainly focused on side effects rather than outlandish conspiracy theories. Covid-19 vaccine-related content from 13 subreddits show that the sentiments expressed in these communities are overall more positive than negative and have not meaningfully changed since December 2020. Keywords indicating vaccine hesitancy were detected throughout the LDA topic modeling. Public sentiment and topic modeling analysis regarding vaccines could facilitate the implementation of appropriate messaging, digital interventions, and new policies to promote vaccine confidence.
Introduction
The study frames COVID-19 vaccine hesitancy as a multifaceted public-health challenge linked to misinformation, safety concerns, and uncertainty. It examines Reddit discussions to assess vaccine sentiment and detect evidence of hesitancy.
- COVID-19 vaccine hesitancy includes delayed acceptance or refusal despite vaccine availability.
- Reported drivers include misinformation, disinformation, conspiracy beliefs, political ideology, safety concerns, side effects, efficacy, access, and fear of the unknown.
- Misinformation and disinformation can spread faster than evidence-based vaccine information and are associated with negative vaccination outcomes.
- Assessing public sentiment is presented as important for understanding vaccine acceptance and informing messaging, interventions, and policies.
- The study investigates whether Reddit vaccine discussions reflect public sentiment and whether they contain evidence of vaccine hesitancy.
Background
Prior work established social media as a source for monitoring public sentiment and health-related discussion, while this study focuses on vaccine-centered Reddit communities. The dataset was assembled from posts and comments collected, cleaned, and organized for analysis.
- Background: Sentiment analysis computationally classifies written text as positive, negative, or neutral polarity.
- Background: Social media provides large volumes of semantically rich text for monitoring public opinion despite concerns about validity, representativeness, confounding, and bias.
- Background: Earlier research used NLP to monitor vaccination, disease occurrence, public-health measures, and pandemic-related opinions across multiple platforms and countries.
- Background: The study addresses prior Reddit research by selecting communities directly focused on COVID-19 vaccines.
- Data source: Researchers combined subreddit data, organized it by date, and queried terms related to COVID-19 vaccination before analysis.
- Data source: The finalized dataset contained 1,401 posts and 10,240 comments, totaling 11,641 items from at least 8,281 authors.
Analytical methods
The study combines lexical sentiment analysis with LDA topic modeling to characterize vaccine-related Reddit text over time. Sentiment scores classify polarity and subjectivity, while LDA estimates latent topic distributions.
- Analytical methods: Lexical sentiment analysis uses dictionaries with preassigned valence scores to calculate sentiment efficiently.
- Analytical methods: Subjectivity scores range from 0 to 1, with values between 0.4 and 0.6 classified as neutral.
- Analytical methods: Polarity scores range from −1.0 to 1.0, representing the most negative and most positive values respectively.
- Analytical methods: The Gensim LDAModel algorithm estimates topic proportions within documents and word proportions across topics until convergence.
- Analytical methods: Researchers modeled combined and monthly datasets, then conducted additional LDA analyses within positive, negative, and neutral polarity groups.
Results
Across the Reddit dataset, vaccine-related posts were more often positive than negative, and LDA identified five broad topics. These topics emphasized vaccine safety, efficacy, side effects, and misinformation-related discussions.
- Results: 56.68% of posts were positive, 27.69% negative, and 15.63% neutral in the combined polarity analysis.
- Results: The combined LDA dataset yielded five optimal latent topics.
- Results: Topics 1–4 centered on vaccine discussions involving safety, efficacy, and side effects ranging from fever and soreness to death.
- Results: Topic 5 contained discussions of vaccine misinformation and autism, including both sarcastic and believing responses.
Monthly analysis
Monthly Reddit discussions remained predominantly positive, though sentiment was least positive and most negative in May. Monthly topics broadly resembled the combined dataset while showing changing emphasis on dosage, side effects, hesitancy, and death.
- Sentiment over time: Most monthly posts were positive, with May recording the lowest positive and highest negative sentiment.May reported 53.05% positive, 30.57% negative, and 16.38% neutral sentiment.
- Latent topics over time: Monthly latent topics generally resembled the complete dataset, although each month contained no more than three topics.December differed most, emphasizing trials, efficacy, and potential side effects.
- Latent topics over time: January and February topics emphasized dosage, immunity, and side effects, while March through May more directly referenced hesitancy, risk, and death.April and May also included discussion of T cells.
Sentiment topic modeling
Polarity-specific topic models shared themes around vaccination, side effects, concerns, timing, and immunity, while negative, neutral, and positive posts showed distinct associated terms.
- Common themes: Across polarity-specific models, common themes concerned the vaccination process, side effects, concerns, time, and immunity.The polarity-specific models were more convoluted than the combined and monthly models.
- Negative polarity: Negative posts additionally contained terms related to government, state, science, employees, risks, and expletives.These terms distinguished negative-topic content from the shared themes.
- Neutral polarity: Neutral posts included terms associated with physicians, Pfizer, research, videos, links, issues, and stories.The topic model grouped these terms with neutral-classified posts.
- Positive polarity: Positive posts included terms related to Moderna, safety, pregnancy, family, responses, and death.The positive topics also contained expletives.
Interpretation
The authors interpret the Reddit discussions as broadly positive but containing vaccine-hesitancy signals, while cautioning that sentiment may reflect interaction patterns and community composition rather than users’ actual feelings.
- Interpretation: Overall positive polarity coexisted with keywords and topics indicating some vaccine hesitancy.The authors report that sentiment did not change significantly during the study interval.
- Interpretation: Topic keywords centered on side effects, reactions, doses, immunity, risk, pregnancy, and longer-term concerns.These themes included terms such as reaction, fever, efficacy, herd immunity, pregnancy, and long-term effects.
- Interpretation: The sentiment signal may reflect user interaction patterns because authors numbered about 9,000, whereas comment upvotes reached 612,217.The broader community included approximately 4.9 million members who mainly consumed content without interacting.
- Interpretation: Qualitative review found that sarcasm could receive positive polarity despite expressing vaccine refusal.The authors therefore caution against equating automated polarity directly with actual vaccination attitudes.
Limitations
The study’s sentiment results are constrained by difficulty detecting sarcasm and by limited demographic and geographic information in Reddit data.
- Sentiment-analysis limitations: TextBlob may misclassify sarcasm, and the authors report modest accuracy of 50–70%.A sarcastic comment was classified as negative, illustrating a source of false positives and false negatives.
- Data limitations: Reddit lacks sufficiently detailed demographic and geocoded information for highly specific geographic or demographic studies.This prevents direct comparison with regional or citywide polling unless a subreddit is explicitly geographic or demographic.
Social media and digital health technologies
Scientific consensus identifies COVID-19 vaccines as protective, while misinformation complicates vaccine-related health messaging. Social-media text analysis can support targeted messaging, digital interventions, and policies.
- Misinformation and fake news complicate accurate COVID-19 vaccine messaging despite scientific consensus that vaccines are protective.The paper links social-media misinformation with challenges in communicating evidence-based health information.
- Social-media textual data enable rapid and inexpensive public sentiment analysis.
- These analyses could facilitate appropriate messaging, digital interventions, and policies.
- Sentiment-informed digital health tools could deliver personalized messages and education based on individuals’ social-media posts.
Conclusion
Across 13 subreddits, COVID-19 vaccine discussions were more positive than negative and did not meaningfully change since December 2020. LDA nonetheless detected vaccine-hesitancy keywords, supporting further work on misinformation and negative sentiment.
- LDA topic modeling detected keywords indicating vaccine hesitancy throughout the analyzed content.
- The authors call for additional research to reach populations with negative vaccine sentiment and combat misinformation.