Source-linked AI summary
Racism is a Virus: Anti-Asian Hate and Counterspeech in Social Media during the COVID-19 Crisis
Bing He, Caleb Ziems, Sandeep Soni, Naren Ramakrishnan, Diyi Yang, Srijan Kumar
TL;DR
Anti-Asian hate spread online during COVID-19, but its longitudinal dynamics and the role of counterspeech were insufficiently understood. The paper builds and analyzes COVID-HATE, combining a large Twitter dataset, social network, and hand-labeled classifier. It finds that counterspeech may discourage users from becoming hateful, while also documenting important scope limitations.
Problem
The study addresses limited knowledge of how anti-Asian hate spreads during a pandemic and how counterspeech affects that spread.
Method
The paper creates COVID-HATE, trains a text classifier on hand-labeled tweets, and conducts longitudinal, linguistic, statistical, and network analyses.
Results
The classifier achieves a macro-F1 score of 0.832, and counterspeech may lower the probability that neighboring users become hateful.
Takeaways & Limitations
Public counterspeech messaging campaigns may provide a potential approach for curbing hate speech on social media.
Takeaways & Limitations
The analysis uses coarse-grained labels, covers only English-language discussions, and does not disentangle external events such as the Atlanta shooting.
Abstract
from arXiv · showhide
The spread of COVID-19 has sparked racism and hate on social media targeted towards Asian communities. However, little is known about how racial hate spreads during a pandemic and the role of counterspeech in mitigating this spread. In this work, we study the evolution and spread of anti-Asian hate speech through the lens of Twitter. We create COVID-HATE, the largest dataset of anti-Asian hate and counterspeech spanning 14 months, containing over 206 million tweets, and a social network with over 127 million nodes. By creating a novel hand-labeled dataset of 3,355 tweets, we train a text classifier to identify hate and counterspeech tweets that achieves an average macro-F1 score of 0.832. Using this dataset, we conduct longitudinal analysis of tweets and users. Analysis of the social network reveals that hateful and counterspeech users interact and engage extensively with one another, instead of living in isolated polarized communities. We find that nodes were highly likely to become hateful after being exposed to hateful content. Notably, counterspeech messages may discourage users from turning hateful, potentially suggesting a solution to curb hate on web and social media platforms. Data and code is at http://claws.cc.gatech.edu/covid.
INTRODUCTION
The paper addresses limited knowledge of anti-Asian hate online during COVID-19 by studying hate and counterspeech together over time on Twitter. It introduces COVID-HATE and finds counterspeech may lower neighboring users’ probability of becoming hateful.
- Over 6,603 racially motivated hateful incidents targeted Chinese and broader Asian communities during the pandemic’s first year.
- Longitudinal analysis examines how hate and counterspeech co-evolve on Twitter as the pandemic progresses.The paper studies both message types together because they coexist and may influence each other.
- COVID-HATE contains over 206 million COVID-19-related tweets collected between January 15, 2020 and March 26, 2021.
- The accompanying user social network contains over 127 million nodes and 910 million edges.
- A classifier trained on 3,355 annotated tweets achieves a macro-F1 score of 0.832 for identifying hate and counterspeech.The dataset identifies 1,227,116 hate tweets and 1,154,289 counterspeech tweets.
- Social contagion analysis finds counterspeech tweets may lower neighboring nodes’ probability of becoming hateful.
COVID-HATE: AN ANTI-ASIAN HATE AND COUNTERSPEECH DATASET DURING COVID-19
COVID-HATE combines broad keyword-based collection of COVID-19 Twitter discussions with tweet filtering and a large user-network crawl. The resulting resource spans 14 months and includes hundreds of millions of tweets and over 127 million network nodes.
- COVID-HATE is presented as a dataset of anti-Asian hate and counterspeech tweets together with their associated social network.
- 42 COVID-19, hate, and counterspeech keywords and hashtags were used to prioritize comprehensive collection with high recall.The broad collection may include borderline-relevant tweets, which are removed during classifier-based filtering.
- Historical tweets were retrieved with Twitter’s Search API, while real-time collection began through the Streaming API on March 28, 2020.
- 206,348,565 English-language tweets from 23,895,911 users were collected between January 15, 2020 and March 26, 2021.Retweets were excluded so analysis focuses on original content.
- The crawled network contains 127,831,666 nodes and 910,630,334 edges.It was built from ego-networks of a randomly sampled subset of users who made COVID-19 tweets.
Annotating Anti-Asian COVID-19 Hate and Counterspeech
The paper uses hand annotation to distinguish anti-Asian hate, explicit COVID-19 counterspeech, and neutral or irrelevant tweets, then trains classifiers to label the larger dataset. BERT features perform best among the compared feature sets.
- Anti-Asian COVID-19 hate requires pandemic context, an Asian target, and intentional othering through hostility, derogation, abuse, or blame.The definition distinguishes hate from strong criticism that does not typify Asians as a distinct outgroup.
- COVID-19 counterspeech explicitly condemns anti-Asian racism or violence, or supports, defends, or expresses solidarity with Asian entities.It may be a direct reply or standalone tweet, but implicit and unspecified solidarity is excluded.
- Two annotators independently labeled 3,255 randomly sampled tweets after training and supervised disagreement review.The annotation process used three broad categories: hate, counterspeech, and neutral or irrelevant.
- BERT features outperform linguistic and hashtag features by a significant margin across the three classification tasks.Hashtag features are typically worst because they convey low semantic meaning.
- The annotators agreed on 68% of tweets, with Cohen’s Kappa scores of 0.448 for hate and 0.590 for counterspeech.Disagreements were removed, leaving 429 hate, 517 counterspeech, and 1,344 neutral tweets for the retained ground truth.
Anti-Asian Hate and Counterspeech Text Classifier
The paper trains classifiers on annotated tweets to identify anti-Asian hate, counterspeech, and neutral content, selecting BERT for labeling the broader dataset.
- The authors train classifiers using linguistic, hashtag, and BERT-based features to categorize tweets as anti-Asian hate, counterspeech, or neutral.The linguistic feature set contains 90 stylistic, metadata, and psycholinguistic features.
- Five-fold cross-validation is conducted on the hand-annotated dataset during model development.
- BERT embedding features significantly outperform linguistic and hashtag features on precision, recall, and F1 evaluation.Ensemble methods combining all three feature sets perform similarly to the BERT model.
- The BERT classifier is used to label the remaining tweets, yielding about 1.337 million hate tweets and 1.154 million counterspeech tweets.Its use also reduces reliance on hashtags and keywords whose popularity may change over time.
LONGITUDINAL CHARACTERIZATION OF COVID-19 HATE AND COUNTERSPEECH
The analysis examines how hate and counterspeech evolve and spread on Twitter, comparing activity during 2020 with the first part of 2021.
- The study analyzes hate and counterspeech evolution, spread, and user characteristics in the COVID-HATE Twitter ecosystem.Temporal statistics compare January 15–December 31, 2020 with January 1–March 26, 2021.
The Ebb and Flow of Hate and Counterspeech
Hate generally exceeded counterspeech during 2020, while counterspeech surged after the March 2021 Atlanta shooting and hate and counterspeech activity showed extended temporal variation.
- Hate tweets outnumbered counterspeech tweets throughout 2020, with several daily hate spikes exceeding counterspeech activity.Activity was negligible to low during January and February 2020, followed by a hate-speech spike from March 16–19.
- Counterspeech tweets increased by 401.2% within one week after the 2021 Atlanta Spa shooting.The increase followed a period when counterspeech volume was typically lower than hate volume during 2020.
- Figure 2 presents user-level distributions of hate and counterspeech tweet counts, which exhibit a long-tail pattern.
- Hateful tweets also rose by 17.9% after the Atlanta shooting, alongside the counterspeech surge.
- The dataset shows spikes in hate and counterspeech throughout the full 14-month period despite collection keywords being selected early in the pandemic.
User Activity and Interaction Behavior
The study categorizes users by their hate and counterspeech activity and examines their activity before becoming activated in either category.
- Users are categorized as hate, counterspeech, dual, or neutral according to whether their tweets include hate, counterspeech, both, or neither.
- Most users make few relevant tweets, while a small number produce most hate propaganda and counterspeech messages.Both hate-tweet and counterspeech-tweet user distributions show a long tail.
- Before activation, hateful users averaged 14.28 COVID-related tweets compared with 28.62 for counterspeech users.The difference was statistically significant under a Mann–Whitney U test (p < 0.001).
- Before activation, hate users wrote shorter tweets, used fewer URLs, tagged others less often, and had more neutral sentiment scores than counterspeech users.The reported character counts were 120.89 versus 160.38, and the URL counts were 0.413 versus 0.528.
Social Network Connectivity Structure
The hate and counterspeech ecosystem forms a highly connected network rather than isolated groups. Both groups preferentially connect within their own categories while also interacting extensively across groups.
- 1,380,613 nodes made at least one COVID-19-related tweet within a crawled network of over 127 million nodes and 910 million edges.
- Counterspeech users were better connected than hate users, both following and being followed by more users.They followed 1201.84 versus 828.40 users and were followed by 1249.42 versus 759.96 users.
- Counterspeech users were 6.92× more likely to connect with counterspeech users, while hateful users connected with hateful users 2.42× more than expected.
- Hateful and counterspeech users connected across groups more than expected, with hateful users following counterspeech users 4.45× more and counterspeech users following hateful users 1.62× more.
- The network analysis indicates that hateful and counterspeech users are highly engaged and closely interact rather than forming isolated polarized communities.
INFLUENCE OF COUNTERSPEECH ON THE SPREAD OF HATE
The study examines whether exposure to neighboring hate or counterspeech affects users’ transitions into hateful behavior. Hate exposure increased hate adoption, whereas counterspeech might deter it relative to a homophily-based baseline.
- The study measures a user’s likelihood of becoming hateful after exposure to hate or counterspeech tweets from neighbors.A neutral-to-hate or neutral-to-counterspeech transition is treated as an infection in a temporal event cascade.
- The infection-risk analysis separates user influence from homophily by comparing observed cascades with a shuffled-event null model.The null model estimates baseline infection risk attributable solely to homophily.
- Exposure to hate speech increased the likelihood of adopting hate speech relative to baseline, and adoption likelihood rose with the number of exposures.
- Counterspeech might deter the spread of hate speech relative to baseline, potentially indicating a low social inhibition effect.The authors note that the change over time may reflect counterspeech mitigation and external societal factors.
RELATED WORK
Prior work studied online hate, anti-Asian content, or counterspeech, but did not jointly analyze anti-Asian hate and counterspeech over time during the COVID-19 pandemic. This paper addresses that gap.
- Hate speech on social media: Existing hate-speech research used text-based and graph-based methods, including deep neural architectures, but did not jointly analyze anti-Asian hate and counterspeech over time during a pandemic.
- Hate on social media during COVID-19: Earlier COVID-19 studies collected anti-Asian hate or controversial-hashtag tweets, but hashtag presence was insufficient for reliably labeling hate.
- Counterspeech on social media: Counterspeech research developed datasets, classifiers, and longitudinal analyses, while quantitative evidence remained limited and COVID-19-related anti-Asian counterspeech was unstudied.
DISCUSSION AND CONCLUSIONS
The paper finds that counterspeech might reduce the probability of neighbors becoming hateful and considers public counterspeech campaigns a potential response to online hate. The authors qualify these findings with important scope limitations.
- Counterspeech might be effective in reducing the probability that neighbors become hateful.
- Public counterspeech messaging campaigns are presented as a potential solution against hate speech on social media.
- The annotation scheme is coarse-grained, the analysis covers only English-language discussions, and external events were not disentangled.The authors do not expect the coarse annotation to change the findings and plan to study external events in follow-up work.