Source-linked AI summary
From Detection to Characterization: A Large-Scale Study of Ragebait on Japanese X
Zhiyang Qi, Kazuhiro Ito, Jinghui Chen, Hibiki Nakamura, Zhangxuan Chen, Erina Murata, Masaki Chujyo, Fujio Toriumi
TL;DR
Reliable large-scale ragebait detection remains limited because ragebait depends on provocative intent and anticipated audience reactions rather than negative language alone. The study uses LLM-assisted labeling to train an ensemble detector for Japanese-language X posts, finding that ragebait is concentrated in contentious topics, spreads faster, and receives more negative emotional reactions than non-ragebait.
Problem
Reliable large-scale ragebait detection and systematic characterization remain limited, while identifying ragebait requires distinguishing provocative intent from negative language alone.
Method
The study builds an LLM-generated pseudo-labeled dataset, validates a sampled subset with human annotators, trains multiple Japanese classifiers, and combines them through majority voting for large-scale X analysis.
Results
Ragebait is more prevalent in politically and socially contentious topics, spreads faster, and receives more anger, fear, disgust, sadness, and surprise in replies and quotes than non-ragebait.
Takeaways & Limitations
The findings support ragebait as a distinct form of emotionally provocative content and provide a practical basis for large-scale analysis and future mitigation efforts.
Abstract
from arXiv · showhide
Ragebait refers to online content intentionally designed to provoke anger or outrage and thereby increase attention and engagement. However, reliable large-scale detection and systematic analysis of ragebait remain limited, hindering efforts to understand its prevalence, impact, and mitigation. This study aims to develop an effective ragebait detection framework and to clarify the characteristics of ragebait at scale, providing a basis for understanding and mitigating emotionally provocative content online. We constructed a labeled dataset with the assistance of a large language model (LLM) and trained several Japanese language models for ragebait detection. The resulting ensemble classifier was then applied to a large-scale dataset of Japanese-language posts on X. Our analysis shows that ragebait is more prevalent in politically and socially contentious topics, including politics, discrimination, public health, and interpersonal conflict. Ragebait posts also spread faster and receive more negative reactions than non-ragebait posts, particularly anger, fear, disgust, sadness, and surprise. These findings demonstrate the utility of the proposed detector and provide a large-scale characterization of ragebait in Japanese online discourse.
I. INTRODUCTION
Ragebait is intentionally provocative content whose scalable detection remains difficult because it depends on communicative intent and anticipated audience reactions, not simply negative language. This study addresses that gap by building a Japanese X detector and applying it to large-scale posts to characterize prevalence, topics, diffusion, and reactions.
- Ragebait deliberately provokes anger or outrage to attract attention and increase engagement.
- Existing ragebait research is limited, especially for general user-generated social media posts beyond professionally produced news headlines.
- Detecting ragebait requires assessing poster intent and likely reader reactions because negative sentiment, offensiveness, or anger alone are insufficient.
- The study constructs an 18,558-instance balanced pseudo-labeled dataset using an LLM and trains multiple Japanese classifiers combined by majority voting.The ensemble achieves 84.05% accuracy and an 84.04% Macro-F1 score.
- The ensemble is applied to more than 150 million Japanese-language X event records from October 2022 to June 2023 to analyze ragebait prevalence and characteristics.The analysis covers lexical characteristics, thematic content, diffusion dynamics, and emotional reactions.
- Ragebait is more common in contentious topics, spreads faster, and receives replies and quotes with more anger, fear, disgust, sadness, and surprise than non-ragebait.These converging patterns support ragebait as a meaningful category distinct from negative language generally.
II. RELATED WORK
Prior work connects ragebait to clickbait and abusive content but distinguishes it by intentional provocation and anticipated audience reaction. Related research also links anger and moral outrage to online engagement, diffusion, and social amplification, while LLMs offer scalable annotation with potential bias concerns.
- Ragebait differs from clickbait, offensive language, hate speech, and toxic content because it incorporates intent to provoke and anticipated reader reactions.
- Anger and moral outrage are associated with increased online engagement, information diffusion, sharing, and wider or deeper cascades.
- LLM-generated labels can support scalable text classification, but subjective labels may be affected by annotation bias and model suggestions.
- The study uses LLM pseudo-labels, independent human validation, and separate encoder-based classifiers for large-scale ragebait detection.
III. DATASET CONSTRUCTION
The dataset construction procedure combines filtered Japanese X data, LLM labeling, balanced sampling, and an initial detector. Because ragebait is rare and costly to identify manually, the procedure uses detector-assisted retrieval to obtain additional likely positive examples.
- The two-stage procedure addresses costly intent-sensitive annotation and the low prevalence of ragebait in random social-media samples.
- The source comprises 153,849,869 event records from a 1% Japanese-language X sample collected between October 2022 and June 2023.Filtering retains posts with at least 20 characters and more than 50 reposts, likes, replies, or quotes in at least one category.
- GPT-5.4 mini labels 20,000 randomly sampled posts as ragebait or non-ragebait based on apparent provocation intent and likely emotional reactions.
- The seed dataset balances all 822 LLM-labeled ragebait posts with an equal number of randomly sampled non-ragebait posts.
- The balanced seed dataset contains 1,644 posts, split into 1,444 training instances and 200 test instances, with equal class counts in both sets.
- The initial Rinna RoBERTa base classifier reaches 0.74 accuracy on the balanced test set and is used only to retrieve candidates, not for final analyses.
D. Detector-Assisted Data Expansion
The initial detector expands the dataset by prioritizing posts that are both likely to be ragebait and sufficiently visible for diffusion and reaction analysis. Reclassification of these candidates substantially increases the positive-instance proportion.
- Posts are selected for expansion using high predicted ragebait probability together with high engagement counts.This prioritizes likely ragebait that is visible enough to generate observable diffusion and audience reactions.
- 8,747 of 30,000 detector-selected candidates are labeled ragebait in the second GPT-5.4 mini classification stage.The remaining 21,253 candidates are labeled non-ragebait.
- The increased positive proportion relative to the initial random sample indicates that the initial detector effectively retrieves likely ragebait candidates.
E. Final Dataset
The final dataset combines balanced ragebait and non-ragebait examples, with independent human evaluation used to assess GPT-generated labels before training Japanese detection models.
- The final dataset contains 18,558 posts, split evenly between 9,279 ragebait and 9,279 non-ragebait instances.It uses 16,558 posts for training and 2,000 for held-out testing, with balanced class distributions in both sets.
- Two annotators independently labeled a random sample of 200 posts without access to GPT-generated labels.The sample included 100 GPT-labeled ragebait posts and 100 non-ragebait posts.
- Human judgments considered authorial intent, personal emotional response, and whether other readers would likely react similarly.These criteria jointly address communicative intention and anticipated emotional effects.
- Human–GPT agreement was 75.0% and 74.5%, with Cohen’s κ values of 0.500 and 0.498, while human–human agreement reached 78.5% with κ = 0.570.The authors describe GPT labels as reasonably reliable for training data, but not equivalent to manually established gold-standard labels.
- Six Japanese pretrained language models were trained for binary ragebait classification using the constructed dataset.The models included BERT, RoBERTa, DistilBERT, and DeBERTa variants.
B. Classification Results
Individual Japanese language models performed similarly overall, while a majority-voting ensemble improved classification performance and robustness relative to any single model.
- Tohoku BERT base v3 achieved the highest individual Accuracy of 83.40% and Macro-F1 of 83.38%.It also achieved the highest Recall of 87.00%, whereas Rinna RoBERTa base achieved the highest Precision of 83.10%.
- The three best individual models used different architectures and pretraining resources, potentially producing different error patterns.The authors motivate ensembling as a way to reduce sensitivity to model-specific biases and occasional misclassifications.
- The final ensemble combines Tohoku BERT base v3, Rinna RoBERTa base, and LINE DistilBERT base using majority voting.
- 84.05% ensemble Accuracy and 84.04% Macro-F1 outperformed all individual models on both metrics.Precision was 82.65% and Recall was 86.20%, indicating a balance between detecting ragebait and limiting false positives.
V. LARGE-SCALE ANALYSIS OF POSTS ON X
The study applies its ensemble detector to a large Japanese-language X dataset and examines lexical differences between ragebait and non-ragebait posts.
- The large-scale dataset contains 153,849,869 event records from a 1% sample of Japanese-language X posts collected between October 2022 and June 2023.Original posts linked to repost events were extracted and deduplicated to preserve engagement counts observed over time.
- The analysis examines lexical characteristics, thematic patterns, and diffusion speed in the X data.
- Word-cloud analysis compares within-group token frequencies after morphological preprocessing and removal of highly general terms.Weighted log-odds z-scores additionally identify terms overrepresented in one group relative to the other.
- Ragebait posts more frequently contain terms related to politics, social identity, public health, and social conflict.Examples include “Japan,” “women,” “LDP,” “vaccine,” “discrimination,” “China,” “crime,” and “LGBT.”
B. Differences in Topic Prevalence
Topic modeling shows that ragebait is concentrated in socially contentious themes, whereas non-ragebait posts are more prevalent in everyday, recreational, and informational topics.
- Structural Topic Modeling compares topic prevalence between ragebait and non-ragebait posts while accounting for temporal variation.The analysis samples 100,000 posts from each group and uses the ragebait label and posting month as metadata.
- The final Structural Topic Model uses K = 25 topics selected using held-out likelihood, exclusivity, and semantic coherence.
- Ragebait has higher prevalence in party politics, discrimination, vaccines, interpersonal conflict, public spending, welfare, and litigation.Non-ragebait is more prevalent in sports, daily interaction, creative production, anime and games, and information exchange.
- 52.22% of mean topic prevalence in ragebait posts comes from four contentious topics, compared with 14.78% in non-ragebait posts.The four topics are party politics and governance, interpersonal conflict and moral condemnation, vaccines and infectious diseases, and gender and discrimination.
C. Diffusion Dynamics
Ragebait and non-ragebait posts were compared by normalized engagement growth over time across reposts, likes, replies, and quotes. Ragebait showed the clearest sustained advantages for replies and quotes, while repost differences were smaller and likes were similar.
- Measurement: Engagement growth was measured for reposts, likes, replies, and quotes, normalized as median growth per 10,000 followers over time.The analysis tracked original posts observed at least five times and compared groups within elapsed-time bins.
- Overall pattern: Growth rates were highest immediately after posting and gradually declined for all four engagement indicators.The temporal comparison covers reposts, likes, replies, and quotes.
- Reposts: Ragebait posts showed slightly higher repost growth after the first hour and across several later time bins.The advantage was smaller than the differences observed for replies and quotes.
- Likes: Like-growth trajectories were very similar for ragebait and non-ragebait posts, with only small differences throughout the observed period.Likes showed the least separation between the two groups.
- Replies and quotes: Ragebait posts had consistently higher reply and quote growth from the first few hours after posting through several days later.These indicators often reflect discussion, disagreement, or reactive engagement, extending beyond passive exposure.
D. Emotional Reactions in Replies and Quotes
Reactions to ragebait posts were substantially more negative than reactions to non-ragebait posts. They also contained higher rates of sadness, surprise, anger, fear, and disgust, while joy, anticipation, and trust were more frequent for non-ragebait posts.
- Binary sentiment: 52.02% of replies to ragebait were negative, compared with 21.44% for replies to non-ragebait.For quote posts, the corresponding rates were 63.49% and 22.74%.
- Binary sentiment: 63.49% of quotes directed at ragebait were negative, compared with 22.74% of quotes directed at non-ragebait.The difference was larger for quotes than for replies.
- Fine-grained emotions: Replies and quotes directed at ragebait contained higher rates of sadness, surprise, anger, fear, and disgust than reactions to non-ragebait.The emotion model classified eight basic emotions independently, allowing multiple labels per reaction.
- Fine-grained emotions: Anger occurred in 26.58% of replies and 35.53% of quotes to ragebait, versus 4.27% and 6.40% for non-ragebait.The contrast was especially clear for anger and disgust.
- Fine-grained emotions: Disgust occurred in 57.18% of replies and 69.46% of quotes to ragebait, versus 19.41% and 21.18% for non-ragebait.Joy, anticipation, and trust were more frequent in reactions to non-ragebait posts.
VI. CONCLUSION
The study developed and evaluated a Japanese-language ragebait detector, then used it for large-scale analysis on X. Ragebait was uncommon but concentrated in contentious topics and associated with stronger reply and quote growth and more negative emotional reactions.
- Conclusion: The ensemble classifier achieved 84.05% accuracy and a Macro-F1 score of 84.04%.It was trained using an LLM-generated pseudo-labeled dataset of approximately 18K posts validated through independent human evaluation.
- Conclusion: Ragebait accounted for only a small proportion of the analyzed posts.The detector was applied to large-scale Japanese-language X data.
- Conclusion: Ragebait was more prevalent in politically and socially contentious topics and showed stronger growth in replies and quotes.These patterns distinguish ragebait from ordinary negative or offensive language.
- Conclusion: Reactions to ragebait contained more negative sentiment and higher rates of anger, fear, and disgust than reactions to non-ragebait.The conclusion summarizes the binary sentiment and fine-grained emotion comparisons.
- Conclusion: Although relatively uncommon, ragebait appeared to have a disproportionate capacity to stimulate conflict-oriented interaction and negative emotional responses.The proposed detector provides a practical basis for large-scale analysis and future mitigation efforts.