Source-linked AI summary
Do Platform Migrations Compromise Content Moderation? Evidence from r/The_Donald and r/Incels
Manoel Horta Ribeiro, Shagun Jhaver, Savvas Zannettou, Jeremy Blackburn, Emiliano De Cristofaro, Gianluca Stringhini, Robert West
TL;DR
The paper examines whether community-level moderation remains effective when toxic communities migrate beyond mainstream platforms, and studies r/The_Donald and r/Incels after their Reddit bans. Using observational data on activity and content, it finds reduced activity and newcomer inflow, alongside increased toxicity and radicalization-related signals in r/The_Donald.
Problem
It is unclear whether community-level moderation remains effective across the broader Web when toxic communities migrate to alternative platforms or standalone websites.
Method
The paper conducts an observational study of r/The_Donald and r/Incels using activity and content signals from Reddit and their post-migration standalone websites.
Results
Moderation reduced posts, active users, and newcomers in both communities, while r/The_Donald showed increased toxicity, hostility, ingroup identification, and othering signals.
Takeaways & Limitations
Community-level moderation can reduce activity and newcomer attraction while coinciding with a more toxic and ideologically radical community in at least one case.
Takeaways & Limitations
Real-world events create confounders that can affect content- and activity-related signals, although the reported effects held when analysis was limited to the period before George Floyd’s killing.
Abstract
from arXiv · showhide
When toxic online communities on mainstream platforms face moderation measures, such as bans, they may migrate to other platforms with laxer policies or set up their own dedicated websites. Previous work suggests that within mainstream platforms, community-level moderation is effective in mitigating the harm caused by the moderated communities. It is, however, unclear whether these results also hold when considering the broader Web ecosystem. Do toxic communities continue to grow in terms of their user base and activity on the new platforms? Do their members become more toxic and ideologically radicalized? In this paper, we report the results of a large-scale observational study of how problematic online communities progress following community-level moderation measures. We analyze data from r/The_Donald and r/Incels, two communities that were banned from Reddit and subsequently migrated to their own standalone websites. Our results suggest that, in both cases, moderation measures significantly decreased posting activity on the new platform, reducing the number of posts, active users, and newcomers. In spite of that, users in one of the studied communities (r/The_Donald) showed increases in signals associated with toxicity and radicalization, which justifies concerns that the reduction in activity may come at the expense of a more toxic and radical community. Overall, our results paint a nuanced portrait of the consequences of community-level moderation and can inform their design and deployment.
1 INTRODUCTION
The paper asks whether community-level moderation remains effective after toxic communities migrate beyond the original platform. Studying r/The_Donald and r/Incels, it finds reduced activity and membership alongside increased toxicity and radicalization signals in r/The_Donald.
- Research questions: The study examines whether moderated toxic communities retain activity, attract members, or become more toxic and ideologically radical after migrating.These dimensions determine whether moderation remains effective across the broader Web ecosystem.
- Approach: The authors analyze over 6 million posts from more than 138 thousand users across Reddit and standalone websites.They combine activity signals with toxicity-related content signals and quasi-experimental matching and regression-discontinuity analyses.
- Activity outcomes: Moderation significantly reduced posts, active users, and newcomers on the new platforms, although migrated users posted more often in aggregate.Matched-user analysis attributes the relative increase mainly to self-selection: migrants were more active on Reddit, then reduced their individual activity.
- Content outcomes: Users in r/The_Donald became more toxic, negative, hostile, and othering toward objects of fixation, while changes in r/Incels were generally statistically non-significant.Pronoun changes also indicated stronger ingroup identification and othering language in r/The_Donald.
- Implications: The findings suggest moderation can reduce toxic communities’ activity and recruitment while potentially making one studied community more toxic and ideologically radical.The authors therefore frame moderation consequences across the Web as a trade-off requiring evidence-based deployment.
2 BACKGROUND AND RELATED WORK
The paper situates its study in Reddit’s quarantine and banning practices, the migration histories of r/The_Donald and r/Incels, and prior research on toxicity and radicalization. It addresses a gap in cross-platform evidence about whether moderation effects persist after communities move to standalone websites.
- Reddit moderation: Reddit quarantine removes communities from search and the front page, while banning makes them inaccessible and removes prior posts.Quarantining commonly precedes banning as a warning for communities to reform.
- Communities of interest: r/The_Donald migrated to thedonald.win after restriction and banning-related events, while r/Incels migrated to incels.co after Reddit banned the subreddit.Both migrations were organized with community involvement and public standalone websites.
- Study scope: The communities were selected because of their large memberships, societal impact, and leader-backed migration to public websites.Public migrations avoided additional challenges associated with studying dispersed private channels.
- Conceptual background: The study’s broader conceptual frame connects online toxicity with automated indicators of radicalization, including fixation and group identification.These behaviors are relevant because online fringe movements can involve harassment, hateful views, and real-world violence.
- Research gap: The paper extends prior within-Reddit research by evaluating community-level moderation through the subsequent development of communities outside their original platform.The study focuses on whether migrated communities retain activity and develop differently after moderation.
3 MATERIALS AND METHODS
The study compares activity and content before and after migration using Reddit data, custom crawls of two standalone websites, cross-platform user matching, newcomer counts, and text-based toxicity and radicalization signals.
- Data collection: The dataset combines Reddit observations before moderation with standalone-website observations after migration.The analysis covers r/The_Donald and r/Incels alongside thedonald.win and incels.co.
- Data collection: Researchers collected around 3 million Reddit comments in 260K submissions and over 2.9 million standalone-website comments and submissions.The standalone data include over 2.5 million items from thedonald.win and over 400K from incels.co.
- User analysis: Matched-user analyses identify users with the same usernames on Reddit and the standalone websites to examine individual-level changes.The matching assumes these usernames represent the same individuals, supported by community evidence and prior research.
- User analysis: Newcomers are estimated by counting daily posts from usernames never previously observed across the relevant Reddit and standalone-platform histories.Additional Pushshift history serves as a buffer against misclassifying earlier users as newcomers.
- Content analysis: Content analysis uses fixation dictionaries, Perspective API Severe Toxicity scores, LIWC categories, and signals linked to fixation and group identification.The Severe Toxicity model scores posts from 0 to 1 for language likely to be rude, disrespectful, or unreasonable.
- Ethics and data scope: The study relies on publicly posted data and does not interact with users, simulate logged-in activity, or collect personal identities.Anonymized reproducibility data and code are reported as available.
4 CHANGES IN ACTIVITY LEVELS
Moderation measures substantially reduced community-wide activity and newcomer inflow after migration, while posts per user increased. User-level analyses indicate that this relative increase reflected self-selection rather than greater individual activity after migration.
- Community-level trends: Regression discontinuity analysis measures changes in daily posts, active users, and newcomers before and after migration, excluding a 15-day grace period around the intervention.The model allows post-migration changes in both the activity level and its slope.
- Community-level trends: Moderation significantly reduced newcomer inflow in both communities, although the supplied passage does not report the corresponding effect sizes.The analysis combines daily activity measures across Reddit and the fringe platforms.
- Community-level trends: Both communities experienced significant decreases in daily posts and active users after migration.TD lost around 14.4k daily posts and 4.7k daily active users; Incels lost around 2.6k posts and 777 daily active users.
- Community-level trends: Posts per user increased by 1.1 for TD and 7.4 for Incels, despite lower overall activity and fewer users.These increases represented 31% and 123% of the respective MVBI values.
- User-level trends: Across all users, mean posts per user were higher on fringe platforms than on Reddit: 73 versus 37.3 for TD and 180.6 versus 19.8 for Incels.These comparisons use the reported 95% confidence intervals for each platform and community.
- User-level trends: Matched users reduced their activity after migration, with mean log-ratios of -0.81 for TD and -0.53 for Incels.The decrease was strongest among users who had been most active before migration, supporting self-selection as the explanation for higher posts per user.
5 CHANGES IN CONTENT
The analysis examines content changes at both community and user levels after migration, finding that TD became more toxic and group-oriented while Incel changes were generally non-significant. Matched-user analyses indicate that TD’s increases were not explained solely by self-selection.
- Analytical approach: Community- and user-level trends were analyzed separately using content signals related to toxicity, fixation, emotions, hostility, and pronoun usage.Community-level trends use a regression discontinuity design with post-length controlled as a covariate.
- Community-level trends: TD showed a 0.2-percentage-point decrease in fixation-dictionary usage, while Incels showed a 0.4-percentage-point decrease; both had post-migration trend increases of 0.002.The decreases represented 44% of the TD MVBI and 21% of the Incel MVBI.
- Community-level trends: TD experienced a 0.9-percentage-point increase in toxic posts containing fixation terms, followed by a declining trend of 0.006 percentage points per day.The post-migration increase represented 42% of the MVBI, and the later decline could still leave toxicity above its pre-migration baseline.
- Community-level trends: TD’s first-person plural usage increased by 0.3 percentage points while third-person plural usage decreased by 0.3 points before their trends reversed, indicating changing group-identification and othering signals.The first-person plural trend changed by -0.002, whereas the third-person plural trend changed by +0.005.
- Cross-community comparison: Content effects differed across communities: TD showed significant increases in radicalization-related signals, whereas Incel changes were often decreases or statistically non-significant.The TD findings remained after excluding the period following 24 May 2020, despite the potential confound of George Floyd’s death.
- User-level trends: Matched TD users showed increases in Toxicity of 32%, We usage of 20%, and They usage of 8%, whereas Incel changes were generally non-significant.Matched-user results also found significant increases in Hostility for both communities and in Negative Emotions for TD.
- User-level trends: Matched-user analyses support behavior change in TD beyond self-selection, although matched users were more active than average and may differ from migrants who changed usernames.The matched sample reproduced several community-level coefficients and suggested that these users became more radical.
6 DISCUSSION & CONCLUSION
Community-wide moderation can reduce activity and newcomer growth while producing a smaller, potentially more extreme community elsewhere. These findings are informative but remain bounded by the study’s two-community scope, migration setting, confounders, and imperfect mapping from online signals to real-world harm.
- Implications for online platforms: Moderation decreased activity and newcomer attraction, but the relative increase in posting reflected self-selection by initially more active migrants.The authors connect reduced activity with the distributional power of mainstream platforms and distinguish community-level activity changes from individual behavior change.
- Implications for online platforms: Mainstream platforms’ discoverability and distribution affordances may help toxic communities recruit members and shape the content they produce.The paper argues that platforms influence not only which content flourishes but also communities’ ability to grow and their resulting content.
- Limitations and future work: The study’s conclusions may not generalize across communities because migration effects can depend on community size, topic, and alternative-platform design.The authors identify analysis of other sanctioned communities as necessary for disentangling these contextual factors.
- Limitations and future work: The analysis captures only one moderator-endorsed destination per community, leaving dispersed migration across multiple fringe platforms insufficiently measured.The authors note that users may have moved to other platforms and that coordinated migration may not occur in every intervention.
- Limitations and future work: Real-world events and rule changes can confound pre- versus post-migration comparisons, although the reported effects persisted when analysis ended before George Floyd’s killing.The study controls for linear trends but acknowledges difficulty handling sudden content bursts tied to events.
- Limitations and future work: Username-matched users are more active than average and may differ from migrants who changed usernames, constraining user-level generalization.The authors use matched users to distinguish self-selection from behavior change and report similar content trends between matched and all users.
- Limitations and future work: Online activity and toxicity signals do not yet map clearly onto real-world harms, so the desirability of moderation trade-offs remains unresolved.The authors call for causal study of links among toxicity, activity, and harm to improve moderation decisions.
- Implications for online platforms: Banning a community may trade broad activity for a smaller but more extreme community elsewhere, although increased toxicity appeared in only one studied community.The authors urge nuance because the analysis covers only TD and Incels, and the toxicity increase was not observed in both.