Source-linked AI summary
The Structure of Toxic Conversations on Twitter
Martin Saveski, Brandon Roy, Deb Roy
TL;DR
Prior toxicity detection has often analyzed content in isolation and after the fact, while less attention has been paid to how conversational and social structure relate to toxic behavior. The authors analyze Twitter conversations at individual, dyad, and group levels and test whether structure forecasts toxicity. They find structural associations with toxicity and show that structure predicts both early conversation outcomes and users' next replies.
Problem
Prior work often detects antisocial behavior from isolated content after the fact, and conversation-structure research has paid less attention to its relationship with toxicity.
Source: §1 · p. 1Method
The authors analyze 1.18 million Twitter conversations involving major news outlets and 2018 US midterm candidates at individual, dyad, and group levels, then test two toxicity-prediction tasks.
Source: §1 · pp. 1–2Results
Toxicity is dispersed among low- to moderately toxic users, associated with weaker dyadic ties and larger, wider, deeper reply trees, while structural features predict conversation toxicity and users' next toxic replies.
Source: §1 · p. 2Takeaways & Limitations
The findings inform healthier-platform design and suggest that early toxicity predictions could help rank conversations or alert their initiators to possible derailment.
Source: §7 · p. 11The study focuses mostly on political conversations on Twitter and identifies associations and predictive power, leaving causal effects and generalization to other domains for future work.
Source: §7 · p. 12
Abstract
from arXiv · showhide
Social media platforms promise to enable rich and vibrant conversations online; however, their potential is often hindered by antisocial behaviors. In this paper, we study the relationship between structure and toxicity in conversations on Twitter. We collect 1.18M conversations (58.5M tweets, 4.4M users) prompted by tweets that are posted by or mention major news outlets over one year and candidates who ran in the 2018 US midterm elections over four months. We analyze the conversations at the individual, dyad, and group level. At the individual level, we find that toxicity is spread across many low to moderately toxic users. At the dyad level, we observe that toxic replies are more likely to come from users who do not have any social connection nor share many common friends with the poster. At the group level, we find that toxic conversations tend to have larger, wider, and deeper reply trees, but sparser follow graphs. To test the predictive power of the conversational structure, we consider two prediction tasks. In the first prediction task, we demonstrate that the structural features can be used to predict whether the conversation will become toxic as early as the first ten replies. In the second prediction task, we show that the structural characteristics of the conversation are also predictive of whether the next reply posted by a specific user will be toxic or not. We observe that the structural and linguistic characteristics of the conversations are complementary in both prediction tasks. Our findings inform the design of healthier social media platforms and demonstrate that models based on the structural characteristics of conversations can be used to detect early signs of toxicity and potentially steer conversations in a less toxic direction.
1 INTRODUCTION
Twitter conversations can connect people with little prior social connection, but antisocial behavior can damage mental and emotional health. This study examines how conversational and social structure relate to toxicity and whether structure can help forecast it as conversations unfold.
1 INTRODUCTION
- Prior antisocial-behavior detection often analyzes content in isolation and after the fact, while conversation-structure research has paid less attention to toxicity.These approaches have limited potential for forecasting or preventing toxic behavior, motivating analysis of social and conversational context.Source: §1 · p. 1
- The study analyzes 1.18 million conversations involving major news outlets and 2018 US midterm candidates to examine structure and toxicity.The collection includes 58.5 million tweets and 4.4 million users.Source: §1 · p. 1
- The analysis spans individual, dyad, and group levels, alongside two tasks that forecast conversation-level toxicity and a specific user's next reply.The conversation-level task uses early conversation stages; the user-level task considers the conversation so far and the user's relationships with participants.Source: §1 · p. 2
- Toxicity is dispersed among many low- to moderately toxic users; toxic replies also tend to come from users with weaker social ties to the poster.At the group level, toxic conversations tend to have larger, wider, and deeper reply trees, while denser follow graphs with fewer components and lower modularity tend to be less toxic.Source: §1 · p. 2
- Structural features predict conversation-level toxicity from the first ten replies and whether a specific user's next reply will be toxic.Combining structural and linguistic features further improves performance, suggesting they capture complementary aspects of conversations.Source: §1 · p. 2
- The findings suggest platform-design changes that could reduce toxicity at scale.Twitter replies can connect people who are closely connected or have little to no connection on the platform.Source: §1 · pp. 1–2
2 DATA
The dataset combines conversations prompted by major news outlets and 2018 US midterm candidates. The collection process reconstructs reply trees and captures participants' follow relationships over time.
2 DATA
- The study selected five politically diverse major news outlets and candidates who ran in the 2018 US midterm elections.Conversations were rooted in tweets posted by or mentioning the selected accounts.Source: §2 · p. 2
- The news and candidate collections together contain 1.18 million conversations, gathered over different periods around news coverage and the 2018 US midterms.The news accounts were tracked from May 2018 to May 2019; candidate accounts were tracked for five months around the election.Source: §2 · p. 2
- Because the Twitter API linked replies to originals but not originals to all replies, the researchers scanned collected posts and recursively linked replies to build conversation trees.They retained trees rooted in tweets posted by or mentioning selected accounts, excluding tweets without replies and reply strings by only one user.Source: §2 · p. 3
- Daily collection of users' friends and followers provided follow-graph snapshots corresponding to when they tweeted; protected accounts were not collected.The researchers downloaded each participating user's friend and follower lists through the Twitter REST API.Source: §2 · p. 3
3 TOXICITY ANNOTATIONS
Toxicity labels came from Google's Perspective API, with human annotations used to assess predictions and tune the classification threshold. The authors report test performance and stronger scores on tweets with annotator consensus.
3 TOXICITY ANNOTATIONS
- The researchers used Google's Perspective API to label toxicity and assessed its annotation quality with human judgments.The API estimates the probability that a reader would perceive a comment as toxic rather than returning a binary label.Source: §3 · p. 3
- Annotators labeled tweets as toxic or nontoxic using the Perspective API definition, with US workers selected for prior task performance.Each task contained five tweets, and the annotation protocol was approved by the MIT institutional review board.Source: §3 · p. 3
- Three workers independently labeled each of 3,000 sampled tweets, and majority vote supplied the single ground-truth label.The sample was representative of the distribution of Perspective API toxicity scores; annotator agreement was fair, with Krippendorff's α = 0.32.Source: §3 · p. 3
- On the test set, threshold T = 0.531 yielded accuracy 0.82, AUC 0.86, and F1 0.63.For the subset with annotator consensus, accuracy was 0.91, AUC 0.95, and F1 0.73.Source: §3 · p. 3
4 ANALYSES
Across individual, dyadic, and group levels, toxicity is associated with user activity and social ties, as well as the structure of reply trees and participant follow graphs.
4.1 Individual Level
- Toxicity is dispersed across many low-to-moderately toxic users; moderately active users have higher toxic-tweet fractions than low- or high-activity users.The highly active users’ average toxic-tweet fraction is lower than that of low-activity users, although estimates for highly active users have wider confidence intervals.Source: §4.1 · p. 4
- Users show moderate homophily by toxic-tweet category, but highly toxic users show no evidence of preferentially connecting to one another.Assortativity is 0.15 in news and 0.125 in midterms for users with no toxic tweets versus at least one; by number of toxic tweets, it is 0.006 and 0.034.Source: §4.1 · p. 4
4.2 Dyads
- Toxic posts are 65% and 64% more likely than nontoxic posts to elicit toxic replies in the news and midterms datasets; unconnected dyads show higher reply toxicity.For toxic posts, the probability is 0.30 versus 0.27 in the two datasets; for nontoxic posts, it is 0.18 versus 0.16.Source: §4.2 · p. 5
- Toxic-reply probability decreases as common friends increase, while larger parent-over-child follower gaps raise it especially after nontoxic posts.For toxic posts, 100 versus 1 common friend corresponds to an 11% lower probability in news and a 9% lower probability in midterms.Source: §4.2 · p. 5
4.3 Reply Tree Structure
- Larger, wider, and deeper reply trees tend to be more toxic in both datasets, although depth and width are positively correlated with tree size.Depth and width correlations with size are 0.53 and 0.97 in news, and 0.48 and 0.97 in midterms.Source: §4.3 · pp. 5–6
- The Wiener-index relationship with toxicity varies by dataset and tree size: size-stratified trends are negative for larger news trees and for midterms trees of all sizes.Regression confirms a negative relationship when controlling for the number of tweets.Source: §4.3 · p. 6
4.4 Follow Graph Structure
- Denser participant follow graphs tend to be less toxic, while more connected components and higher modularity tend to accompany greater toxicity in both datasets.Most conversations have modularity zero: 69.1% in news and 76.35% in midterms.Source: §4.4 · pp. 6–7
5 PREDICTING TOXICITY
The paper tests whether conversation structure can forecast toxicity at the conversation and individual-user levels, following earlier analyses of structural correlations. It evaluates predictions from early reply prefixes, next-reply behavior, and transfer between news and midterm conversations.
5.1 Conversation Toxicity Predictions
- Prefix labels compare suffix toxicity within buckets matched on prefix toxic-tweet counts, and predictors include content, reply-tree, graph, embeddedness, alignment, and timing features.Buckets with fewer than 200 conversations are excluded, as are conversations smaller than twice the prefix size.Source: §5.1 · pp. 7–8
- Combining structural and content-toxicity features significantly improves future-conversation predictions over either feature type alone.Structural features alone outperform content-toxicity features, while their combination captures complementary aspects of the prefix.Source: §5.1 · p. 8
- 0.61–0.64 accuracy (news) and 0.61–0.63 (midterms) are achieved by all features across prefix sizes; reply-graph features perform best in midterms.Reply-graph and embeddedness features perform best across prefix sizes in the news dataset.Source: §5.1 · p. 8, Figure 7 · p. 9
- Accuracy can decline with longer prefixes as eligible conversations dwindle and labels become more nuanced; ten replies arrive within five minutes for half of news conversations and within an hour for midterms.The authors attribute the decline partly to reduced training data and the greater number of prefix-toxicity buckets.Source: §5.1 · p. 9
5.2 Next Reply Toxicity Predictions
- The paired design compares toxic and nontoxic tweets from each conversation to control for root-tweet content, assuming the identity of the next user is known.It excludes self-replies, direct replies to the root, and tweets with toxicity scores close to the threshold.Source: §5.2 · pp. 9–10
- 0.712 accuracy and 0.797 AUC (news), and 0.737 accuracy and 0.829 AUC (midterms), using all features to predict whether the next reply is toxic.Combining conversation-state and structural features significantly improves performance in both datasets.Source: §5.2 · pp. 10–11, Figure 8 · p. 11
5.3 Domain Transfer
- Next-reply prediction accuracy drops by less than 1% across domains, while future-conversation accuracy drops by 1.7%–6.2% in news and 1.7%–4.7% in midterms.The authors report that models in both tasks still make sensible out-of-domain predictions.Source: §5.3 · p. 11
6 FURTHER RELATED WORK
This section distinguishes the paper’s focus on conversational and social context from prior work on antisocial behavior, blocking, and election-related interactions.
6 FURTHER RELATED WORK
- Prior work studied antisocial users, dyadic relationships, and community-level behavior; this paper instead examines the conversational and social context of toxic behavior.Source: §6 · p. 11
- Unlike Zhang et al., who predict blocking with a hypergraph of prior interactions, this study predicts toxicity using reply-tree, user-reply, and follow structures.Source: §6 · p. 11
- Midterm-election studies focused on adversarial users targeting candidates, whereas this paper studies toxicity in conversations started by or mentioning candidates.Source: §6 · p. 11
7 DISCUSSION AND CONCLUSION
The study links conversation structure on Twitter to toxicity and tests whether structural features predict future toxicity. It reports patterns across individual, dyad, and group levels, alongside practical implications and limitations.
7 DISCUSSION AND CONCLUSION
- The analysis covers 1.18M conversations prompted by major news outlets and 2018 midterm candidates, and focuses mostly on political conversations.The authors call for testing generalizability to other domains and platform-specific differences.Source: §7 · pp. 11–12
- Toxicity is spread among many low- to moderately toxic users; toxic posts are more likely to receive toxic replies, often from users without a social connection to the poster.Toxic conversations also tend to be larger, with wider and deeper reply trees but less dense follow graphs.Source: §7 · p. 11
- Structural features predict whether a conversation will become more toxic and whether a specific user’s next reply will be toxic.Reported accuracies reach 61.6% for conversation-level prediction and 70.5% for next-reply prediction.Source: §7 · p. 11
- Early toxicity predictions could help rank conversations or alert their initiators, while next-reply predictions could help rank threads to make a toxic reply less likely.The authors also suggest experiments exposing social connections to potentially encourage more civil behavior.Source: §7 · p. 11
- The study uses Perspective API and a broad toxicity definition, examines associations and predictive power, and does not test whether interventions reduce toxicity.The authors identify A/B tests as a next step and note that finer-grained antisocial behaviors remain to be examined.Source: §7 · pp. 11–12