PaperlayerView exact revision on arXiv

Source-linked AI summary

Anyone Can Become a Troll: Causes of Trolling Behavior in Online Discussions

Justin Cheng, Michael Bernstein, Cristian Danescu-Niculescu-Mizil, Jure Leskovec

arXiv:1702.01119v1cs.SIcs.CYcs.HCstat.AP

TL;DR

Trolling disrupts online discussions, but it remains unclear whether it reflects antisocial individuals or situational influences. Using an experiment and large-scale observational analysis, the paper finds that mood and discussion context significantly increase trolling, with combined effects doubling baseline rates.

  • Problem

    The paper asks whether trolling stems from particularly antisocial individuals or situational factors, and which conditions affect users’ likelihood of trolling.

  • Method

    The authors combine a field experiment with longitudinal analysis of over 16 million CNN.com posts to examine mood and discussion context.

  • Results

    Negative mood and prior troll posts each increase trolling, together doubling baseline rates; trolling also tracks mood shifts, persists across discussions, and decays over time.

  • Takeaways & Limitations

    Ordinary users can engage in trolling under the right conditions, and trolling can spread through discussion contexts and persist across discussions.

  • Takeaways & Limitations

    The experiment’s payment, pseudonymous usernames, and required commenting may alter participation incentives or comment quality, while different initial posts may elicit different responses.

Abstract

from arXiv · show

In online communities, antisocial behavior such as trolling disrupts constructive discussion. While prior work suggests that trolling behavior is confined to a vocal and antisocial minority, we demonstrate that ordinary people can engage in such behavior as well. We propose two primary trigger mechanisms: the individual's mood, and the surrounding context of a discussion (e.g., exposure to prior trolling behavior). Through an experiment simulating an online discussion, we find that both negative mood and seeing troll posts by others significantly increases the probability of a user trolling, and together double this probability. To support and extend these results, we study how these same mechanisms play out in the wild via a data-driven, longitudinal analysis of a large online news discussion community. This analysis reveals temporal mood effects, and explores long range patterns of repeated exposure to trolling. A predictive model of trolling behavior shows that mood and discussion context together can explain trolling behavior better than an individual's history of trolling. These results combine to suggest that ordinary people can, under the right circumstances, behave like trolls.

INTRODUCTION · BACKGROUND · Antisocial behavior in online discussions

The paper examines why ordinary users troll in online discussions, challenging the view that trolling is mainly innate. It focuses on mood and discussion context, using experiments, large-scale observation, and predictive modeling to assess their effects.

  • INTRODUCTION: Online antisocial behavior, including trolling, harassment, and bullying, causes emotional distress, can lead to offline threats, and affects 40% of internet users.These harms motivate studying trolling in online discussions.
  • INTRODUCTION: Prior research contrasts an innate account of trolling, based on distinctive traits and motivations, with evidence that environments can influence aggressive behavior.The paper investigates whether trolling is made as well as born.
  • INTRODUCTION: The study combines a field experiment with observational analysis to separate causal mechanisms from finer-grained patterns occurring in a large news discussion community.It examines user mood and surrounding discussion context, including exposure to others’ trolling.
  • INTRODUCTION: Negative mood and prior troll posts each increased subsequent trolling, and together doubled participants’ baseline rates of engaging in trolling behavior.The experiment simulated an online news comment section.
  • INTRODUCTION: In over 16 million CNN.com posts, one out of four abuse-flagged posts came from users without a prior record of such posts.The analysis also found that trolling propensity tracked population-level mood shifts and examined repeated exposure over time.
  • INTRODUCTION: A logistic regression model predicted trolling with AUC=0.78 and indicated that discussion context explained behavior better than an intrinsic account.The model evaluated the relative importance of mood and context against trolling history.
  • BACKGROUND: The review asks how trolling spreads because ordinary users may temporarily engage in it under online disinhibition, anonymity, reduced accountability, and environmental influence.The paper frames trolling as potentially contagious and more situational than innate.
  • Antisocial behavior in online discussions: Online antisocial behavior extends offline aggression, harassment, and bullying, while public discussions use ranking, moderation, identification, interface redesign, and comment disabling to combat it.Trolling includes flaming, griefing, swearing, and personal attacks outside community standards.

Causes of antisocial behavior

The section identifies mood and discussion context as two possible causal triggers of trolling, whose effects are examined through a controlled experiment and a large-scale longitudinal analysis. It hypothesizes that negative mood and prior troll posts increase users’ likelihood of trolling.

  • Motivations: Prior research proposes boredom, fun, and venting as motivations for antisocial behavior, but their applicability to the general population remains largely unknown.The prior literature is described as largely qualitative and non-causal.
  • Mood: Negative mood may increase trolling by impairing self-regulation, reducing satisfaction with life, and producing less favorable impressions of others.Aversive events have also been linked to increased aggression toward others.
  • Mood: Mood-related circumstances, including time of day, day of week, and delays after trolling, may alter trolling rates and susceptibility.The passage links higher negative mood at the start of the week and late at night with potentially higher trolling, while calming-down periods may reduce aggression.
  • Discussion context: Discussion context may shape contributions through discussion starters, inferred social norms, and patterns of commenters following positive or negative posts.The section also notes that less thoughtful posts can influence subsequent contributions, though the supplied passage truncates that finding.
  • Discussion context: Prior troll posts, regardless of author, are expected to produce more subsequent trolling, while the discussion topic is also expected to affect trolling.The context hypothesis is explicitly stated as H2: discussion context affects a user’s likelihood of trolling.

Influence and antisocial behavior

Prior research suggests antisocial behavior may spread through social influence, emotional and behavioral transfer, and negativity bias. The paper therefore hypothesizes that trolling can spread between users and tests this through experiments and longitudinal analysis of CNN.com discussions.

  • Influence and antisocial behavior: Social influence and person-to-person transfer of emotions and behavior suggest that one user’s trolling could trigger participation by others.The paper frames this potential contagion as a possible flame war.
  • Influence and antisocial behavior: The “Broken Windows” hypothesis proposes that poor-taste comments can invite worse comments and help antisocial behavior become normalized.Such behavior may perpetuate within a community despite being undesirable.
  • Influence and antisocial behavior: Negativity bias predicts that antisocial behavior is especially influential and persistent because negative entities spread more readily and bad impressions resist disconfirmation.These mechanisms motivate the expectation that negative behavior will have disproportionate influence.
  • Influence and antisocial behavior: The paper hypothesizes that trolling behavior can spread from user to user.This claim is formalized as H3.
  • Influence and antisocial behavior: The authors test the hypothesis by studying how CNN.com discussions evolve and developing a model of trolling spread between users.The broader study combines controlled experiments with discussion analysis to test related hypotheses and extend the findings.

EXPERIMENT: MOOD AND DISCUSSION CONTEXT · Experimental Setup

The experiment used a two-by-two design to test how induced mood and discussion context affect trolling behavior and discussion affect. Participants completed a quiz-based mood manipulation before joining a simulated news discussion seeded with either benign or troll-like comments.

  • EXPERIMENT: MOOD AND DISCUSSION CONTEXT: A two-by-two between-subjects design varied mood with positive or negative prior stimuli and context with benign or troll-like initial posts.The study examined POSMOOD versus NEGMOOD and POSCONTEXT versus NEGCONTEXT.
  • EXPERIMENT: MOOD AND DISCUSSION CONTEXT: Discussion quality was measured by whether participants wrote troll-like posts and by the resulting discussion’s positive or negative affect.Trolling was assessed through expert labeling, while affect was measured using sentiment analysis.
  • Experimental Setup: The experiment was conducted on Amazon Mechanical Turk with US-based participants who could participate once and received $2.00 compensation.The reported compensation corresponded to an hourly rate of $8.00, and the study’s purpose was withheld beforehand.
  • Experimental Setup: Participants completed a 15-question quiz with a five-minute limit, using substantially harder questions in the NEGMOOD condition to influence pre-discussion mood.Question composition and order were otherwise held constant across mood conditions.
  • Experimental Setup: Mood was checked with 65 Likert-scale questions from the Profile of Mood States questionnaire, covering axes such as anger and fatigue.The questionnaire quantified participants’ mood after the quiz.
  • Experimental Setup: Participants then used a news-site-like interface to read an article, comment, reply, and vote while believing the study tested a comment-ranking algorithm.Each participant was required to leave at least one comment.
  • Experimental Setup: The context manipulation seeded discussions with either troll-like or innocuous comments abridged from real online discussions.The examples addressed voting and gender, and were drawn from the original article’s comments and forums such as Reddit.
  • Experimental Setup: Eight independent universes were created for each condition, producing 32 universes to reduce the risk that outcomes reflected chance path dependence.Participants were randomized among universes within each condition, which otherwise shared the same seeded comments.

Results

The experiment validated its mood and context manipulations and found that both negative mood and prior troll posts increased trolling, with the combination producing the highest troll-post rate. Negative context also increased negative affect, while mood and context showed nonsignificant trends toward shorter posts and fewer upvotes.

  • Manipulation checks: Total mood disturbance was 12.2 in POSMOOD versus 40.8 in NEGMOOD, confirming that the quiz induced a more negative mood.NEGMOOD participants also scored higher on anger, confusion, depression, fatigue, and tension, and lower on vigor.
  • Manipulation checks: NEGCONTEXT seed posts received 36% upvotes versus 90% for POSCONTEXT, confirming that participants perceived them as more troll-like.The difference was significant (t(507)=15.7, p<0.001).
  • Trolling behavior: 68% of posts were trolling under NEGMOOD and NEGCONTEXT, compared with 35% under POSMOOD and POSCONTEXT.Intermediate conditions produced 47% and 49% troll posts, respectively; regression showed significant effects of both NEGMOOD and NEGCONTEXT (p<0.05).
  • Affective language: Negative context significantly increased negative-word usage, whereas negative mood did not; neither factor significantly affected positive affect.Negative-word proportion may miss sarcasm or off-topic trolling.
  • Other results: Posts averaged 44 words in POSMOOD/POSCONTEXT versus 29 in NEGMOOD/NEGCONTEXT, while upvotes fell from 79% to 75%; neither trend was significant.Troll posts ranged from swearing and personal attacks to veiled insults, sarcasm, and off-topic statements.

Discussion.

The discussion concludes that mood and discussion context significantly affect trolling likelihood, suggesting trolling can be generally induced rather than limited to atypical users. It also identifies article topic as an additional mediator and notes experimental incentive-related limitations.

  • Mechanisms: Exposure to others’ negative reactions may make expressing negativity more acceptable, while negative mood accentuates perceived negativity and reduces self-control.The passage presents these mechanisms as an explanation for why NEGCONTEXT and NEGMOOD increased trolling.
  • Limitations: The experiment’s required commenting and participant payment may affect participation incentives, although the design enabled isolation of mood and discussion context.Participants were required to post at least one comment, and payment may have altered incentives to participate.
  • Overall findings: Mood and discussion context significantly affect a user’s likelihood of engaging in trolling behavior, indicating that trolling can be generally induced.The authors argue that these effects require substantial susceptibility across the population, not only among a small fraction of atypical users.
  • Context effects: Discussion context also includes the accompanying article’s topic, which the authors find mediates trolling behavior.This extends the analysis of discussion context beyond prior trolling exposure.

DATA: INTRODUCTION

The study analyzes CNN.com discussions hosted on Disqus, distinguishing whole discussions from reply-based sub-discussions. It focuses on general users and measures trolling primarily through posts flagged by users for guideline violations.

  • Study setting: CNN.com users discuss news articles on Disqus and can post, reply, vote, flag posts, or face moderator deletion and bans.The site covers topics including politics and technology, with moderation governed by community guidelines.
  • Units of analysis: The analysis distinguishes discussions following an article from sub-discussions consisting of a top-level post and its replies.This distinction accounts for discussions reaching thousands of posts, where users may not read earlier responses, while replies necessarily involve a preceding post.
  • Sample construction: The study filters banned users and users whose posts were all deleted to examine mood and discussion-context effects in the general population.Many banned users are clearly identifiable trolls.
  • Trolling measure: Flagged posts are the primary trolling measure because moderator deletions are incomplete, written negative affect misses some trolling, and downvotes can reflect disagreement.Flagging is defined as users marking posts for violating community guidelines.
  • Measure validation: Flagged posts contain negative-affect words in 3.7% versus 3.4% of words, receive 58% versus 30% of votes as downvotes, and are deleted in 79% versus 21% of posts.These flagged-versus-non-flagged differences are statistically significant for negative affect, downvotes, and moderator deletion.

DATA: UNDERSTANDING MOOD · Happy in the day, sad at night · Anger begets more anger

Longitudinal observational data link trolling patterns to mood proxies, with trolling indicators lowest in the morning and highest late at night and early in the work week. Trolling and exposure to trolling also predict later flagged behavior across unrelated discussions, consistent with persistent negative mood and transmitted norms.

  • DATA: UNDERSTANDING MOOD: The analysis uses large-scale longitudinal data and mood correlates because mood cannot be measured directly.It examines time-of-day and day-of-week patterns, as well as whether aggression persists beyond an initial unpleasant event.
  • Happy in the day, sad at night: Mood-linked trolling is evaluated through flagged posts, negative affect, and downvotes across time of day and day of week.Prior work indicates that positive affect peaks in the morning and on weekends, motivating these comparisons.
  • Happy in the day, sad at night: Flagged posts, negative affect, and downvotes are lowest in the morning, highest in the evening, and peak on Monday.These patterns align with prior evidence that mood is worst in the evening and at the start of the work week.
  • Happy in the day, sad at night: 4.1% vs. 4.3%, d=0.01, flagged-post rates show a small but significant increase in negative behavior from 6 am–12 pm to 11 pm–5 am for the same users.The paired comparison addresses whether differences reflect users’ posting schedules rather than time-linked changes within users.
  • Happy in the day, sad at night: Overall, trolling patterns correspond predictably with mood despite the absence of direct user mood measurements.This conclusion follows from the observed daily and weekly variation in trolling indicators.
  • Anger begets more anger: The study tests whether negative mood spills over from prior discussions into subsequent unrelated discussions, including effects of direct and indirect exposure to negative behavior.It distinguishes users who directly engaged in flagged behavior from users who merely participated in discussions where trolling occurred.
  • Anger begets more anger: Users with no prior flagged posts are matched on prior posting activity to assess whether earlier flagged behavior predicts later trolling among ordinary users.The design ensures that neither sampled user had any previously flagged posts and matches them on total prior posts.
  • Anger begets more anger: Both prior trolling and participation in discussions where trolling occurred affect whether users troll in future discussions.The results suggest persistent negative mood and transmission of trolling norms across discussions, rather than merely stable individual troll tendencies.

Time heals all wounds · DATA: UNDERSTANDING DISCUSSION CONTEXT

The data analysis examines whether waiting between posts reduces mood-related trolling and shows that discussion context also shapes subsequent trolling. Short intervals after a flagged post are associated with higher flagging probability, while prior flagged posts, their number and order, and discussion topic affect later flagging.

  • Time heals all wounds: The analysis tests whether longer waits between posts reduce the carryover of negative mood after a flagged post.The proposed mechanism is a “time-out” that allows users to calm down before posting again.
  • Time heals all wounds: High flagging probability occurs when the interval after a flagged post is five minutes or less, with the next post placed in a new discussion.The design changes discussion context by using different other users, isolating the time-between-posts association.
  • Time heals all wounds: After ten minutes, the probability of being flagged gradually decreases as more time passes.Users with better impulse control may wait longer when angry, so separating waiting behavior from mood remains future work.
  • Time heals all wounds: These findings support rate-limiting posts as a way to reduce the persistence of negative mood across discussions.The paper relates this result to rate-limiting mechanisms introduced by some forums.
  • DATA: UNDERSTANDING DISCUSSION CONTEXT: Posts are more likely to be flagged when others’ prior posts in the discussion were also flagged.This section verifies and extends the experiment’s finding that discussion context influences trolling.
  • DATA: UNDERSTANDING DISCUSSION CONTEXT: The number and ordering of flagged posts affect the probability of subsequent trolling, as does the discussion topic.The analysis treats these features as components of discussion context.

“FirST!!1” · From bad to worse: sequences of trolling

Initial flagged posts create a strong, lasting precedent for later trolling, while repeated and more conspicuous troll posts progressively increase the likelihood that subsequent users will troll. These effects support the conclusion that trolling spreads from user to user through discussion context.

  • “FirST!!1”: The analysis matched discussions on article topic, posting day, and total number of posts to estimate how initial flags affect subsequent flagging.It compared discussions of at least 20 posts with flagged versus unflagged first posts using propensity score matching.
  • “FirST!!1”: 3.1% vs. 1.7% of subsequent posts were flagged after an initial flagged post versus an unflagged one, a significant difference that persisted in discussion halves.The overall effect was d=0.32, t(1545)=9.1, p<0.001; in the second half, it remained 2.1% vs. 1.3%, d=0.19, t(1545)=5.4, p<0.001.
  • “FirST!!1”: 7.1% vs. 1.7% of subsequent posts were flagged when the first three posts were all flagged versus when none were flagged.The gap was significant: d=0.61, t(113)=4.6, p<0.001.
  • “FirST!!1”: Controlling for article effects, the results collectively indicate that initial discussion posts establish a strong, lasting precedent for later trolling.The control examined sub-discussions consisting of a top-level post and all its replies, which more closely represent conversations between users.
  • From bad to worse: sequences of trolling: The same increasing trend appeared for users new to and returning to a sub-discussion, although returning users were more likely to be subsequently flagged.The analysis separately considered users new to the sub-discussion and users who had already posted there.
  • From bad to worse: sequences of trolling: 2% with no prior flagged posts, 7% with one, and 49% with four: the probability that a fifth post by a new user was flagged increased monotonically with prior flags.All pairwise differences were significant with Holm correction (χ2(1)>7.6, p<0.01).
  • From bad to worse: sequences of trolling: When exactly one of the first four posts was flagged, a fifth post was more likely to be flagged when that troll post was closer in position.This pattern held for both new and previously participating user groups.
  • From bad to worse: sequences of trolling: The increasing impact of troll-post presence and conspicuousness, together with prior-discussion exposure, supports the hypothesis that trolling behavior spreads from user to user.Figure 4 also shows that discussion topic influences the probability that a post is flagged.

Hot-button issues push users’ buttons? · Summary · A MODEL OF HOW TROLLING SPREADS

Discussion topic, mood, and prior trolling shape users’ likelihood of trolling, with context providing the strongest predictive signal. These findings support a situational account in which ordinary users troll when mood and discussion context prompt such behavior.

  • Hot-button issues push users’ buttons?: Flagging is near 4% in health, justice, showbiz, sport, US, and world sections, versus near 2% in opinion, politics, tech, and travel.The authors suggest higher rates may reflect coverage of controversial issues.
  • Hot-button issues push users’ buttons?: The interplay of personal values, group membership, and topic with trolling remains future work.The passage identifies these factors as unresolved aspects of topic-related trolling.
  • Hot-button issues push users’ buttons?: Large variation across article sections suggests discussion topic influences baseline trolling, with hot-button topics sparking more troll posts.Controversial topics may divide communities and lead to more trolling.
  • Summary: Bad mood induces trolling, whose rates vary by time of day and day of week; bad mood may persist across discussions but diminish with time.The authors derive these findings through experimentation and data analysis.
  • Summary: Prior troll posts increase the likelihood of future troll posts, with an additive effect as the number of troll posts increases.The summary also reports that mood and discussion context can induce trolling behavior.
  • A MODEL OF HOW TROLLING SPREADS: The predictive model sampled posts from discussions at random (N=116,026) and balanced flagged and non-flagged users, making random guessing 50% accurate.The analysis included users whose posts had previously been flagged.
  • A MODEL OF HOW TROLLING SPREADS: Discussion-context features performed best (AUC=0.74), and whether the previous post was flagged was the individually most predictive feature.This result suggests context alone can predict trolling behavior and explains trolling better situationally than innately.
  • A MODEL OF HOW TROLLING SPREADS: Context strongly predicts later trolling beyond intrinsic propensity, while recent posting history also predicts trolling, suggesting mood carries over and past trolling predicts future trolling.The model represents mood through seasonality and recent posting history, context through preceding posts, and innate propensity through user identity and overall trolling history.

The spread of negativity

On CNN.com, flagged posts and users with flagged posts rose over time, suggesting that trolling became more common and spread to a growing fraction of users. Comparing the dataset’s first and second halves, both proportions increased significantly.

  • The spread of negativity: The analysis asks whether induced trolling can carry over across discussions, cascade, and worsen the community over time.This frames the spread of negativity as a possible reinforcing process.
  • The spread of negativity: On CNN.com, the proportions of flagged posts and users with flagged posts rose over time, suggesting trolling became more common and widespread.The upward trends indicate that a growing fraction of users engaged in trolling behavior.
  • The spread of negativity: 0.03 vs. 0.04 and 0.09 vs. 0.12, p<0.001: flagged-post and flagged-user proportions increased from the first to the second half of the CNN.com dataset.The reported comparison covers both the proportion of flagged posts and the proportion of users with flagged posts.

Designing better discussion platforms · Limitations and future work · CONCLUSION

The paper proposes platform interventions that address mood and discussion context, while identifying limitations and future research directions concerning mood signals, trolling types, social cues, and user heterogeneity. It concludes that both innate and situational factors shape trolling, so mitigation should complement bans with measures targeting situational triggers.

  • Designing better discussion platforms: Inferring users’ mood and selectively rate-limiting posts could discourage heat-of-the-moment trolling, while comment retraction and reducing interface frustration may minimize regret and frustration.Suggested mood signals include recent heated debates and keystroke movements.
  • Designing better discussion platforms: Hiding troll comments and prioritizing constructive ones may increase perceived civility and reduce users’ likelihood of following suit in trolling.The proposal focuses on altering discussion context through comment ranking.
  • Designing better discussion platforms: User feedback can reduce exposure to downvoted content, but downvoting may worsen subsequent comments; selectively exposing positive feedback could avoid this negative loop.The suggested alternative makes positive signals public while hiding negative signals.
  • Limitations and future work: Future work should improve mood signals and model discussion reply structure, sentiment changes, and idea flow to clarify how context affects trolling.The current results indicate an overall mood effect but not a fully nuanced relationship.
  • Limitations and future work: Different trolling strategies may differ in prevalence and severity, so studying specific forms could support measures targeted to the most pertinent behaviors.Examples range from undirected swearing to targeted harassment and bullying.
  • Limitations and future work: Future research could distinguish consistently trolling users from those whose trolling emerges situationally before community bans.Prior work identified users whose posts were consistently deleted and users whose deletions began shortly before banning.
  • CONCLUSION: Trolling reflects both innate and situational factors, so communities should complement bans with design measures that mitigate situational triggers.The conclusion emphasizes different affordances for managing each type of trolling.
Loading 1702.01119v1…