Source-linked AI summary
Exploring Cyberbullying and Other Toxic Behavior in Team Competition Online Games
Haewoon Kwak, Jeremy Blackburn, Seungyeop Han
TL;DR
Cyberbullying and other toxic behavior are widespread concerns in competitive online games, but large-scale empirical evidence about their patterns and perceptions is limited. The paper analyzes millions of League of Legends reports, crowdsourced decisions, and match logs to test theories of toxic behavior. It finds low reporting engagement, significant effects of explicit reporting requests, ally-enemy and cultural biases, and a link between match results and toxic behavior.
Problem
Competitive online games expose players to toxic behavior, yet large-scale evidence about its occurrence, perceptions, and social patterns remains limited.
Method
The paper quantitatively analyzes League of Legends Tribunal reports, crowdsourced decisions, and detailed match logs using theories from sociology and psychology.
Results
The study finds low reporting engagement, significantly increased reporting after explicit requests, ally-enemy and cultural biases, and a significant link between match results and toxic behavior.
Takeaways & Limitations
The large-scale dataset provides empirical understanding of toxic behavior and suggests avenues for designing systems to detect, prevent, and counteract it.
Takeaways & Limitations
Results drawn from League of Legends should be applied cautiously to other domains and games because game-specific features may differ.
Abstract
from arXiv · showhide
In this work we explore cyberbullying and other toxic behavior in team competition online games. Using a dataset of over 10 million player reports on 1.46 million toxic players along with corresponding crowdsourced decisions, we test several hypotheses drawn from theories explaining toxic behavior. Besides providing large-scale, empirical based understanding of toxic behavior, our work can be used as a basis for building systems to detect, prevent, and counter-act toxic behavior.
EXECUTIVE SUMMARY
The paper examines toxic behavior in competitive online games using large-scale League of Legends reports and crowdsourced decisions. It finds low reporting engagement, increased reporting after explicit requests, differing perceptions and biases, cultural variation, and links between match outcomes and toxicity.
- Findings: Players are surprisingly not engaged in actively reporting toxic behavior.
- Findings: Engagement can be significantly increased via explicit pleas to report.
- Findings: Perceptions of toxic behavior significantly vary between people who experienced it and neutral third-party judges.
- Findings: Reporting exhibits biases between allies and enemies, while perceptions of toxic behavior also differ significantly across cultures.
- Findings: The result of a match is significantly linked to the appearance of toxic behavior.The authors suggest avenues for designers of video games and computer-mediated communication systems.
- Motivation and contribution: Millions of League of Legends reports provide large-scale evidence for studying cyberbullying and other toxic behavior in competitive online games.The dataset includes reports on hundreds of thousands of accused players and crowdsourced judgments about whether the behavior was toxic.
BACKGROUND
League of Legends is a match-based, two-team competition game where players interact through chat and can report one another after matches. Reports use predefined toxicity categories, including cyberbullying and domain-specific behaviors.
- League of Legends as a Team Competition Game: League of Legends is a match-based MOBA in which two teams compete to destroy the enemy Nexus, typically over 30 to 40 minutes.In the most popular mode, each team has five players who may be friends or similarly skilled strangers.
- Player Reports on Toxic Behavior: After a match, the scoreboard shows the match summary, participating players, and chat, with a button for reporting toxic players.Each player can submit one report for every other player per game, and reports are submitted only after the match.
- Player Reports on Toxic Behavior: Player reports are not considered a strategic element during the match itself.
- Player Reports on Toxic Behavior: Reports use predefined categories such as assisting the enemy, intentional feeding, offensive language, verbal abuse, negative attitude, spamming, and leaving the game.The categories are grouped into cyberbullying—offensive language and verbal abuse—and domain-specific toxicity comprising the remaining categories.
- Player Reports on Toxic Behavior: Intentional feeding means deliberately dying to the opposing team, while leaving the game or AFK means remaining inactive throughout the match.Both behaviors usually strengthen the opposing team because of the game’s design.
LoL Tribunal as a Crowdsourcing System
The LoL Tribunal crowdsources punishment decisions for heavily reported players by presenting anonymized case information to experienced player reviewers. Verdicts use majority voting and can lead to suspensions or permanent bans.
- LoL Tribunal as a Crowdsourcing System: Players reported more than a few hundred times are brought to the Tribunal, where up to five reported matches are aggregated into a case.
- LoL Tribunal as a Crowdsourcing System: Reviewers receive full chat logs, player performance, match duration, and report categories with optional comments.Match data anonymizes all players to support unbiased decisions.
- LoL Tribunal as a Crowdsourcing System: About 100-150 reviewers vote on each case, and a majority determines the verdict.Guilty verdicts can produce bans lasting one week, several months, or longer, including permanent suspensions.
DATA COLLECTION
The study analyzes Tribunal reports, crowdsourced decisions, and detailed match logs across three League of Legends regions. This witness-report dataset is intended to support scalable, quantitative analysis of observed toxic behavior, though its completeness is difficult to measure.
- DATA COLLECTION: The study collects player reports, crowdsourced decisions, and detailed match logs from the Tribunal.This combined collection forms the basis for the paper’s quantitative analysis.
- DATA COLLECTION: Third-party witness reports avoid some recall and social-desirability biases associated with self-reports and hypothetical scenarios.The authors also describe crowdsourced decisions as offering a more objective viewpoint on reported behavior.
- DATA COLLECTION: The analysis focuses on the North America, Western Europe, and South Korea regions.The regions were selected for cultural representativeness and author familiarity.
- DATA COLLECTION: In April 2013, the dataset contained about 11 million reports from 6 million matches involving 1.5 million potentially toxic players.The South Korea portion was smaller because its Tribunal began later than the North America and Western Europe Tribunals.
- DATA COLLECTION: The dataset’s completeness is difficult to measure because player reports are internally managed.The authors nevertheless state that reports and votes were not removed from the servers, providing what they describe as a complete Tribunal picture.
RESEARCH QUESTIONS AND HYPOTHESES
The paper frames toxic behavior in League of Legends as a product of team competition and anonymity, then applies sociological and psychological theories to player behavior and reactions.
- The study formulates research questions about how toxic players behave and how other players respond.
- League of Legends provides a large-scale setting for applying theories of toxic behavior in real-world gameplay.Its team competition and anonymous features may increase aggressive behavior and reduce accountability.
- The conceptual framework covers bystander effects, vague perceptions of toxicity, intergroup dynamics, intra-group conflict, socio-political factors, and team cohesion.
Bystander Effect and Vague Nature of Toxic Playing
The first research questions examine whether players actively report toxic behavior and how ambiguity in toxicity affects Tribunal decisions. The paper connects under-reporting to the bystander effect and differing perceptions of severity.
- The study asks how active players are in reporting toxic behavior and links this question to the design of systems such as the Tribunal.
- In anonymous, temporary teams, the bystander effect may discourage witnesses from reporting toxic players.The theory describes reduced helping by observers immersed in a group.
- Explicit requests to report toxic players are hypothesized to increase the number of reports.
- The study also asks how the vague nature of toxic playing affects Tribunal decisions.
- Different perceptions of toxicity severity between reporters and Tribunal reviewers may produce pardons.
In-group Favoritism and Out-group Hostility
The paper examines whether team competition produces in-group favoritism and out-group hostility, predicting that teammates report equally harmful toxic behavior less often than opponents.
- The study asks how reporting differs between a toxic player’s teammates and opponents.
- In-group favoritism means favoring teammates over similarly likable opponents and disliking opponents more than similarly dislikable teammates.
- Deindividuation theory connects immersion in anonymous crowds with reduced self-awareness and responsibility, offering a possible mechanism for in-group favoritism.
- SIDE theory argues that anonymity alone is insufficient; anti-normative behavior depends on anonymity combined with a relevant out-group context.
- Because opposing teams compete directly for one win, League of Legends creates clear in-groups and out-groups and encourages group identification.
- For behavior affecting both teams equally, teammates are hypothesized to submit fewer reports than opponents.
Intra-group Conflicts and Socio-political Factors
The paper investigates intra-group conflict and regional differences in toxic behavior, focusing on how poor individual performance can threaten team success and how socio-political context may shape reporting and punishment.
- Intra-group Conflicts and Socio-political Factors: A poorly performing teammate can become a target because that player makes the opposing team relatively stronger and affects the match outcome.
- Intra-group Conflicts and Socio-political Factors: League of Legends teams resemble impersonal, task-oriented associations, so players may harass teammates viewed as obstacles to winning.
- Intra-group Conflicts and Socio-political Factors: The game’s global user base enables study of regional differences in intra-group conflict, motivating the question about socio-political factors.
- Intra-group Conflicts and Socio-political Factors: Korean gaming culture includes Wang-tta, described as isolating and bullying the worst player in a peer group.
- Intra-group Conflicts and Socio-political Factors: North America and Western Europe are characterized as more individualistic, with less ingrained hostility toward another player’s poor individual performance.
- Intra-group Conflicts and Socio-political Factors: Cyberbullying offenses are hypothesized to be less likely to be punished in Korea than in other regions.
- Intra-group Conflicts and Socio-political Factors: Collectivist cultures place greater emphasis on cooperation, group goals, and belonging, making deliberate harm to the group subject to intense derision.
- Intra-group Conflicts and Socio-political Factors: Reports about behavior affecting match results are hypothesized to be more common and more likely to be punished in Korea than elsewhere.
Team-cohesion and Performance
The paper examines whether team performance and negative outcomes are associated with toxic behavior, reporting, and punishment. It frames losing-team outcomes as a context in which reporting may increase and pardons may be more common.
- RQ4 asks how toxic behavior, player reports, and team performance are related.
- Attribution theory predicts that people search for causes of failure, potentially blaming individual teammates for poor team performance.
- Counterfactual thinking can make a single player’s mistake appear responsible for a negative match outcome.
- The authors hypothesize that more reports come from matches where the accused player’s team loses.
- The authors also hypothesize that accusations against losing-team players are more likely to be pardoned than those against winning-team players.
RESULTS
The results show limited reporting overall, category- and region-dependent perceptions, and strong links between toxic behavior, team affiliation, culture, and match outcomes. Explicit requests substantially increase enemy reporting, while losing teams generate more reports.
- Bystander Effect and Vague Nature of Toxic Playing: 1.812 mean and 1 median reports per match indicate that players do not actively report toxic players.The majority of matches have fewer than three reports despite many exposed players.
- Bystander Effect and Vague Nature of Toxic Playing: 16.37 times higher opponent-reporting probability follows an explicit request to report toxic players.The authors interpret explicit requests as neutralizing the bystander effect.
- Bystander Effect and Vague Nature of Toxic Playing: Intentional feeding and assisting the enemy receive more reports per match than other toxicity categories.Category differences are significant under Kruskal-Wallis and post-hoc Wilcoxon tests.
- Bystander Effect and Vague Nature of Toxic Playing: 26% pardoned cases show that reviewers and reporting players sometimes disagree about whether behavior is toxic.A pardon means reviewers do not regard the reported player as toxic.
- Intra-group Conflicts and Socio-political Factors: 16,339 ally-only versus 23,966 enemy-only inappropriate-name reports indicate greater forgiveness by allies when offenses affect both teams neutrally.The authors identify this pattern as supporting in-group favoritism.
- Intra-group Conflicts and Socio-political Factors: 17.1% of Korean cyberbullying reports are pardoned, versus 14.3% in North America and 9.7% in EUW.The regional effect on pardons is statistically significant.
- Intra-group Conflicts and Socio-political Factors: 1.482, 1.714, and 1.75 are the mean teammate-report counts for NA, EUW, and KR, respectively, for assisting-enemy or intentional-feeding cases.Regional differences in teammate reports are statistically significant.
- Team-cohesion and Performance: Under 15% winning ratios for intentional feeding and assisting the enemy show that reported accused players are disproportionately on losing teams.The overall winning ratios for reported toxic behavior categories are below 50%.
DISCUSSION AND CONCLUSION
The discussion interprets large-scale LoL reports and crowdsourced decisions as evidence about reporting, toxic-behavior ambiguity, and group influences. It extends these findings to the design of online games, gamified systems, online communities, distributed teams, and big-data HCI research.
- Low participation in voluntary reporting limits the effectiveness of report-based systems, while explicit requests significantly increase reporting likelihood.The authors suggest actively encouraging reports when designing systems to address toxic behavior.
- Over 10% of Tribunal cases were pardoned, indicating that experienced crowdsourcing can protect players wrongly reported for poor skills or aggressive but non-toxic language.The authors propose quality-control mechanisms and machine-learning augmentation to improve Tribunal efficiency and quality.
- In-group favoritism and out-group hostility alter willingness to report, creating potential bias in observations across team, guild, party, and implicit-group settings.Because competition and group belonging are common across online games, the authors argue these findings extend beyond explicit team competition.
- Toxic behavior in interactive games can also inform understanding of anonymous online communities, distributed workplace conflicts, and goal-oriented groups lacking social connections.The authors connect toxic playing and cyberbullying through the disconnect between virtual and real-world identities, and link remote collaboration to blame among unfamiliar partners.
- The study argues that big data and testable hypotheses can effectively support human-behavior research in HCI, while gaming-derived toxicity may also appear in gamified citizen-science systems.The authors expect undesirable elements of gaming culture to accompany the broader importation of gaming mechanics.
Caveats and Limitations
The study’s findings have important scope and data limitations. Results may not generalize beyond League of Legends or fully capture players’ social relationships and behavior over time.
- Applying the findings to other domains requires care because the dataset comes from a game.The authors also note that domain-specific concerns may limit transferability even across games in the same genre.
- The results come specifically from League of Legends, whose communication features differ from other multiplayer online battle arenas.For example, SMITE lacks all-talk chat, while SMITE and Dota 2 include integrated voice communication that may affect cyberbullying.
- The study excludes players’ social-network relationships because the necessary data were unavailable.The authors identify playing with friends as relevant to performance and toxicity and suggest richer network data for more sophisticated hypotheses.
- Anonymization prevents tracking toxic players’ behavior before and after matches were aggregated into cases and decisions.The dataset also does not show how other players typically behave, leaving some questions unanswered.