PaperlayerView exact revision on arXiv

Source-linked AI summary

STFU NOOB! Predicting Crowdsourced Decisions on Toxic Behavior in Online Games

Jeremy Blackburn, Haewoon Kwak

arXiv:1404.5905v1cs.SIcs.CYphysics.soc-ph

TL;DR

Toxic-behavior judgments in League of Legends rely on crowdsourced Tribunal reviews that require substantial human effort. This paper trains supervised classifiers on crowdsourced data and finds they predict decisions effectively, including across regions.

  • Problem

    The Tribunal addresses difficult-to-define toxic behavior through crowd judgments, but requires substantial cost, time, and human effort.

  • Method

    The paper trains supervised classifiers using large-scale crowdsourced decisions and information available to Tribunal reviewers, including performance, reports, and chat features.

  • Results

    The classifier discriminates guilty from innocent behavior about 80% of the time and overwhelming agreement on innocence about 88% of the time.

  • Takeaways & Limitations

    The findings support using machine learning to assist toxic-behavior detection, with potential portability across regions and practical cost savings and victim protection.

  • Takeaways & Limitations

    Classifier accuracy might improve when training data comes from the same meta-game as the cases being examined.

Abstract

from arXiv · show

One problem facing players of competitive games is negative, or toxic, behavior. League of Legends, the largest eSport game, uses a crowdsourcing platform called the Tribunal to judge whether a reported toxic player should be punished or not. The Tribunal is a two stage system requiring reports from those players that directly observe toxic behavior, and human experts that review aggregated reports. While this system has successfully dealt with the vague nature of toxic behavior by majority rules based on many votes, it naturally requires tremendous cost, time, and human efforts. In this paper, we propose a supervised learning approach for predicting crowdsourced decisions on toxic behavior with large-scale labeled data collections; over 10 million user reports involved in 1.46 million toxic players and corresponding crowdsourced decisions. Our result shows good performance in detecting overwhelmingly majority cases and predicting crowdsourced decisions on them. We demonstrate good portability of our classifier across regions. Finally, we estimate the practical implications of our approach, potential cost savings and victim protection.

0. EXECUTIVE SUMMARY

Toxic behavior threatens online-game communities, while human-led and crowdsourced systems require substantial effort and time. This paper uses machine learning to predict Tribunal decisions and reports strong discrimination of guilt, innocence, and broad practical relevance.

  • Motivation: Toxic behavior threatens game communities and can impose substantial costs on operators and players.The passages associate toxicity with customer-support burden, player fatigue, churn, and community damage.
  • Existing responses: Human intervention is accurate but does not scale, while the League of Legends Tribunal remains slow and labor-intensive.Tribunal cases require multiple reports before review by community panelists.
  • Approach: The paper analyzes millions of matches and corresponding Tribunal decisions to identify how reviewers judge accused players.Reviewers use in-game performance, other players’ performance, chat logs, and reports.
  • Results: About 80% of cases are discriminated between guilty and innocent behavior, while 88% are discriminated for overwhelming agreement on innocence.These are the reported discrimination rates for the two decision settings.
  • Implications: The findings suggest machine learning can assist crowdsourced moderation and may support automatic detection of related harmful behavior.The paper identifies possible gaming-industry impact and potential applications such as cyberbullying detection.

1. INTRODUCTION

Toxic behavior is pervasive yet difficult to define consistently, and existing reporting systems have practical and perceived shortcomings. The paper addresses this challenge with large-scale supervised learning over crowdsourced Tribunal data and examines portability and practical implications.

  • Problem: Toxic behavior is widespread in multiplayer games, but its boundaries vary across games and individuals.Different customs, rules, and ethics make griefing and related behavior difficult to define objectively.
  • Problem: Existing reporting systems can be ineffective or misused, including reports that allegedly target innocent players.The introduction describes both claimed reductions in toxic communication and contrary player experiences.
  • Tribunal: The Tribunal applies majority voting to subjective toxicity judgments but requires substantial time, cost, and human effort.Its crowdsourced judgments address behavioral vagueness while creating a scaling challenge.
  • Method: The study uses over 10 million reports involving 1.46 million toxic players, 534 features, and a Random Forest classifier.Features come from in-game performance, reports, and chats.
  • Contributions: The classifier detects overwhelming-majority cases, identifies decision features, transfers across regions, and supports estimates of savings and victim protection.The paper presents these as central contributions and practical implications.

2. BACKGROUND

League of Legends is a five-player-team competition in which players can report suspected toxicity after matches. The Tribunal aggregates reported matches and uses community majority votes, with safeguards intended to support careful review.

  • Game: League of Legends is a match-based team competition typically played by randomly matched teams of five.Players choose lanes and roles while attempting to destroy the opposing Nexus.
  • Reports: Players can report suspected toxicity in ten predefined categories, including intentional feeding, offensive language, and leaving the game.The categories also include negative attitude, spamming, and refusing to communicate.
  • Toxicity: Intentional feeding means deliberately allowing deaths, while leaving the game means remaining at base for the entire match.The background describes these as examples of toxic play.
  • Tribunal: The Tribunal determines whether reported players should be punished through majority votes that incorporate subjective community judgments.Its purpose is to account for differing perceptions of toxic behavior.
  • Case construction: A case can contain up to five randomly selected matches, with reports, comments, chat logs, and scoreboards for reviewers.Players are brought to the Tribunal after being reported hundreds of times across dozens of matches.
  • Review process: Reviewer safeguards include daily case limits, minimum decision times, and a skip option for difficult cases.Reviewers later see final decisions and agreement levels, with accuracy and ranking used for gamification.

3. RELATED WORK

Prior work frames toxic gaming behavior as a form of cyberbullying with unclear boundaries and examines crowdsourcing for noisy subjective judgments. This paper builds on research combining human effort with machine learning for scalable labeling and prediction.

  • Toxicity research: Toxic gaming behavior is related to cyberbullying because it involves harmful behavior through electronic channels.The literature connects hostility in computer-mediated communication with the absence of face-to-face social cues.
  • Toxicity research: Griefing has unclear boundaries because gaming customs, rules, ethics, and individual perceptions differ.This vagueness can prevent even griefers from recognizing their own behavior as griefing.
  • Toxicity research: Prior studies categorize griefing by behaviors such as harassment, power imposition, scamming, and greed play, and examine its relation to anger.These studies emphasize behavioral categories and intentions.
  • Crowdsourcing: Crowdsourcing research identifies noisy judgments as a challenge even when labelers are experts on subjective tasks.Majority voting and repeated labeling are established approaches for improving annotation quality.
  • Crowdsourcing: Crowdsourced labels can support machine learning, with high-agreement training data previously associated with better classifier accuracy.Other work also finds that incorporating individual labeler accuracy can affect overall performance.
  • Human-machine systems: CrowdFlow and CrowdSynth motivate treating supervised learning and human crowds as complementary mechanisms for balancing workloads.The cited systems combine classifiers with human labor or guide human labeling with predictions.

4. DATA COLLECTION

The study crawled Tribunal cases from three League of Legends regions, using distributed crawlers to assemble a large crowdsourced dataset. Each case contains multiple matches, which contain one or more reports.

  • Data collection: 1.46 million Tribunal cases comprising 10 million user reports were collected with distributed crawlers designed not to degrade Riot’s web servers.The crawlers paused between requests and remained idle for seconds before new requests.
  • Data collection: Data were collected from North America, Western Europe, and South Korea to represent regional and cultural variation.The authors assumed most players connect to the geographically closest regional server.
  • Data collection: The collection crawled all available Tribunal cases from the three regions in April 2013.A Tribunal case includes multiple matches, and each match contains one or more reports.
  • Data collection: Table 1 summarizes the collected data across the three regional servers.

5. RESEARCH QUESTIONS

The paper asks whether machine learning can assist the Tribunal’s time-consuming majority-vote decisions and what predictive features reveal about toxic behavior. It considers task design, training-set agreement, linguistic portability, and feature interpretation.

  • Research questions: RQ1 asks whether machine learning can predict crowdsourced Tribunal decisions about subjective toxic behavior.The motivation is to assist a majority-rule process that requires enough votes and therefore takes substantial time.
  • Research questions: Possible tasks include classifying decision and agreement combinations, separating punished from pardoned cases, and identifying high- or low-agreement cases.The paper emphasizes that application matters alongside classifier accuracy.
  • Research questions: The authors create and compare training sets because Riot’s divisions among majority, strong-majority, and overwhelming-majority agreement are unknown.Prior work suggests high-agreement crowdsourced responses are better for classifier training.
  • Research questions: Non-linguistic features are examined to improve portability because Tribunal chats may use different languages, including Korean.The paper argues that greater reliance on non-linguistic features could increase generality.
  • Research questions: RQ2 asks which features predict Tribunal decisions and what those features imply about toxic behavior across games or online communities.The authors view transferable features as evidence that the classifier may extend beyond League of Legends.

6. FEATURES

The feature set summarizes Tribunal cases using in-game performance, participant reports, and chat valence. The design accounts for variable matches, limits of performance metrics, and linguistic evidence associated with punishment.

  • Feature sources: The Tribunal is modeled as two stages: player reports from observed matches followed by case-level judgments using aggregated evidence.The feature representation combines match-derived domain values with information supplied by Stage 1 participants.
  • In-game performance: Variable match counts, unordered sequences, and cross-player comparability constrain interpretation of extracted in-game values.A low-skill non-toxic player and a high-skill toxic player may both be reported for intentional feeding, making KDA alone insufficient.
  • In-game performance: Per-case performance features average statistics across matches by report type and add standard deviations to represent inter-match differences.
  • User reports: User-report features include allied and enemy report counts, optional offense text, and the most common behavior type across matches.
  • In-game performance: The in-game feature set contains 364 performance features, including kills, deaths, assists, damage, gold, and time played.The large feature count results from grouping per-match values by the most common report type.
  • Chats: Around 60% of cases have more than about 25% of matches reported for offensive language or verbal abuse, motivating chat-based features.
  • Chats: Chat valence is computed from ANEW word scores and frequencies, with vtext set to 0 when no ANEW words occur.The method is described as robust to typos and abbreviations across text genres.
  • Chats: Punished cases tend to have lower chat valence than pardoned cases, while punished and pardoned offender means are 5.725 and 5.779, respectively.The distributions indicate that chat logs contain quantifiable toxicity-related information.

7. MODELS

The paper builds separate models for in-game performance, user reports, chats, and all features to compare information sources. The models reflect performance, perceived severity, and conversational context in Tribunal decisions.

  • Model design: Four models compare feature sources: in-game performance, user reports, chats, and a full model containing all features.
  • In-game performance model: The performance model separates offender performance from other players’ performance to represent intentional feeding and team-based attribution.It is trained on reviewer inference of intention rather than a judgment of player skill.
  • In-game performance model: KDA alone has no correlation with Tribunal decisions, with the best- and worst-KDA players about equally likely to be punished.
  • User report model: More reports in a single match are highly correlated with the likelihood of punishment, motivating the user-report model.
  • Chat model: The chat model supplies context about match events and other players’ perceptions that reviewers cannot recover from final performance alone.It targets behavior expressed through verbal abuse and offensive language.

8. RESULTS

The results show that classifier performance improves when training emphasizes cases with stronger reviewer agreement, while decision prediction depends on chat, report, and performance information. The models perform especially well on clear-cut cases, with some portability across regions but measurable regional differences.

  • 8.1 Decision confidence and classifier training: Training on overwhelming-majority cases produces the highest AUC across agreement levels, while majority-case training performs considerably worse.Strong-majority training has similar performance to overwhelming-majority training.
  • 8.1 Decision confidence and classifier training: Clear-cut pardon and punish cases can be extracted and classified accurately, potentially assigning easy cases to machines while leaving borderline cases to human reviewers.The experiment also indicates that discriminating among less egregious offenses is more difficult.
  • 8.2 What are Tribunal decisions based on?: 0.7991 AUC is achieved by the full model for predicting Tribunal decisions, compared with 0.7187, 0.7195, and 0.7157 for performance, report, and chat models.The individual models show comparable performance, while the full model performs best.
  • 8.2 What are Tribunal decisions based on?: Chat and report features dominate the five most important full-model features, while performance features remain predictive but require contextual interpretation.Total case valence is the top feature, and the second-ranked feature comes from the report-only model.
  • 8.2 What are Tribunal decisions based on?: 0.8811 AUC is obtained for overwhelming pardons and 0.7461 for overwhelming punishments using all features.The corresponding performance, report, and chat model AUCs are 0.8049, 0.8055, 0.8269 for pardons and 0.6509, 0.6886, 0.6190 for punishments.
  • 8.2 What are Tribunal decisions based on?: The best model differs by decision type: chat performs best for overwhelming pardons, whereas user reports perform quite well for overwhelming punishments.The authors relate this difference to the information reviewers use when establishing innocence versus toxicity.
  • 8.3 Classifier portability: English-based ANEW linguistic features are unusable for the Korean dataset because English words are nearly absent, although fewer than 1% of EUW cases have undefined vtext.The language issue constrains cross-region feature portability.
  • 8.3 Classifier portability: A classifier trained on North American data predicts EUW overwhelming-majority pardons as well as on North American data, although overall EUW decision prediction is weaker.The result supports portability alongside regional differences.

9. DISCUSSION

The paper estimates the Tribunal’s operational costs and considers how supervised prediction could reduce review burden and protect players. It also identifies limitations involving changing game strategies and possible adaptation by toxic players, while proposing earlier, real-time intervention.

  • Operational estimates: 125.85 seconds are required to reach a verdict for one Tribunal case under the paper’s estimate.The estimate assumes 187.5 votes per case and 1.49 votes cast per second.
  • Operational estimates: The Tribunal’s overall accuracy was comparable with the classifier relative to Riot’s in-house decisions.The paper notes approximately 80% agreement between the Tribunal community and Riot’s in-house team.
  • Operational estimates: $470,000 was the estimated first-year cost of Tribunal votes, rising above $1 million by March 2013.The calculation assumes 50% of votes fall into majority decisions and each correct vote costs $0.02.
  • Limitations and consequences: Classifier performance may improve when training data match the meta-game of the cases being examined.Changes in the meta-game affect performance-related features and models, even though the datasets span multiple metas.
  • Limitations and consequences: Toxic players could adapt their language or play style to avoid detection, although adaptation may reduce the negative impact of their behavior.The paper gives sarcastic praise and intentionally limited feeding as examples.
  • Future directions: Real-time detection and adaptive interfaces could protect players before they are exposed to toxic behavior.The paper contrasts this prospective intervention with the Tribunal’s post-exposure review process.
  • Future directions: In-game penalties are proposed as another direction because immediate punishment may be more effective than delayed punishment.The proposal is motivated by the competitive origins of some toxic behavior and the immediacy of reinforcement.

10. SUMMARY

The paper uses crowdsourced decisions to train classifiers for detecting toxic behavior and its severity. It reports accuracy on clear-cut cases, improved performance from high-agreement training, robustness across regions, and potential relief for human reviewers.

  • Summary: The paper trains classifiers using in-game performance, victim reports, and chat-language features to detect toxic behavior and severity.The models use sparse information available to Tribunal reviewers.
  • Summary: Training with high-agreement decisions improved accuracy on low-agreement decisions, while the classifier was especially adept at detecting clear-cut innocence.These findings summarize the paper’s model comparisons and decision patterns.
  • Summary: A classifier trained on North American data performed adequately on a European dataset, indicating robustness across cultural regions.The paper presents this as evidence of portability across regions.
  • Summary: The classifier has the potential to alleviate at least some of the burden placed on human reviewers.The paper describes this implication as preliminary and bases it on a preliminary cost analysis.
Loading 1404.5905v1…