PaperlayerView exact revision on arXiv

Source-linked AI summary

"Shut Up and Let Me Enjoy My Otome": Understanding and Measuring the Toxicity in Otome Game Communities

Yage Zhang, Xinyue Shen, Yukun Jiang, Michael Backes, Yang Zhang

arXiv:2609.08009v1cs.CRcs.SI

TL;DR

Otome-community toxicity remains systematically underexplored despite the genre’s distinct social structures. OtomeSCAN analyzes posts from Weibo and Reddit and finds platform-specific toxicity patterns, including different prevalence levels across the two platforms.

  • Problem

    Existing work focuses largely on mixed-gender games, leaving otome-community toxicity—including its prevalence, targets, temporal dynamics, and linguistic characteristics—without systematic understanding.

    Source: §1 · pp. 1–2

  • Method

    OtomeSCAN analyzes 620,045 posts from Weibo and Reddit and evaluates toxicity detectors using 4,308 manually annotated posts.

    Source: §1 · p. 2

  • Results

    Weibo otome discussions are more toxic than general gaming (22.20% vs. 3.43%), while Reddit rates are similar (3.71% vs. 3.67%); real-world events coincide with toxicity surges.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The findings offer game developers and platform moderators insights for understanding and managing otome-community environments, and the authors call for greater attention to their unique challenges.

    Source: §1 · pp. 2–3

    The general-game comparison includes Genshin Impact as a game-level comparator, selected for similar story updates and monetization.

    Source: §3.1.1 · p. 4

Abstract

from arXiv · show

Otome games, a romance simulation genre primarily targeting female, have emerged as a major force in the global gaming market, attracting hundreds of millions of players and billions in revenue. Despite their popularity, otome game communities face pervasive online toxicity, which has been largely unexplored. In this work, we present the first large-scale measurement of toxicity in otome game communities across social platforms. We introduce OtomeSCAN, a framework for collecting, evaluating, and analyzing 620,045 posts from Weibo and Reddit spanning 18 months. To support robust analysis, we manually annotated a ground-truth dataset of 4,308 posts, identifying eight target groups such as players and game developers. We evaluate seven toxicity detectors on the dataset, including general-purpose models and our proposed LLM-based detectors, with our best model achieving F1-scores of 0.82 (Weibo) and 0.78 (Reddit). Our analysis reveals significant platform-based differences in toxicity: 22.20% of otome-related posts on Weibo are toxic, compared to 3.71% on Reddit. Besides, real-world events like in-community conflicts can rapidly escalate toxicity, with toxicity ratios increasing to 37.09% in just 72 hours during an external attack on Weibo. We also flag 191 potential-coordination clusters in otome game communities, 64.40% of which target game developers, with several accounts participating repeatedly across multiple clusters. We hope our work inspires further research on community-specific toxicity and contributes to building healthier online spaces for marginalized gaming communities.

1 Introduction

Otome-game toxicity remains systematically underexplored, despite the genre’s distinct community structures and widespread hostility. The paper introduces OtomeSCAN to measure toxicity across platforms and analyze its targets, timing, language, and possible coordination.

Main Findings

  • Real-world events coincide with toxicity surges; during an external attack, toxicity toward otome players rose from 22.20% to 37.09% within 72 hours.Across events, the distribution of targeted groups shifts over time and increasingly includes players.Source: §1 · p. 2

Our Work

  • OtomeSCAN analyzes 620,045 posts across Weibo and Reddit, uses 4,308 manually annotated posts to evaluate seven detectors, and examines prevalence, targets, interactions, events, language, and potential coordination.The annotations include binary toxicity labels and target groups; the best-performing detector achieves F1-scores of 0.82 on Weibo and 0.78 on Reddit.Source: §1 · p. 2

Main Findings

  • Weibo otome discussions are far more toxic than general gaming (22.20% vs. 3.43%), while Reddit rates are similar (3.71% vs. 3.67%).On Weibo, players receive 47.2% of toxic targeting, compared with 27.4% in general gaming; on Reddit, otome toxicity shifts toward NPCs rather than game companies.Source: §1 · p. 2
  • 39.89% of model-detected unique toxic spans on Weibo use variation strategies, compared with 22.77% on Reddit, where toxicity is more direct.Strategies include slang, abbreviation, and substitution, which may help toxic content evade platform moderation.Source: §1 · p. 2
  • OtomeSCAN flags 191 potential-coordination clusters, 64.40% targeting game developers, and several accounts recur across multiple clusters.The authors characterize these as observable patterns warranting further investigation.Source: §1 · p. 2

Contributions

  • The paper releases its annotated dataset and classifier, and argues that the findings offer insights for developers and moderators addressing otome-community toxicity.It calls for greater attention to the unique challenges faced by otome game communities.Source: §1 · pp. 2–3

2 Preliminaries and Related Work

Otome games have socially and emotionally consequential communities, but their toxicity has received little systematic study. Prior work motivates a distinct investigation by highlighting genre-specific social structures and challenges for toxicity detection.

Otome Games

  • Otome games are narrative-driven romance simulations in which players typically adopt a female protagonist and develop relationships with male non-player characters.Players gather in platform-specific spaces such as Weibo Super Topics and Reddit subreddits for discussion and collective action.Source: §2 · p. 3
  • Single-gender player demographics, parasocial bonds, and fan-circle practices may shape toxicity patterns differently from those in mixed-gender gaming communities.The practices include coordinated comment control, voting campaigns, and targeted harassment.Source: §2 · p. 3

Related Work

  • Prior otome-game research examines emotional attachment, social support, intimacy, gender, and fan labor, while toxicity prevalence, targets, and interaction patterns remain underexplored.The paper adopts a toxicity definition requiring direct insults, slurs, threats, or harassment toward a person, group, or character; profanity or strong dissatisfaction alone is insufficient.Source: §2 · p. 3
  • Existing toxicity detectors may be evaded by coded speech and homophone substitutions, while slang-filled otome discourse poses a specific challenge for current models.Prior gaming research documents gender-based hostility and harassment, but quantitative understanding of otome-community toxicity remains limited.Source: §2 · pp. 3–4

3 OtomeSCAN

OtomeSCAN collects and analyzes otome-community posts across Weibo and Reddit, using annotated data to evaluate toxicity detectors before labeling the full corpus for analysis.

3.1.1 Investigated Communities

  • The study compares four otome communities with Genshin Impact and general-game control communities across its two platforms.Genshin Impact serves as a game-level comparison, while the general-game groups provide genre-level comparisons.Source: §3.1.1 · p. 4

3.1.2 Data Collection

  • Reddit posts come from six specified subreddits, while Weibo collection uses game-specific Super Topics and keyword searches for broader groups.The platforms were selected for fan communities with moderators or hosts and for their relevance to observing player discussion.Source: §3.1.2 · pp. 4–5

3.1.3 Data Pre-Processing

  • The dataset contains 239,880 Weibo and 380,165 Reddit posts collected from January 1, 2024, to May 31, 2025.After deduplication and removal of deleted or removed posts, language checks found 99.04% of Weibo posts were Chinese and 94.86% of Reddit posts English.Source: §3.1.3 · p. 5

3.1.4 Data Sampling and Human Annotation

  • Annotators labeled sampled posts for toxicity and coded targets in toxic posts, producing eight target groups and full-annotation Cohen’s Kappa scores of 0.84 and 0.82.The respective scores are for binary toxicity and eight-category target-group labels.Source: §3.1.4 · pp. 5–6

3.2 Detector Evaluation

  • The detector evaluation compares general-purpose tools with LLM-driven models and adopts a 5-shot Reasoning prompt as the best balance of performance and cost.The selected setup uses in-context prompting rather than fine-tuning on the small domain-specific annotated sample.Source: §3.2.1 · p. 6, §3.2 · p. 6, §3.2.2 · p. 7

3.2.3 Experimental Results and Model Selection

  • DeepSeek-V3 achieves the highest Weibo toxicity-detection F1-score (0.82), while GPT-4o-mini leads on Reddit with 0.78; target-group accuracy reaches 0.89 and 0.85, respectively.The study reports these results for binary toxicity detection and, separately, target-group identification.Source: §3.2.3 · p. 7, Table 4 · p. 7, Table 5 · p. 7

3.3 Analysis

  • The selected detectors label all 620,045 posts for subsequent analyses of prevalence and patterns, temporal dynamics, and linguistic expression.DeepSeek-V3 is selected for Weibo and GPT-4o-mini for Reddit.Source: §3.3 · pp. 7–8, §3.2.3 · p. 7

4 Prevalence and Patterns of Toxicity

Toxicity prevalence, targets, and engagement patterns differ between the sampled Weibo and Reddit communities. Weibo toxicity more often centers on factional conflict, while Reddit toxicity more often concerns characters, game content, or developers.

Overall Toxicity Analysis

  • 22.20% of Weibo otome posts are toxic, compared with 3.71% on Reddit; general-game rates are 3.43% and 3.67%, respectively.The reported relative risk for the otome-platform gap is 5.99.Source: §4 · p. 8

Impacts of Toxicity on User Interaction

  • Toxicity correlates with fewer likes and comments on Weibo but more comments on Reddit; Weibo posts using evasion strategies receive more comments and reposts.For Love and Deepspace on Weibo, the comment coefficient is -2.01 (p < .001); posts using evasion strategies average 26.76 comments versus 8.37 and 24.64 reposts versus 10.98.Source: §4 · p. 8

Target Group Analysis

  • 47.20% of Weibo otome toxic posts target players and 29.84% developers, whereas Reddit posts more often target NPCs (36.49%) or developers (22.75%).The distributions differ significantly (χ2 test, p < 10^-180); Figure 5 presents target-group proportions.Source: §4 · p. 8, Figure 5 · p. 9

Thematic Analysis of Toxic Posts

  • Factional attacks or identity-marking language occur in 55.4% of coded Weibo posts versus 9.8% on Reddit, while 51.0% of coded Reddit posts concern parasocial or consumer/developer grievances.The coded Weibo toxicity more often centers on factional conflict; Reddit toxicity more often concerns characters, game content, or developers.Source: §4 · p. 9

5 Temporal Dynamics

The temporal analysis identifies eight events associated with toxicity peaks and finds that most are in-community conflicts. The reported relationships are correlational, not causal.

Temporal Analysis

  • Five of the eight identified events associated with toxicity peaks are in-community conflicts; game updates, external attacks, and consumer-rights protests account for one each.Events were identified through normalized time series and peak detection, then grouped using keyword similarity in a 14-day window.Source: §5 · p. 9, Table 6
  • The luxury-brand promotion event coincided with a +4.30% Weibo toxicity increase, with discussion moving from criticism of developers to cultural debate and player infighting.The debate concerned whether female players should be regarded as romantic partners within the game world or as fans.Source: §5 · pp. 9–10
  • 14.89% toxicity surge coincided with rappers publicly disparaging otome players on Weibo; targets then shifted from identity groups to developers and players.The target shift is reported in the event analysis.Source: §5 · p. 10

Take-Aways

  • Toxicity peaks on Weibo and Reddit show little temporal overlap in the studied communities, likely reflecting their distinct languages and user bases.The authors describe event-peak relationships as correlational rather than causal.Source: §5 · pp. 9–11

6 Linguistic Features of Otome Toxicity

The linguistic analysis extracts toxic spans from model explanations and finds platform differences in both toxic vocabulary and variation strategies. The variation estimates are limited to spans identified by the selected classifiers.

6.1 Toxic Spans

  • The analysis identifies 4,013 unique Chinese and 606 unique English toxic spans, with Weibo vocabulary including fan-circle jargon and group-specific derogatory terms absent from the Reddit examples.Reddit toxic spans include terms expressing dissatisfaction with game content or company operations.Source: §6.1 · p. 11
  • The span-extraction method intersects tokens in detector-generated toxicity explanations with tokens in the original posts to retain text the model identifies as toxic.Weibo posts are tokenized with Jieba, while Reddit posts use alphabetic tokens of at least two letters.Source: §6.1 · p. 11

6.2 Variation Strategies in Toxic Spans

  • 39.89% of Weibo unique toxic spans use at least one variation strategy, compared with 22.77% on Reddit.Slang and memes are the most common strategy on both platforms; these rates are lower-bound estimates because the analysis starts with classifier-identified toxic posts.Source: §6.2 · p. 11

7 From Individual Toxicity to Potential Coordination

The section flags clusters of similar toxic posts as potential coordination, then examines their scale, persistence, targets, and contributing accounts. The observed patterns are candidates for further investigation, not proof of coordinated intent.

Potential-Coordination Detection

  • Clusters are formed from posts with cosine similarity of at least 0.80, then filtered for a toxic post and manually reviewed as candidates.The review found template-like or near-duplicate wording in 49.74% of candidate clusters; the authors retain candidates rather than claiming confirmed coordination.Source: §7 · p. 12

Potential-Coordination Cluster Analysis

  • Most clusters are small: the median is 2 posts and 2 contributing accounts, while median duration is about 13 days.The 75th-percentile duration is close to 128 days, indicating that a minority persist substantially longer.Source: §7 · p. 12, Figure 8 · p. 12

Categorization of Potential-Coordination Clusters

  • Low-scale, short-persistence clusters are most common (42.93%), while high-scale, short-persistence clusters are least common (6.81%).Developers are the most frequent target in all four categories; long-persistence categories have more diverse targets.Source: §7 · p. 12

Case Study: Repeated Participation Across Clusters

  • A repeated group of accounts appears in six high-scale, short-persistence clusters, including a verified influencer with over 2,000 followers.Repeated participation is observable, but does not establish coordinated effort or exclude independent participation.Source: §7 · p. 13

Potential-Coordination Cluster Analysis

  • The procedure flags 191 potential-coordination clusters; game developers are the most frequent target across categories, while long-persistence clusters involve broader targets.The study also notes repeated account participation, but does not establish coordination or participants’ intent.Source: §7 · p. 13

8 Discussion

The discussion characterizes Weibo and Reddit as having qualitatively different conflict patterns, while noting that platform mechanisms are not isolated by the study. It argues for evaluating moderation approaches suited to each platform’s patterns.

Cross-Platform Patterns of Conflict

  • On Weibo, 55.4% of coded toxic posts involve factional attacks or identity-marking language, and 42.4% use in-group slang.The authors describe this as consistent with identity signaling in factionalized fan subcultures, but say post content does not establish user motivations.Source: §8 · p. 13
  • On Reddit, 51.0% of coded toxic posts concern parasocial or consumer/developer grievances, often about writing or monetization.The authors suggest threaded discussion may support sustained argument, but do not isolate platform effects from language, user composition, or moderation.Source: §8 · p. 13

Implications for Socio-Technical Security

  • For Weibo, the authors propose evaluating reversible interventions such as algorithmic de-amplification or reputation systems.The possible amplifying role of Super Topics remains a hypothesis because the study lacks a controlled comparison with non-Super-Topic posts.Source: §8 · p. 13
  • For Reddit, the authors propose testing credible grievance channels and tools that distinguish strong criticism from identity-based harassment.They caution that heavy censorship could reinforce distrust.Source: §8 · p. 13

9 Conclusion

The conclusion presents OtomeSCAN as a large-scale investigation of toxicity across multiple dimensions in otome game communities. The appendices describe the released annotated dataset and ethical safeguards and constraints.

9 Conclusion

  • The study presents its work as the first large-scale investigation of toxicity in otome game communities, measuring prevalence, targets, interactions, temporal dynamics, linguistic features, and potential coordination.The authors hope it advances research on sociotechnical factors and safer, more inclusive digital environments.Source: §9 · pp. 13–14

A Open Science

  • The authors publicly release 4,308 manually annotated posts, with toxicity and target labels, through the OtomeSCAN dataset repository.The released dataset includes 2,255 Weibo posts and 2,053 Reddit posts across six communities per platform.Source: A Open Science · p. 17

B Ethical Considerations

  • The study uses only public Weibo and Reddit posts and acknowledges that not consulting moderators limits its account of community-specific research-use expectations.The authors remove or pseudonymize identifying information, but note that re-identification risk cannot be completely eliminated.Source: B Ethical Considerations · p. 17
  • Two domain-expert authors performed manual annotation to avoid exposing external annotators to potentially distressing content.They could take breaks or stop annotation to mitigate potential psychological impact.Source: B Ethical Considerations · p. 17
  • The authors identify dual-use risks in findings about linguistic evasion and potential-coordination clusters, and withhold the full corpus while limiting released materials.They also commit to responsible disclosure to Weibo and Reddit safety and security teams.Source: B Ethical Considerations · p. 17

C Sample Size Determination for Annotation

The appendices document the annotation sample-size calculation, Weibo keyword collection and validation, toxicity boundary rules, and alternative prompt designs for classification.

C Sample Size Determination for Annotation

  • For annotation, the authors calculate an initial sample size of 384 at 95% confidence and a 0.05 margin of error, then apply a finite-population correction.For the Mr. Love super topic, the adjusted example is approximately 374 posts.Source: C Sample Size Determination for… · p. 17

D Keyword Search Strategy

  • Weibo collection uses an iteratively expanded list of otome keywords and filters posts by specified Super Topics; a random 1% review found genre-specific precision above 99%.Reddit data are obtained by subreddit rather than keyword search.Source: D Keyword Search Strategy · pp. 17–18

E Boundary Cases for Toxicity Annotation

  • Annotation distinguishes criticism of an artifact from attacks on people, groups, or characters; profanity alone does not make criticism toxic.The authors acknowledge that some boundary cases remain genuinely ambiguous.Source: E Boundary Cases for Toxicity Annotation · p. 18

F Prompt Design Details

  • The prompt designs range from multi-step chain-of-thought to concise toxicity and target instructions, with a reasoning variant that adds brief explanations for both labels.Chinese and English example variants are used; the reasoning fields are the structural difference between the concise and reasoning prompts.Source: F Prompt Design Details · p. 18

G Error Analysis

The selected classifiers still miss culturally specific and obfuscated toxic expressions, while false-positive patterns also differ across Weibo and Reddit. Additional checks find platform-specific target confusions but high agreement between isolated-post and thread-context labels.

False negatives

  • 20.0% of Weibo and 22.0% of Reddit toxic posts are false negatives for the selected classifiers, with culturally specific or obfuscated language especially challenging.On Weibo, misses often involve homophones, abbreviated profanity, and cultural allusions; on Reddit, they more often involve genre-specific slang, sarcasm, and context-dependent profanity.Source: G Error Analysis · pp. 19–20

False positives

  • False positives include sarcastic complaints about game companies on Weibo and profanity used for emphasis or humor on Reddit.These errors point to lexical-cultural mismatch between general-purpose language understanding and otome-specific discourse.Source: G Error Analysis · p. 20

Target identification

  • Players and developers are the most frequent target-confusion pair, especially in gacha complaints, and this confusion appears more often on Weibo.Ambiguous outbursts and insider slang also prevent specific target attribution.Source: G Error Analysis · p. 20

Toxicity granularity

  • 68% of sampled Weibo toxic posts were direct hostility, while 46% of sampled Reddit posts were aggressive disagreement; benign false positives were 2% and 4%, respectively.The sample included 100 model-labeled toxic posts per platform.Source: G Error Analysis · p. 20

Thread-level validation

  • Thread-context re-annotation agreed strongly with isolated-post labels (κ = 0.91 for toxicity; κ = 0.87 for targets), though context can enrich interpretation.The table of boundary cases supports distinguishing disagreement from toxicity.Source: G Error Analysis · p. 20, Table 9 · p. 20

H Prompt Optimization and Ablation Studies

Prompt design, example count, target composition, and robustness checks qualify how the study’s detection and cross-platform findings should be read. Prompt 3 performs best, while the target-pattern checks preserve the distinction between human and NPC targets.

H Prompt Optimization and Ablation Studies

I Target Composition

  • +14.89% toxicity coincided with the August 2024 rapper attack; identity-group targeting fell from 38.1% while developer targeting rose from 7.8% to 19.5%.Player-on-player toxicity remained roughly 27–30%, indicating a shift in target composition as well as a spike.Source: I Target Composition · pp. 21–22, Table 14 · p. 22

J.1 Interaction Regression Details

  • 10,000 bootstrap iterations produced non-overlapping player and developer target-proportion confidence intervals on both platforms.Weibo: players [43.8,50.6]% vs. developers [26.5,33.2]%; Reddit: players [6.1,13.3]% vs. developers [17.6,27.9]%.Source: §J.1 · p. 22, Table 18 · p. 23
  • After merging players and developers, Weibo toxicity remains overwhelmingly human-targeted (92.9%), while Reddit toxicity is predominantly NPC-targeted (51.1%).Sampled NPC-targeted posts more often concern narrative or character design, while developer-targeted posts more often concern operations, monetization, or content quality.Source: §J.1 · p. 22
  • Only one toxic post appeared in each sampled non-otome platform-community pair, a sanity check rather than a population-level estimate.The authors note that larger cross-community samples are needed for a definitive comparison.Source: §J.1 · p. 22, Table 17 · p. 23

K Sensitivity Analysis

The sensitivity analysis finds developer-targeted clusters remain dominant across similarity thresholds, while the proposed evasion and early-warning approaches are tailored to observed patterns. Their stated scope leaves semantic-diverse and slow-posting coordination to complementary methods.

K Sensitivity Analysis

  • 64.40% of 191 flagged potential-coordination clusters target developers, and that target dominance remains stable across tested similarity thresholds.Lower thresholds include looser event-related posts; higher thresholds retain more near-duplicates, with θ = 0.80 chosen to balance recall and precision.Source: K Sensitivity Analysis · p. 23

L Toxicity Detection Model Details

  • The evaluated detector set includes general-purpose tools such as Perspective API and COLD, alongside LLMs including DeepSeek-V3 and GPT-4o mini.COLD is a Chinese offensive-language classifier; the named LLMs are included in the model descriptions.Source: L Toxicity Detection Model Details · pp. 23–24

M Evasion-Aware Detection

  • Nearly 40% of Weibo toxic spans use variation strategies, and those evasion-laden posts receive more engagement than direct toxic posts; the proposed modules could reduce false negatives.The current Weibo false-negative rate is reported as 19.8%.Source: M Evasion-Aware Detection · p. 24
  • The four proposed preprocessing modules target slang and memes, abbreviations, homophone or visual substitution, and emoji substitution.Examples include lexicon updates, abbreviation expansion, phonetic canonicalization, and an emoji-in-context classifier.Source: M Evasion-Aware Detection · p. 24

N Early-Warning Indicators for Potential Coordination

  • Five candidate early-warning signals cover content similarity, new-account influx, toxicity velocity, target concentration, and high-reach-account activity.The signals draw on observed lexical homogeneity, concentrated timing, new accounts, toxicity increases, concentrated targets, and a verified account in six clusters.Source: N Early-Warning Indicators for… · p. 24
  • The signals are intended to prioritize human review, not establish coordination automatically; thresholds should be calibrated separately for each community.The proposed tiered design uses two co-activated signals for expedited review and three or more for temporary, reversible visibility reduction pending review.Source: N Early-Warning Indicators for… · p. 24
  • The framework is tailored to lexical homogeneity, temporal concentration, and low-activity accounts; semantic-diverse or slow-posting patterns require complementary detection approaches.Source: N Early-Warning Indicators for… · pp. 24–25

O Platform-Specific Intervention Experiments

The paper proposes separate, lightweight and reversible experiments for Weibo and Reddit, reflecting their opposing toxicity–engagement patterns. Both designs are proposed for moderator piloting rather than reported as evaluated interventions.

Weibo experiment

  • Weibo’s proposed test de-amplifies posts flagged as toxic in hot-post and Super Topic feeds while retaining direct-search access, with matched communities compared over at least eight weeks.Primary outcomes are weekly toxicity ratio, non-toxic engagement, and retention.Source: O Platform-Specific Intervention… · p. 25

Reddit experiment

  • Reddit’s proposed test routes developer-critical posts to a weekly structured feedback megathread, compared with a status-quo subreddit.Outcomes include developer-targeted toxic posts in general threads and an LLM-assessed constructiveness score.Source: O Platform-Specific Intervention… · p. 25

Implementation

Loading 2609.08009v1…