PaperlayerView exact revision on arXiv

Source-linked AI summary

Hidden in Plain Sight: Advancing Harmful Content Understanding via Perception-Aware Adversarial Self-Play

Yisong Xiao, Benlei Cui, Haiwen Hong, Shiji Zhao, Zonghao Ying, Yaoyuan Zhang, Longtao Huang, Aishan Liu, Hui Xue, Ting Ma, Xianglong Liu

arXiv:2610.15449v2cs.AIcs.CL

TL;DR

LLMs can miss harmful intent concealed by visual and phonetic evasions that humans understand. PASP combines multi-perceptual views with adversarial self-play; experiments report lower adaptive attack success rates on two benchmarks and identify Chinese-focused evaluation as a scope boundary.

  • Problem

    LLMs may fail to identify harmful intent in expressions humans understand, while evasions can exploit this gap to avoid moderation.

    Source: §1 · p. 1

  • Method

    PASP combines textual, phonetic, and visual views with attacker–defender self-play, using defender feedback for evasions and intent-invariance rewards for defender training.

    Source: §1 · p. 2

  • Results

    PASP reduces adaptive attack success rates by up to 73.71% on ChineseHarm-Bench and 74.49% on EvoHarmBench versus supervised fine-tuning.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Evaluations report robustness to unseen attack strategies and attackers, minimal degradation of general capabilities, and effectiveness interpreting evasive expressions in e-commerce.

    Source: §1 · p. 2, §6 · p. 10

    Evaluation focuses on Chinese content, whose evasion patterns and platform conventions may not transfer directly to other languages or moderation environments.

    Source: §6 · p. 10

Abstract

from arXiv · show

Despite significant advances in language understanding, Large Language Models (LLMs) can still miss meaning hidden in plain sight, failing to identify harmful intent in expressions readily understood by humans. Malicious users exploit this gap by strategically disguising harmful content to evade automated moderation, posing a serious threat to online platform safety. The vulnerability reflects a fundamental cognitive asymmetry between human readers, who integrate contextual, phonetic, and visual cues to decipher the intent behind deliberate obfuscation, and text-based LLMs, which rely primarily on token sequences and have limited exposure to evolving evasions. To bridge this asymmetry, we propose PASP (Perception-Aware Adversarial Self-Play), which combines multi-perceptual views with adversarial self-play to improve understanding of harmful intent as evasions evolve. Specifically, PASP constructs textual, phonetic, and visual views of each input, preserving its original context while exposing pronunciation and glyph cues obscured in token space. During adversarial self-play, an attacker uses defender feedback to generate intent-preserving evasions, while the defender is trained using an intent-invariance reward to understand harmful intent across changing surface forms. Extensive experiments show that PASP reduces adaptive attack success rates by up to 73.71% and 74.49% on ChineseHarm-Bench and EvoHarmBench, respectively, preserves model capabilities with minimal degradation, and effectively interprets real-world evasive expressions in e-commerce.

1 INTRODUCTION

LLMs can miss harmful intent hidden by phonetic and visual obfuscation, while existing defenses often rely on fixed attacks or text-only cues. PASP combines multi-perceptual views with adaptive self-play and reduces attack success rates on two benchmarks.

1 INTRODUCTION

  • LLMs may miss harmful intent that humans recognize in homophone substitutions or glyph mutations, which malicious users exploit to evade moderation.The authors connect this gap to harmful content circulating online and undermining platform safety.Source: §1 · p. 1
  • Existing defenses often use known attacks or handcrafted heuristics, leaving text-only systems without explicit phonetic and visual cues and fixed attack collections reactive to past evasions.Malicious users can iteratively craft unseen variants in response to moderation feedback.Source: §1 · p. 2
  • Human readers integrate visual, phonetic, and contextual cues and use pragmatic reasoning to infer why a writer chose an unusual form under moderation pressure.The paper presents these capabilities as a model for improving interpretation of adversarial expressions.Source: §1 · p. 2
  • PASP combines textual, phonetic, and visual views with adversarial self-play to improve harmful-intent understanding as evasions evolve.The attacker uses defender feedback to generate intent-preserving variants, while the defender is optimized with an intent-invariance reward.Source: §1 · p. 2
  • PASP reduces adaptive attack success rates by up to 73.71% on ChineseHarm-Bench and 74.49% on EvoHarmBench versus supervised fine-tuning.Experiments use Qwen3.5-9B and GLM-4.6V-Flash.Source: §1 · p. 2

2 RELATED WORKS

Prior work addresses harmful-content detection, adversarial examples, perceptual cues, and safety self-play, but the paper identifies a remaining challenge: tracking disguised harmful intent as attackers adapt. PASP applies self-play to user-supplied content whose intent is obscured.

2 RELATED WORKS

  • Static benchmarks evaluate harmful-content understanding on fixed examples, whereas EvoHarmBench permits attackers to combine and refine tactics such as homophones, character substitutions, and metaphorical framing.The cited benchmarks include ToxiGen, COLD, and ChineseHarm-Bench (R¨ottger et al., 2021; Hartvigsen et al., 2022; Deng et al., 2022; Liu et al., 2025b; Xiao et al., 2024).Source: §2 · p. 2
  • Existing defenses address predefined camouflage with generated examples, phonetic or glyph cues, or Pinyin prompting, but leave open invariant-intent understanding as attackers adapt to moderation feedback.The cited approaches include adversarial-example augmentation and Pinyin-based prompting (Han & Tsvetkov, 2020; Ocampo et al., 2023; Yuan et al., 2024; Li et al., 2020; Guo et al., 2025).Source: §2 · p. 2
  • Prior safety self-play addresses automated red teaming, jailbreak defense, or persona-based attacks, while PASP targets user-supplied content with disguised harmful intent.PASP uses human-readable evasions to expose failures in intent understanding and train defenders to interpret concealed meaning.Source: §2 · p. 3

3 METHODOLOGY

PASP treats harmful-content moderation as recovering intent and risk labels despite readable surface disguises. It combines multi-perceptual inputs with feedback-guided attacker–defender self-play and trains the defender to interpret harmful and benign rewrites consistently.

3.1 PRELIMINARIES

  • The attacker disguises harmful text to mislead the defender while preserving latent intent and human readability; the defender must recover that intent and make correct risk judgments across variants.The problem setup defines intent and readability scores calibrated against predefined thresholds.Source: §3.1 · p. 3

3.2 MULTI-PERCEPTUAL VIEW CONSTRUCTION

  • PASP expands each input into textual, phonetic, and visual views, preserving the original tokens and context while exposing pronunciation and glyph or layout cues.The phonetic view can use Pinyin; the visual view renders text as an image.Source: §3.2 · p. 4

3.3 ADVERSARIAL SELF-PLAY

  • Self-play repeatedly lets an adaptive attacker probe the current defender and uses generated evasions to update the defender to understand intent across variants.The interaction is intended to address the strategic dynamics of evolving disguises.Source: §3.3 · p. 4, §3.3.2 · p. 6

3.3.1 FEEDBACK-GUIDED ADVERSARIAL EXPRESSION SEARCH

  • The attacker selects from 22 composable strategies in six families, using Thompson sampling and defender feedback to guide candidate generation and revise or switch strategies based on prior judgments.The bounded search stops when a candidate meets the adversarial-expression criterion or its step budget is exhausted.Source: §3.3.1 · pp. 4–5

3.3.2 INTENT-INVARIANT DEFENDER LEARNING

  • Defender training rewards intent alignment, audit correctness, and format validity, with greater weight on intent and audit; it also includes readable benign counterparts so disguise alone is not treated as harmful.When all sampled audit judgments are incorrect, a source-intent reference response replaces the lowest-reward response in the group update.Source: §3.3.2 · p. 5

4 EXPERIMENTS

Experiments evaluate PASP on two harmful-content benchmarks, testing static and adaptive performance, false positives on rewritten benign inputs, and retained general capabilities. Further analyses isolate its components and test generalization to unseen attack strategies and attackers.

4.1 EXPERIMENTAL SETUP

  • The study uses ChineseHarm-Bench and EvoHarmBench, evaluates up to 12 feedback-guided attack rounds, and assesses capability retention on four benchmarks.Defenders are Qwen3.5-9B and GLM-4.6V-Flash; GPT-5 mini serves as attacker, and DeepSeek-V3.2-Exp judges candidate validity and intent alignment.Source: §4.1 · p. 6

4.2 MAIN RESULTS

  • PASP achieves the highest Macro-F1 across both benchmarks and backbones, exceeding each setting’s strongest baseline by 1.48, 0.97, 0.46, and 1.11 points.It also records the best abuse and benign-content F1 across all four settings, though some baselines lead on other harmful categories.Source: §4.2 · p. 6, Table 1 · p. 7
  • 73.71% and 74.49% relative ASR reductions on ChineseHarm-Bench and EvoHarmBench, respectively, are reported for GLM-4.6V-Flash versus SFT.PASP also has the lowest ASR in every harmful category across both benchmarks and backbones.Source: Table 2 · p. 7, §4.2 · p. 7
  • PASP’s rewritten-benign FPRs remain close to SFT’s on Qwen3.5-9B, while GLM-4.6V-Flash results vary by benchmark.With Qwen3.5-9B, PASP records 1.11% and 8.14% versus SFT’s 1.00% and 7.89%; with GLM-4.6V-Flash, it records 1.01% and 8.05% versus SFT’s 2.00% and 5.88%.Source: Table 3 · p. 7, §4.2 · p. 7

4.3 ABLATION STUDIES AND FURTHER ANALYSIS

  • Across three unseen attackers, PASP’s mean ASR is 25.65%, versus 61.03% for SFT and 45.66% for PICL.Against Qwen3.8 Max, the most effective attacker in this evaluation, PASP records 32.29% ASR versus 69.80% for SFT and 49.63% for PICL.Source: Figure 5 · p. 8, §4.3 · pp. 8–9
  • With all five context-and-structure strategies withheld from training, PASP reaches 17.32% average ASR versus SFT’s 32.47%; including them lowers PASP’s ASR to 7.95%.The evaluation uses the held-out strategies, and direct training exposure improves resistance across all five.Source: Figure 4 · p. 8, §4.3 · p. 8

4.2 MAIN RESULTS

  • PASP’s average accuracy drops 0.56 percentage points on Qwen3.5-9B, while it rises from 83.54% to 83.68% on GLM-4.6V-Flash.The Qwen3.5-9B drops are 0.84 points for offline adversarial training and 0.91 for PICL.Source: Table 4 · p. 8, §4.2 · p. 8

4.3 ABLATION STUDIES AND FURTHER ANALYSIS

  • Adding perceptual views lowers average ASR from 44.93% to 40.28% under SFT and from 17.61% to 11.46% under self-play; self-play lowers it from 32.10% to 11.46% with views held constant.The full PASP configuration achieves the lowest ASR across all harmful categories.Source: Table 5 · p. 8, §4.3 · p. 8

5 IN-THE-WILD EVALUATION ON E-COMMERCE CONTENT

PASP is evaluated on disguised marketplace posts and brand-name variants, with stronger reported results than the cited comparison systems on both tasks.

Auditing Marketplace Posts

  • PASP achieves 79.33% F1 and 70.33% recall on evasive marketplace posts, compared with PICL’s 76.17% and 65.11%, respectively.The evaluation includes 11,465 evasive pornographic posts and 11,465 benign posts.Source: Table 6 · p. 9, §5 · p. 9

Interpreting Brand Variants

  • PASP scores 2.44 mean and 21.20% score-at-least-four rate on brand variants, versus 2.34 and 15.40% for offline adversarial training.The task evaluates 1,000 variants for brand recovery, variation explanation, and recognition of evasive purpose.Source: Table 6 · p. 9, §5 · p. 9

Overall

  • The e-commerce results support PASP’s ability to recover meaning and evasive intent in platform-governance tasks.The reported tasks cover concealed violations in marketplace posts and disguised brand names.Source: §5 · pp. 9–10

6 CONCLUSION AND FUTURE WORK

The conclusion presents PASP as a multi-perceptual self-play framework for harmful-intent understanding, while noting limits in language coverage and attacker capability.

6 CONCLUSION AND FUTURE WORK

  • PASP combines multi-perceptual views with adversarial self-play and an intent-invariance reward to interpret harmful intent across evolving evasions.The conclusion also reports robustness to unseen attack strategies and attackers, with minimal degradation of general capabilities.Source: §6 · p. 10
  • Evaluation focuses on Chinese content, and attack rewrites may be constrained by the underlying LLM’s capabilities.The authors note that Chinese evasion patterns and platform conventions may not transfer directly to other languages or moderation environments.Source: §6 · p. 10

A MORE DETAILS OF THE METHOD

The appendix details the attacker’s strategy library and feedback-driven rewrites, the defender’s perceptual interpretation, and judge-based assessments of intent preservation.

A.1 ATTACKER

  • The attacker selects from 22 base strategies across six families, applying them individually or in combination and refining intent-preserving rewrites using defender feedback.The prompt instructs it to preserve core entities, actions, purpose, and stance; all five CTX strategies are held out in the unseen-family evaluation.Source: §A.1 · pp. 14–15, Table 7 · p. 14

A.2 DEFENDER

  • The defender combines textual, phonetic, and visual-layout cues to identify entities, actions, purpose, and stance, judging underlying meaning rather than surface form.Homophones, symbols, or unusual layouts alone are not treated as evidence of a violation.Source: §A.2 · p. 15

A.3 JUDGE

  • The judge assesses readability and intent preservation, while the training reward compares the defender’s interpretation with a fixed reference intent.The comparison considers agreement on core entities, actions, purpose, and stance; changes in pronunciation, glyphs, layout, or wording alone do not imply changed intent.Source: §A.3 · p. 16

B.1.1 DATASETS

The appendix describes fixed reference intents for training and evaluation, four capability benchmarks, and baseline and component-analysis setups.

B.1.1 DATASETS

  • DeepSeek-V3.2-Exp generates fixed reference intents summarizing entities, actions, purpose, and stance, which supervise defender understanding and evaluate intent preservation.Harmful-source references also include the annotated risk category.Source: §B.1.1 · p. 17
  • Capability retention is evaluated on HellaSwag, ARC Challenge, MMLU, and CMMLU.These benchmarks cover commonsense reasoning, science questions, multidisciplinary English knowledge, and Chinese knowledge and reasoning, respectively.Source: §B.1.1 · p. 17

B.1.2 BASELINES

  • SFT initializes subsequent adversarial training; offline adversarial training uses a fixed attack set, whereas PICL regenerates attacks against the current defender.The component analysis distinguishes perception-aware inputs (PA), PA with offline adversarial training, and text-only online adversarial training.Source: §B.1.2 · p. 17

B.1.3 IMPLEMENTATION DETAILS

Implementation details specify the attacker and judge configurations, self-play and evaluation budgets, and defender training procedure. Multi-perceptual inputs use phonetic and rendered-image views, with a fixed-size response group during updates.

B.1.3 IMPLEMENTATION DETAILS

  • Self-play uses six attack rounds and evaluation uses twelve, with both validity thresholds set to 0.6.GPT-5 mini generates attacks; DeepSeek-V3.2-Exp evaluates readability, intent preservation, and defender intent alignment.Source: §B.1.3 · p. 17
  • Defender training samples four responses per input and replaces the lowest-reward response with a reference response when all moderation decisions are incorrect.The reference is included in advantage estimation and policy updates, keeping the group size fixed at four.Source: §B.1.3 · p. 17
  • Training renders tone-free Pinyin and text on a fixed grid, resizing images to a maximum side length of 448 pixels.The text grid uses 28 × 28 pixel cells.Source: §B.1.3 · p. 17

B.2 EXAMPLES

The examples show PASP interpreting gambling promotion across disguises and illustrate how an adaptive attack can eventually pass SFT. The section also compares attacker rewrites and reports human validation of the LLM judge.

B.2 EXAMPLES

  • PASP identifies gambling promotion when a platform identifier and domain are split across fields and presented as a connectivity test.The example describes the underlying intent as gambling-platform promotion despite its technical framing.Source: §B.2 · p. 18
  • By the third round, SFT accepts a lottery-streaming entry progressively reframed as an internal archival record.The example presents three consecutive attack rounds against SFT.Source: §B.2 · p. 18
  • Four attackers rewrite the same WeChat contact request using administrative, field-based, offline-notification, or technical-configuration framing.The examples illustrate wording and organizational variation across attackers within a similar framing.Source: §B.2 · p. 20

B.3 HUMAN VALIDATION OF THE LLM JUDGE

  • 79.4% average precision: human judgments validated 73.6% of judge-accepted ChineseHarm-Bench rewrites and 85.2% of judge-accepted EvoHarmBench rewrites.The validation sampled 500 adversarial rewrites from each benchmark, with 250 judge-accepted and 250 judge-rejected examples; human annotators found more missed valid rewrites than false acceptances in these subsets.Source: Table 9 · p. 21, §B.3 · p. 21

B.4 IN-THE-WILD E-COMMERCE EVALUATION

The in-the-wild evaluation covers marketplace moderation and disguised-brand interpretation. Examples show PASP recognizing implicit sexual solicitation and identifying a brand from phonetic and symbol-obfuscated forms.

B.4 IN-THE-WILD E-COMMERCE EVALUATION

  • 11,465 pornographic and 11,465 benign examples are evaluated by jointly reading product titles and comments for sexual solicitation or pornographic content.The evaluation reports recall and binary F1, treating pornographic examples as the positive class.Source: §B.4 · p. 21
  • PASP flags a late-night, 2,500-yuan offer as sexual solicitation, while SFT treats it as a local-guide inquiry.The example contrasts SFT's “No” decision with PASP's pornography classification.Source: §B.4 · p. 22
  • Brand-variant evaluation uses 1,000 examples with questions about disguised names and their identity and variation patterns.Responses are compared with reference answers for brand identification, transformation patterns, and evasive intent; reported measures are mean score and the proportion scoring at least 4.Source: §B.4 · p. 22
  • In the Chantecaille example, PASP identifies the brand from Chinese phonetic variants and symbol-obfuscated English forms; “c#$n@ecaille” corresponds to Chantecaille and receives a judge score of 5/5.Source: §B.4 · pp. 23–24
Loading 2610.15449v2…