PaperlayerView exact revision on arXiv

Source-linked AI summary

StealthRL: Reinforcement Learning Paraphrase Attacks for Multi-Detector Evasion of AI-Text Detectors

Suraj Ranganath, Atharv Ramesh

arXiv:2602.08934v2cs.LGcs.AIcs.CR

TL;DR

AI-text detectors remain vulnerable to semantic-preserving adversarial paraphrasing, especially under strict low-false-positive operation. StealthRL trains a reinforcement-learning paraphrase policy against a detector ensemble with semantic-preservation rewards, achieving severe detection failure that transfers to held-out detectors. The paper frames its released protocol and evaluation pipeline as a testbed for measuring adversarial robustness.

  • Problem

    AI-text detectors face poorly understood robustness to adversarial paraphrasing, while deployed systems must operate at low false positive rates.

  • Method

    StealthRL trains a LoRA-adapted Qwen3-4B-Instruct paraphrase policy with GRPO using a composite detector-evasion and semantic-similarity reward.

  • Results

    0.024 mean TPR@1%FPR and near-zero detection on three of four detectors were achieved, with strong transfer to two held-out detectors.

  • Takeaways & Limitations

    The evaluation reveals significant robustness gaps in the tested AI-text detectors and provides a rigorous testbed for adversarial robustness measurement.

  • Takeaways & Limitations

    The evaluation excludes watermark-based detectors and does not investigate defensive strategies such as adversarial training or certified robustness.

Abstract

from arXiv · show

AI-text detectors face a critical robustness challenge: adversarial paraphrasing attacks that preserve semantics while evading detection. We introduce StealthRL, a reinforcement learning framework that stress-tests detector robustness under realistic adversarial conditions. StealthRL trains a paraphrase policy against a multi-detector ensemble using Group Relative Policy Optimization (GRPO) with LoRA adapters on Qwen3-4B, optimizing a composite reward that balances detector evasion with semantic preservation. We evaluate six attack settings (M0-M5) on the full filtered MAGE test pool (15,310 human / 14,656 AI) against four detectors: RoBERTa, Fast-DetectGPT, Binoculars, and MAGE. StealthRL achieves near-zero detection on three of the four detectors and a 0.024 mean TPR@1%FPR, reducing mean AUROC from 0.79 to 0.43 and attaining a 97.6% attack success rate. Critically, attacks transfer to two held-out detectors not seen during training, revealing shared architectural vulnerabilities rather than detector-specific brittleness. We additionally conduct LLM-based quality evaluation via Likert scoring on 500 matched samples per method, analyze detector score distributions to explain why evasion succeeds, and provide per-detector AUROC with bootstrap confidence intervals. Our results expose significant robustness gaps in current AI-text detection and establish StealthRL as a principled adversarial evaluation protocol. Code and evaluation pipeline are publicly available at https://github.com/suraj-ranganath/StealthRL.

1 Introduction

AI-text detectors are increasingly deployed despite limited evidence about their robustness to adaptive paraphrasing. StealthRL addresses this gap by evaluating semantic-preserving evasion at the strict 1% FPR operating point across multiple detector families.

  • Motivation: Adaptive paraphrasing exposes a critical gap between clean-distribution detector evaluation and real-world robustness.Adversaries can refine paraphrases, query detector APIs, and target detector decision boundaries.
  • Motivation: At 1% FPR, detectors prioritize avoiding false accusations of human writers, making adversarial recall loss especially consequential.Strict thresholds sacrifice recall to maintain precision.
  • Approach: StealthRL trains a paraphrase policy with GRPO and LoRA against a multi-detector ensemble while preserving semantic fidelity.The policy uses Qwen3-4B-Instruct and minimizes detection confidence under adaptive conditions.
  • Results: 0.024 mean TPR@1%FPR demonstrates catastrophic robustness failure across four detectors, including strong transfer to held-out detectors.The protocol evaluates evasion, transfer, and fidelity and includes complete reproducibility code.
  • Evaluation: The evaluation combines detector score distributions, LLM-based quality assessment, and per-detector AUROC with bootstrap confidence intervals.These analyses characterize both evasion behavior and paraphrase quality.

2 Related Work

Prior work spans several detector architectures and attack strategies, but robustness under adaptive, cross-detector paraphrasing remains insufficiently characterized. StealthRL extends reinforcement-learning evasion from single-detector training to ensemble training with held-out transfer and strict operating points.

  • Detector families: AI-text detectors include fine-tuned classifiers, zero-shot statistical methods, and paired-LM approaches with distinct detection signals.These families use labeled discrimination, probability distributions, or comparisons between language models.
  • Motivation: Theoretical arguments suggest reliable detection may become fundamentally impossible as language models improve, motivating empirical robustness evaluation.The cited work provides theoretical arguments rather than an empirical result from StealthRL.
  • Evasion attacks: Existing evasion attacks range from paraphrasing and detector-guided candidate selection to homoglyph substitution, with readability trade-offs reported for character-level attacks.Prior paraphrasing attacks include DIPPER and universal adversarial paraphrasing.
  • Positioning: AuthorMist trains against one detector, whereas StealthRL uses multi-detector ensemble training, held-out evaluation, strict low-FPR testing, and quality analysis.This positions StealthRL as an extension of single-detector reinforcement-learning evasion.
  • Scope: Watermark detection is outside the evaluated post-hoc setting because it requires control over the generation process.Watermarks embed statistical signals during generation and may provide stronger guarantees.
  • Defensive context: Adversarial training may improve resilience but faces scalability challenges because paraphrasing strategies are vast and attack-specific defenses may not generalize.The cited discussion motivates diverse robustness benchmarks.
  • Optimization background: GRPO reduces memory requirements by replacing a separate value network with group-level relative rewards.This makes GRPO an efficient alternative to PPO for reinforcement learning from human feedback.
  • Optimization background: LoRA enables parameter-efficient RL fine-tuning, and StealthRL combines it with GRPO on Qwen3-4B-Instruct for evasion-policy learning.Only low-rank weight matrices are adapted.

3 Method

StealthRL treats paraphrasing as a black-box optimization problem: produce fluent, semantically equivalent text with lower detector confidence. It trains a LoRA-adapted Qwen3-4B-Instruct policy using GRPO, a composite evasion-and-similarity reward, and held-out transfer evaluation.

  • Threat model: The attacker seeks a fluent paraphrase y of AI text x with detector confidence below threshold τ while maintaining semantic similarity above δ.Detector access is limited to scalar confidence scores without gradients or internal parameters.
  • Threat model: Transfer is measured on Binoculars and MAGE, two detectors excluded from training to test generalization across architectures.The training ensemble contains RoBERTa and Fast-DetectGPT.
  • Reward design: The composite reward combines detector evasion and E5 semantic similarity, with α = 1.0 and β = 0.1 controlling the trade-off.The semantic term discourages empty or unrelated outputs.
  • Reward design: The detector reward is one minus a weighted ensemble score, assigning RoBERTa weight 0.6 and Fast-DetectGPT weight 0.4.The weighting prioritizes the fine-tuned classifier during training.
  • Optimization: GRPO uses an implicit KL penalty with λ_KL = 0.05 against a frozen reference policy to preserve fluency and prevent catastrophic forgetting.The reference policy remains frozen during optimization.
  • Training procedure: Training uses Qwen3-4B-Instruct with LoRA rank 32, three epochs, 10,000 MAGE AI samples, group size 8, and learning rate 2.8 × 10^-4.The training ensemble is RoBERTa at 60% and Fast-DetectGPT at 40%.
  • Inference: At inference, one paraphrase is generated per input with no candidate selection, reranking, iterative refinement, or detector queries.Evaluation uses temperature 1.0, top-p 0.9, and a 512-token maximum.
  • Training procedure: The training procedure samples groups of candidates and computes ensemble-evasion rewards before group-relative advantage normalization.Algorithm 1 identifies group size G = 8 candidates.

4 Experimental Setup

Experiments use the filtered MAGE benchmark, six attack settings, four detector paradigms, and deployment-relevant operating points. Evaluation also measures semantic preservation and judged paraphrase quality on matched samples.

  • Dataset: The filtered MAGE test pool contains 15,310 human-written and 14,656 AI-generated samples.Training uses 10,000 AI-generated samples from the MAGE training split without human samples.
  • Detectors: The four detectors span supervised classification, zero-shot curvature scoring, paired-LM comparison, and a Longformer-based benchmark detector.Binoculars and MAGE are held out for transfer evaluation.
  • Attack settings: Six settings compare no attack, simple paraphrasing, StealthRL, detector-guided paraphrasing, single-detector AuthorMist, and homoglyph substitution.StealthRL is the RL-trained multi-detector method, using GRPO and LoRA for three epochs.
  • Metrics: TPR@1%FPR is the primary metric, using thresholds fixed at each detector’s empirical 99th percentile on human samples.TPR@5%FPR is supplementary, while ASR equals 1 − TPR@1%FPR.
  • Metrics: AUROC measures threshold-independent discriminability, while E5 Similarity measures semantic preservation through embedding cosine similarity.Confidence intervals use 500 bootstrap iterations with seed 42.
  • Quality evaluation: An LLM judge scores linguistic quality and semantic similarity on separate 1–5 Likert scales.The evaluation uses 500 matched AI samples per method with identical sample IDs and blinded method labels.

5 Results

StealthRL substantially weakens detection across detectors at strict operating points, with strong transfer to held-out architectures. The attack also exposes a tradeoff between evasion and paraphrase quality, while score distributions indicate why detection fails.

  • Main Detection Evasion Results: 0.024 mean TPR@1%FPR and 97.6% attack success rate show StealthRL’s strong evasion at the strict deployment operating point.Mean AUROC falls from 0.79 without attack to 0.43.
  • Cross-Architecture Transfer: 0.002 TPR@1%FPR on Binoculars and 0.089 on MAGE demonstrate transfer to detectors absent from training.StealthRL also achieves 0.002 on both in-ensemble detectors.
  • Cross-Architecture Transfer: Cross-architecture transfer suggests shared vulnerabilities in surface-level statistical cues rather than detector-specific weaknesses.The cited analysis identifies token distributions, perplexity patterns, and embedding geometry as disrupted by paraphrasing.
  • Detector Score Analysis: AI-sample scores for StealthRL and Homoglyph fall below the 1% FPR threshold, making them statistically indistinguishable from human-written text to detectors.Other methods retain substantial scores above the threshold, explaining their higher detection rates.
  • Per-Detector AUROC: 0.055 and 0.089 AUROC on Binoculars and Fast-DetectGPT contrast with 0.691 on RoBERTa and 0.891 on MAGE.The 0.5 dashed line marks random chance; AUROC measures global ranking, whereas TPR@1%FPR measures performance at one strict operating point.
  • Evasion–Quality Tradeoff: M2 reaches 0.976 mean ASR with 0.901 E5 similarity, while M1 and M3 achieve 0.974 and 0.973 similarity but weaker evasion.LLM judging rates M2 at 2.51 quality versus 3.78 and 3.77 for M1 and M3, indicating a fidelity cost.

6 Limitations and Broader Impact

The evaluation is limited by detector coverage, dataset scope, and a measurable quality gap, while the released pipeline supports reproducible robustness assessment and defensive research.

  • Detector coverage: The study omits watermark-based detectors, which may be more robust to paraphrasing attacks.Evaluating StealthRL against watermarked text is identified as future work.
  • Dataset diversity: The evaluation uses a single English benchmark, so broader datasets, domains, and languages are needed to establish generalizability.The MAGE benchmark may not capture all deployment scenarios.
  • Quality gap: 0.901 E5 similarity and 2.51 Likert quality leave StealthRL below simpler baselines M1 and M3 at 0.974/3.78 and 0.973/3.77.Improving semantic preservation while maintaining strong evasion is identified as an important direction.
  • Defense evaluation: The study does not evaluate defenses such as adversarial training, certified robustness, or ensemble diversification.Its stated focus is exposing vulnerabilities to motivate defensive research.
  • Ethical considerations: StealthRL is positioned as a stress-testing tool rather than a production evasion system because adversarial paraphrasing is dual-use.The authors connect near-zero TPR@1%FPR with vulnerabilities relevant to high-stakes deployment.
  • Broader impact: Released code and the evaluation pipeline enable reproducible robustness assessment and support a shift toward adversarially robust detector development.The authors frame transparent attack capabilities and vulnerability disclosure as supporting trustworthy AI-text detection.

7 Conclusion and Future Work

StealthRL exposes severe robustness gaps in AI-text detectors under adaptive RL-based paraphrasing, while providing a testbed for evaluating adversarially robust detection. The paper also outlines future directions spanning detector training, semantic modeling, robustness guarantees, optimization, and broader evaluation.

  • StealthRL achieves near-zero detection on three of four detectors and 0.024 mean TPR@1%FPR, with strong transfer to two held-out detectors.The evaluation combines detection metrics, quality assessment, and score-distribution analysis to characterize the evasion–quality tradeoff.
  • Future work: Future work includes adversarial training, semantic-aware detectors, provable robustness, multi-objective optimization, and broader evaluation across datasets, languages, and detector families.The proposed scope includes watermark-based methods among additional detector families.
  • The released evaluation framework and code provide a rigorous testbed for measuring progress on adversarially robust AI-text detection.

A Full Main Results on the MAGE Test Pool

The main-results tables report detector and aggregate performance on the full filtered MAGE test pool at the 1% FPR operating point, alongside supplementary results at 5% FPR.

  • 1% FPR main results: Table 2 reports Binoculars, Fast-DetectGPT, and MAGE results at 1% FPR, where lower TPR@1%FPR and AUROC favor the attacker and higher ASR is better.Bold values indicate the best value per metric.
  • 1% FPR main results: Table 3 reports RoBERTa results and mean TPR@1%FPR and mean ASR aggregates across the four-detector panel.Bold values indicate the best value per metric.
  • 5% FPR supplementary results: Table 4 provides supplementary TPR@5%FPR results with thresholds recalibrated on 15,310 human evaluation samples.Lower TPR@5%FPR favors the attacker, and bold values indicate the lowest value per column.

B Qualitative Examples

The qualitative appendix presents representative paraphrases from the filtered MAGE test split across health, automotive, science, technology, economics, and energy domains, with configuration details provided separately.

  • Representative paraphrases: Tables 5–7 show representative paraphrases across different domains from the filtered MAGE test split.The listed examples cover health, automotive, science, technology, economics, and energy.
  • Reproducibility: Table 8 provides the complete hyperparameters and configuration for reproducibility.

D Per-Detector Results with Confidence Intervals

The per-detector appendix supplies complete results with bootstrap confidence intervals for four detectors and documents the LLM-based Likert procedure used to assess paraphrase quality and meaning preservation.

  • Per-detector results: Table 9 reports complete per-detector results with 95% bootstrap confidence intervals for RoBERTa, Fast-DetectGPT, Binoculars, and MAGE.The intervals use 500 bootstrap iterations, and bold values indicate the best value per metric.
  • Per-detector results: StealthRL (M2) achieves near-zero TPR@1%FPR on three of the four detectors.
  • LLM quality evaluation: The gpt-5-nano judge rates each original–paraphrase pair on 1–5 quality and similarity Likert dimensions.Quality measures fluency, grammaticality, and naturalness, while similarity measures preservation of the original meaning.
  • LLM quality evaluation: The judge is instructed to return quality and similarity scores with textual justifications in JSON format.

F Dataset Statistics

The Likert evaluation uses 500 AI samples per attack method, with matched sample IDs across methods M1–M5.

  • 500 AI samples per attack method (M1–M5) are used for Likert evaluation with matched sample IDs.
Loading 2602.08934v2…