Attacks on Machine-Text Detectors Retain Stylistic Fingerprints

• LLMs have benign uses but can be abused (Hazell, 2023), while increasingly sophisticated evasion attacks challenge zero-shot and supervised detectors (Koike et al., 2024; Krishna et al., 2023).Proposed defenses include zero-shot detectors, supervised detectors, and watermarking approaches.Source: §1 · p. 1
• Generic evasion attacks degrade standard detectors but leave stylistic fingerprints: StyleDetect remains robust across attack categories, even when style is explicitly targeted by preference optimization.The authors argue that popular attacks target features distinct from those indicating authorship style.Source: §1 · p. 1, Table 1 · p. 2
• The authors introduce a style-aware paraphraser that uses target-author excerpts, supervised fine-tuning, and preference optimization to mimic human style while preserving meaning and evading style-based detection from a single sample.Applied iteratively to machine-generated text, it produces outputs indistinguishable from human-written text when only one sample is available for detection.Source: §1 · pp. 1–2
• Detection performance recovers as the number of available documents grows (N > 1), making human and machine text distributions distinguishable again.The authors argue this points toward moving beyond single-document analysis.Source: §1 · p. 2

2. Preliminaries: The Stylistic Feature Space

• A style representation maps a document to a fixed-dimensional vector such that distance reflects authorship similarity rather than meaning or topic.Documents by the same author should have high cosine similarity even when their topics differ.Source: §2 · p. 2
• These representations are typically trained contrastively on large-scale authorship verification tasks, often using low-resource human authors, and have been found to distinguish different LLMs (Rivera Soto et al., 2024).Training optimizes the embedding space to cluster documents by author regardless of meaning.Source: §2 · p. 2

Attack strategies

• Attacks included detector-guided DPO optimization and post-hoc DIPPER or GPT-4o-mini paraphrasing, OUTFOX adversarial prompting, style-aware prompting, and TinyStyler style transfer.DPO selects the response a target detector rates as more human-like; OUTFOX uses detector prediction labels as in-context examples, while style-aware prompting targets a specific author.Source: §3 · pp. 2–3

Evaluation setup

• Tests used held-out Reddit comments, comparing DPO variants with their base checkpoints across three open-weight model families and applying post-hoc transformations to outputs from Mistral, Llama, and GPT-4o-mini.The optimization results average detection performance across Mistral-7B-Instruct, Qwen-2.5-7B-Instruct, and Mistral-Nemo-Instruct; post-hoc tests use Mistral-7B-Instruct, Llama-3-8B-Instruct, and GPT-4o-mini.Source: §3 · p. 3

Results

• FastDetectGPT’s detection AUROC fell from 65 to 35 under optimization, while StyleDetect remained robust across attack categories and SemDetect did not match it.The comparison uses StyleDetect and SemDetect support sets containing 100 examples from the unoptimized base LLM.Source: §3 · p. 3
• TinyStyler lowered StyleDetect AUROC to 0.87, but Binoculars’ performance rose from 79 to 92, trading stylistic disruption for easier zero-shot detection.The passage describes the rise in zero-shot performance as a result of statistical artifacts introduced by the transformation.Source: §3 · p. 3

Why style features are robust

• Generic evasion is ill-posed because human authors have idiosyncratic styles, and StyleDetect captures fine-grained author features rather than broad categories such as formal or casual.The representations are trained to distinguish low-resource human authors with 100 posts or fewer; without a specific human author target, generic optimization fails to converge to a human stylistic fingerprint.Source: §3 · p. 3

Mimicking human writing styles

• The model is instruction-tuned to map machine paraphrases back to originals using in-context original–paraphrase pairs from the same author (Wei et al., 2022).Generating multiple paraphrases per human-written text increases the available exemplars without collecting more human-written samples.Source: §4 · p. 3

Avoiding machine-text detectors

• Direct preference optimization selects the most human-like of 20 candidate outputs using a detector trained to identify the system’s generations.The authors argue that this detector captures more of the system’s identifiable features than optimizing against a zero-shot detector, as in prior work.Source: §4 · pp. 3–4

Inference

• At inference, the paraphraser iteratively generates 10 candidates and keeps the top-P that best preserve meaning according to SBERT, then selects the best-preserving candidate in the final iteration.This procedure is intended to bridge the mismatch between training on paraphrases of human text and processing fully machine-generated text.Source: §4 · p. 4

Connection to other paraphrasers

• Unlike DIPPER (Krishna et al., 2023), the system supports low-resource author-style conditioning and is applied to rewriting machine-generated text.The authors describe it as the first such paraphraser to their knowledge and report state-of-the-art performance in undetectability and traditional style transfer (§6.1, Appendix D).Source: §4 · p. 4

5.1. Baselines

• Baselines include prompt-based paraphrasing, DIPPER, OUTFOX, TinyStyler, style-aware prompting, and detector-guided DPO.The proposed system tunes Mistral-7B with LoRA to evade detectors without author representations, unlike TinyStyler’s 800M-parameter Reddit-trained paraphraser using pretrained author representations. Detector-guided DPO uses FastDetectGPT humanness scores to choose between two Mistral-7B generations for each human exemplar (Bao et al., 2024).Source: §5.1 · pp. 4–5

5.2. Datasets

• Training, preference tuning, and evaluation use Reddit, Amazon reviews, and Blogs, with evaluation texts matched to human-text lengths.Training draws stylistically stratified comments from the Reddit Million Users Dataset (Khan et al., 2021), with five Mistral-7B-Instruct paraphrases per comment. Preference data comprise 10,000 distinct-author Reddit and Amazon samples and 6,000 Blog samples; the proposed method uses Reddit only, while Detector-Guided DPO uses all three domains. Evaluation generates text using Mistral-7B-Instruct, gpt-4o-mini, or Llama-3-8B-Instruct, applies the baselines (except DPO, which generates directly), and supplies 16 texts from a randomly selected target author when exemplars are needed.Source: §5.2 · p. 5

5.3. Metrics and Detectors

• Evaluation measures detection with AUROC and AUROC(1), meaning preservation with SBERT, style transfer with CISR, and performance across nine detectors.AUROC(1) is partial area for false-positive rates at or below 1%; SBERT and CISR use cosine similarity between embeddings. The detectors comprise trained classifiers (RADAR, ReMoDetect), zero-shot methods (Rank, LogRank, FastDetectGPT, Binoculars), and few-shot stylistic methods (StyleDetect, StyleDetect-CISR, StyleDetect-SD); StyleDetect uses K = 100 machine-text exemplars.Source: §5.3 · p. 5

6.1. Machine-Text Detection

• Our paraphraser is least detectable across Reddit, Amazon, and Blogs, though it becomes detectable after five samples in Blogs.The authors attribute the Blogs result to the mismatch between informal Reddit training text and more formal blog text.Source: Figure 1 · p. 6, §6.1 · p. 7
• Similarity ≥0.85 and +43 character edits versus regular paraphrasing characterize our method’s meaning retention and transformation.Table 3 reports character edit distance 199.09 for our method versus 227.39 for DIPPER, and semantic similarity 0.85 versus TinyStyler’s 0.78.Source: Table 3 · p. 6, §6.1 · p. 7

6.2. Visualizing the Space of Writing Styles

• Across LUAR, CISR, and StyleDistance, our paraphraser qualitatively best closes the gap between human-written and machine-generated styles.The comparison uses Reddit generations from Detector-Guided DPO, DIPPER, OUTFOX, and the style-aware paraphraser.Source: Figure 3 · p. 7, §6.2 · p. 8

6.1. Machine-Text Detection

• Detector-Guided DPO remains identifiable by StyleDetect, whereas our approach offers a better trade-off across zero-shot and stylistic detectors.Figure 2 reports that Detector-Guided DPO becomes detectable with five samples, while our approach remains robust up to 50.Source: Figure 2 · p. 6, §6.1 · p. 7
• Three generation-and-SBERT-reranking iterations with 10 candidates each make the attack slower per sample, raising the cost of large-scale deployment.Per-sample inference times were measured on one NVIDIA A100 (80GB), batch size 1, averaged across 100 samples after a five-sample warmup.Source: §6.1 · p. 7

6.3. Ablations

• Increasing M target exemplars or P paraphrases improves target-style similarity, while DPO helps maintain undetectability.The evaluations examine similarity as P and M vary and compare detectability with and without DPO as sample size grows.Source: Figure 4 · p. 8, Figure 5 · p. 8, §6.3 · p. 8
• SBERT cosine similarity remains above 0.87 across all three domains after three iterations, indicating little semantic drift.The measure compares the original machine text with the output at each stage of the iterative procedure.Source: Table 5 · p. 8, §6.3 · p. 8

Machine-text detection

• Detection research includes zero-shot token-distribution methods, supervised detectors, style-feature detectors, and watermarking applied during generation.Zero-shot methods use predicted token-wise conditional distributions; supervised detectors can be sensitive to test-time distribution shifts, while watermarking assumes access to generation.Source: §7 · p. 8

Style-aware paraphrasing

• Style-aware paraphrasing covers coarse-grained styles and low-resource authorship, using inverse models, prompting, lightweight models, diffusion, and energy-based sampling.Examples include formality, informality, Shakespearean English, and poetry (Krishna et al., 2020; Liu and May, 2025); low-resource methods include prompting (Patel et al., 2024), lightweight models (Liu et al., 2024), diffusion (Horvitz et al., 2024a), and energy-based sampling (Khan et al., 2024).Source: §7 · p. 8
• The proposed approach targets low-resource authors while jointly optimizing for author-specific style and undetectability, without style-indicative embeddings.It also optimizes against features exploited by most detectors.Source: §7 · pp. 8–9

Defeating detectors

• Prior detector-evasion approaches use paraphrasing, prompt optimization, spacing, homoglyphs, or DPO post-training, but manipulate surface features without closing the human–machine distribution gap.The cited approaches include paraphrasing (Krishna et al., 2023), prompt optimization (Lu et al.), spacing (Cai and Cui, 2023), homoglyphs (Creo and Pudasaini, 2025), and DPO post-training.Source: §7 · p. 9

Conclusion

• Style-aware paraphrasing evades all tested detectors, including style-based ones, but human and paraphrased-text distributions become separable as enough documents are analyzed.The authors propose making authenticity decisions about a source from multiple writing samples rather than document by document.Source: §8 · p. 9
• Multi-document detection is practical where defenders already retain author histories, and is argued necessary to reduce costly false positives in settings such as academia.Examples include students’ prior coursework, reviewers’ submissions, and users’ post histories on platforms.Source: §8 · p. 9
• The attack requires human-style exemplars and LLM paraphrases, with semantic-drift risk and iterative inference costs that limit low-compute use.The authors note these constraints may practically limit the attack’s applicability.Source: §8 · p. 9

Impact Statement

• Convincing machine-generated text poses a significant risk of abuse.Source: Impact Statement · p. 9
• Studying attacks clarifies limitations of current state-of-the-art defenses and may inform future improvements in machine-text detection.Source: Impact Statement · p. 9
• The findings suggest that certain feature spaces may be inherently more robust for machine-text detection.Source: Impact Statement · p. 9

A. Breakdown of Performance by Method, Dataset, and Detector

• Ours is the only attack that simultaneously suppresses all nine detectors at small N across three domains, although detectability recovers as N grows.Prior attacks degrade some detectors but leave at least one strong detector intact, typically a stylistic one.Source: A. Breakdown of Performance by Method,… · p. 13
• At baseline, every detector reaches AUROC(1) ≈1.0 with only a handful of documents per author.Figure 6 shows per-detector AUROC(1) for the baseline and prior attacks across Reddit, Amazon, and Blogs as N varies.Source: A. Breakdown of Performance by Method,… · p. 13, Figure 6 · p. 13

B. Breakdown of Edit Distance and Semantic Similarity by Dataset

• Within Reddit, Amazon, and Blogs, the method matches DIPPER on character edit distance while preserving semantics substantially better than TinyStyler.Table 6 breaks down mean character edit distance and SBERT semantic similarity by domain; Detector-Guided DPO has no reference for comparison because it generates text from scratch.Source: B. Breakdown of Edit Distance and… · p. 13, Table 6 · p. 14

C. AUROC Performance

• The method is the least detectable approach against the strongest detector across Reddit, Amazon, and Blogs, including Amazon and Blogs, which were not seen during training.Figure 8 reports full AUROC, for which lower values indicate better evasion.Source: Figure 8 · p. 15, C. AUROC Performance · p. 15

D. Style-Transfer Performance

• Against TinyStyler, the approach raises stylistic similarity from 0.71 to 0.83 (+0.12) and semantic similarity from 0.74 to 0.83 (+0.09).Stylistic similarity is measured with CISR embeddings; the authors report gains on both evaluation axes.Source: D. Style-Transfer Performance · p. 15

E. Training Hyperparameters and Compute Resources; F. Dataset details

• Training uses Mistral-7B on Reddit comments from 63,184 stratified authors, while detection evaluation covers held-out Reddit, Amazon, and Blogs texts generated by three LLMs.The style-aware paraphraser is trained for one epoch with LoRA and preference tuning; training uses 8 80Gb A100s for one day, followed by three hours of post-training, with at most one A100 needed for inference. Dataset sizes are 12,000 Reddit comments, 12,000 Amazon reviews, and 7,000 blog posts; texts are 32–128 tokens long.Source: E. Training Hyperparameters and… · p. 15, F. Dataset details · p. 16

H. Prompts

• The appendices describe plain Mistral-7B paraphrasing and length-controlled prompts for generating Reddit comments, Amazon reviews, and blog snippets.The generation prompts set a target word count based on the original human text; the paraphrasing prompt requests only a paraphrase, while style transfer instructs the model to preserve meaning and match a target author's style.Source: §H.1 · p. 17, §H.3 · p. 17, §H.4 · p. 20

I. Generalization to Newer Generators

• Across Qwen3-8B, Qwen3-14B, and Mistral-Nemo, the attack lowers strongest-detector AUROC at every sample size and domain; at N=1 it falls below chance, but distributions separate as N grows, especially in Blogs.The study reports that this pattern generalizes to the three newer generators across Reddit, Amazon, and Blogs.Source: Table 12 · p. 20, Table 13 · p. 20, §I · pp. 20–21

J. LLMs as Few-Shot Machine-Text Detectors

• Qwen3-14B prompting yields chance-level detection in zero- and few-shot settings, while the stronger LLM-based OUTFOX detector is also fooled by the attack.The test used 16 in-context examples on a balanced Reddit evaluation sample; chance-level results hold for unmodified text and the style-aware paraphrases.Source: Table 15 · p. 21, J. LLMs as Few-Shot Machine-Text… · p. 21

K. Cross-Model Robustness of StyleDetect

• The cross-model evaluation uses a 100-sample StyleDetect support set and scores Reddit text from target generators, with rows indexing support-set generators and columns indexing targets.All reported scores are AUROC(1).Source: K. Cross-Model Robustness of StyleDetect · p. 21
• StyleDetect remains effective with support samples from a generator different from the target, and cross-model AUROC(1) scores are close off-diagonal and on-diagonal.The results suggest that the stylistic fingerprints are largely shared across modern instruction-tuned LLMs rather than model-specific artifacts.Source: Table 16 · p. 21, K. Cross-Model Robustness of StyleDetect · p. 21

L. Essay-Level Detection: Full Document vs. Chunk & Merge

• The essay-level study uses 250 essays from the GEDE dataset (Gehring and Paaßen, 2025) to test how the attack adapts to long-form text.Earlier evaluation and attack training used 32–128 token documents as units.Source: L. Essay-Level Detection: Full… · p. 21
• In the Full Essay attack, the attacker paraphrases the entire essay in one pass for three iterations.Source: L. Essay-Level Detection: Full… · p. 22
• In Chunk & Merge, the attacker splits an essay into paragraphs, paraphrases each independently for three iterations, and re-concatenates them in the original structure.Source: L. Essay-Level Detection: Full… · p. 22
• Detection treats each essay as a separate document and reports the best detector AUROC(1) as N ranges from one to five paragraphs.Source: L. Essay-Level Detection: Full… · p. 22
• Full-essay paraphrasing is detectable after two paragraphs, while Chunk & Merge makes detection substantially harder.The authors attribute the segmentation benefit to keeping paraphrased units small enough to stay within the training dataset’s length distribution; the multi-document conclusion also carries over to essays.Source: Table 17 · p. 22, L. Essay-Level Detection: Full… · p. 22