PaperlayerView exact revision on arXiv

Source-linked AI summary

Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense

Kalpesh Krishna, Yixiao Song, Marzena Karpinska, John Wieting, Mohit Iyyer

arXiv:2303.13408v2cs.CLcs.CRcs.LG

TL;DR

AI-text detectors may fail when generated text is paraphrased, raising questions about their robustness. The paper introduces DIPPER to stress-test these detectors and retrieval over stored API generations as a defense, finding that retrieval detects 80%–97% of paraphrases at a 1% false-positive rate. The authors also discuss scalability, privacy, and access limitations.

  • Problem

    The robustness of AI-generated-text detectors to paraphrases remains unclear, despite concerns about malicious LLM-generated content.

  • Method

    The paper trains DIPPER, an 11B-parameter contextual paraphraser with controllable lexical diversity and content reordering, and evaluates retrieval over semantically similar stored API generations.

  • Results

    80% to 97%: retrieval detects paraphrased generations across settings while classifying 1% of human-written sequences as AI-generated; DIPPER also sharply reduces detector accuracy.

  • Takeaways & Limitations

    Paraphrasing can evade several existing detectors, while provider-maintained retrieval offers a defense that remains effective against semantic-preserving paraphrases.

  • Takeaways & Limitations

    Retrieval-based detection carries privacy risks and cannot be applied to open-source LLMs without a provider-maintained generation database.

Abstract

from arXiv · show

The rise in malicious usage of large language models, such as fake content creation and academic plagiarism, has motivated the development of approaches that identify AI-generated text, including those based on watermarking or outlier detection. However, the robustness of these detection algorithms to paraphrases of AI-generated text remains unclear. To stress test these detectors, we build a 11B parameter paraphrase generation model (DIPPER) that can paraphrase paragraphs, condition on surrounding context, and control lexical diversity and content reordering. Using DIPPER to paraphrase text generated by three large language models (including GPT3.5-davinci-003) successfully evades several detectors, including watermarking, GPTZero, DetectGPT, and OpenAI's text classifier. For example, DIPPER drops detection accuracy of DetectGPT from 70.3% to 4.6% (at a constant false positive rate of 1%), without appreciably modifying the input semantics. To increase the robustness of AI-generated text detection to paraphrase attacks, we introduce a simple defense that relies on retrieving semantically-similar generations and must be maintained by a language model API provider. Given a candidate text, our algorithm searches a database of sequences previously generated by the API, looking for sequences that match the candidate text within a certain threshold. We empirically verify our defense using a database of 15M generations from a fine-tuned T5-XXL model and find that it can detect 80% to 97% of paraphrased generations across different settings while only classifying 1% of human-written sequences as AI-generated. We open-source our models, code and data.

1 Introduction

The paper shows that paraphrasing can evade existing AI-text detectors, then proposes retrieval over prior generations as a defense. DIPPER preserves semantics while sharply reducing detection, whereas retrieval detects many paraphrases at low false-positive rates.

  • Existing detectors’ robustness to paraphrases remains unclear despite concerns about malicious LLM-generated content.
  • DIPPER is an 11B-parameter paraphraser that rewrites paragraphs in context and controls lexical diversity and content reordering.
  • 70.3% to 4.6%: DetectGPT detection of GPT2-XL generations drops after DIPPER paraphrasing at a constant 1% false-positive rate.The paper reports minimal semantic modification and validates paraphrase quality with automatic and human evaluations.
  • 97.3% of PG19 and 80.4% of Wikipedia paraphrases are detected by retrieval at a 1.0% false-positive rate in a database exceeding 15M generations.The defense compares candidate-text semantics with stored API generations using information-retrieval or semantic-vector methods.
  • The paper contributes comprehensive attack experiments, human paraphrase evaluations, a stronger paraphraser, and a retrieval-based defense, with models, code, and data released.

2 Background on detectors of AI-generated text

The background divides AI-generated-text detection into watermarking, statistical outlier detection, and classifier methods. The paper positions its work as a broader paraphrase-attack study that also introduces retrieval-based defense.

  • Watermarking modifies generated text imperceptibly so detectors can identify it after generation.
  • Statistical outlier detectors identify AI text from artifacts such as entropy and perplexity rather than modifying generation.
  • Classifier methods train models to distinguish human-written from AI-generated text across domains, tasks, or decoding strategies.
  • Compared with concurrent work, this paper evaluates more tasks, detectors, and larger language models while adding discourse-level DIPPER and retrieval defense.

3 Building a controllable discourse paraphraser

DIPPER is trained as a controllable discourse paraphraser using aligned paragraph-level translations. Its training pipeline teaches contextual rewriting, sentence reordering, and adjustable lexical and order diversity.

  • The attack requires a context-aware, controllable paraphraser because paraphrasing alters detector-relevant statistics and reduces watermarked tokens.
  • DIPPER training uses aligned paragraph-level paraphrases from multiple English translations of non-English novels.
  • Sentence alignments are computed with semantic similarity and dynamic programming before aligned subsets are selected for training.
  • Selected aligned sentences are randomly reordered, and lexical and order diversity control codes are computed from token overlap and Kendall-Tau correlation.
  • At inference, marked sentence spans receive desired lexical and order diversity values, allowing controlled paraphrasing within surrounding context.
  • The final dataset contains 6.3M paraphrase pairs, and DIPPER fine-tunes an 11B-parameter T5-XXL sequence-to-sequence Transformer.

4 Experiments attacking detection algorithms with DIPPER

Experiments show that DIPPER paraphrasing substantially reduces detection across language models, detectors, tasks, and false-positive thresholds while largely preserving semantics. Retrieval-based detection is substantially more effective than baseline detectors, although RankGen performs poorly and repeated paraphrasing requires detector access.

  • Overall results: DIPPER evades detectors across three language models, including GPT-3.5, while preserving paraphrase semantics.Across tested paraphrases, 88%-99% achieve a P-SP score above the median score of human-written paraphrases.
  • Detector evasion: Detection rates decrease as DIPPER’s lexical and ordering diversity increase, with nearly perfect semantic similarity.Table 1 and Figure 3 evaluate performance at 1% FPR; GPT3.5 DetectGPT uses a separate 20% FPR calculation because its 1% FPR score is 0%.
  • Detector comparison: RankGen is an exception: it scores paraphrases as AI-generated more often than non-paraphrased text and has low overall performance.The authors attribute this pattern to paraphrases being poorer prompt continuations, while RankGen was not trained for this task.
  • Detector comparison: Non-watermarking detectors generally perform poorly on larger-model outputs, while watermarking is the most resilient to paraphrasing.GPTZero and RankGen detect fewer than 15% of non-paraphrased AI-generated text, and DetectGPT drops from 74.9% on GPT2-XL to 29.8% on OPT-13B.
  • Detector evasion: Paraphrasing reduces detection across false-positive thresholds, including the 0%-1% FPR range evaluated for GPT2-XL.Figure 4 compares detectors before and after paraphrasing using ROC curves.
  • Alternative attacks: A single DIPPER paraphrase is used in the reported attacks; repeated sampling could strengthen attacks but assumes access to a detector.The repeated-sampling attack is not performed because detector access may be a strong assumption.

5 Defense against paraphrase attacks using retrieval

The paper proposes retrieval over previously generated sequences as a defense against paraphrase attacks, with an API-side database and similarity-based querying. Across controlled and larger-corpus experiments, retrieval remains effective, especially with BM25 and sufficiently long queries, but requires provider infrastructure and has privacy and open-source deployment limitations.

  • 5 Defense against paraphrase attacks using retrieval: The defense stores every LLM output in a private API-side database and searches it for semantically similar matches to candidate text.The database is dynamically updated, inaccessible to clients except through the API, and can use semantic encoders or retrievers such as BM25.
  • 5.1 Formulating the retrieval defense: The detector flags a candidate when its maximum similarity to a stored generation exceeds an API-provider-selected threshold T.Unperturbed machine-generated text is expected to score 1.0, while paraphrasing can lower the score; lowering T improves sensitivity but raises false positives.
  • 5.2 Controlled comparisons of retrieval with other AI-generated text detectors: The controlled evaluation compares retrieval with baseline detectors using original AI-generated text, DIPPER paraphrases, and human-written text across multiple retrieval corpora.The corpora contain 3K task-specific sequences, 9K sequences from three LMs, or 46K sequences including ShareGPT-Vicuna responses; queries contain at least 50 tokens.
  • 5.2 Controlled comparisons of retrieval with other AI-generated text detectors: 97.8% detection on GPT2-XL's highest-diversity paraphrases at 1% FPR, compared with 55.8% for the strongest competing detector.BM25 detects 97.3% on OPT-13B and 96.2% on GPT-3.5, while augmented databases retain detection rates from 94.4% to 97.5%.
  • 5.3 Is retrieval an effective detector with a large retrieval corpus?: 80.4% detection on Wikipedia and 97.3% on Project Gutenberg at 15M generations, despite slight performance declines as the corpus grows.BM25 continues to outperform P-SP across corpus sizes.
  • 5.3 Is retrieval an effective detector with a large retrieval corpus?: Retrieval detection is poor below 25% accuracy at 20 tokens but rises rapidly and plateaus at 50 tokens or more.Shorter queries produce more spurious matches, whereas longer queries better distinguish paraphrased generations.

6 Conclusion

The paper presents DIPPER as a controllable, context-aware paraphraser that exposes weaknesses in AI-text detectors, then proposes retrieval-based detection as a scalable defense. The authors report strong retrieval performance, while noting semantic collisions and limitations for short or constrained outputs.

  • 6 Conclusion: DIPPER paraphrases paragraphs in context, and its outputs can evade existing AI-text detectors while preserving input semantics.The paper open-sources DIPPER, its data, and code to support future research.
  • 6 Conclusion: Retrieval-based detection searches previously generated LLM responses for semantically similar matches to a candidate text.The approach depends on an API provider maintaining the generation corpus.
  • 6 Conclusion: 5TB of monthly ChatGPT outputs is estimated as the storage requirement for a retrieval corpus based on 2B monthly visits and 500 tokens per response.The paper compares this estimate with industrial-scale information-retrieval infrastructure.
  • 6 Conclusion: 1 second per retrieval on a 15M-sized corpus is reported on a 14-core CPU, with larger-scale searches described as parallelizable and GPU-compatible.The authors mention FAISS and large-scale search systems as relevant infrastructure.
  • 6 Conclusion: Semantic collisions may increase in larger databases, while retrieval is most effective for longer generations and may struggle on constrained or very short outputs.The paper also reports 100% retrieval accuracy for unperturbed AI-generated text.
  • 6 Conclusion: 0.8% is the reported accuracy drop when scaling from a 1M to 10M database with PG19-BM25.The authors characterize this result as encouraging despite using a basic retriever.

A.2 Limitations of retrieval for detection

The retrieval defense has practical scope and infrastructure constraints: it requires provider-held databases, applies mainly to closed APIs and longer outputs, and raises privacy and calibration concerns. The paper nevertheless reports small degradation at larger corpus sizes and suggests operational and retrieval improvements.

  • A.2 Limitations of retrieval for detection: Retrieval detection is specific to the API whose generations populate its database and cannot detect outputs from another API.This dependence distinguishes retrieval from general-purpose detectors.
  • A.2 Limitations of retrieval for detection: Retrieval is limited to closed-source LLMs because open-source users can generate outputs without storage in a central database.The paper notes that watermarking shares this limitation, unlike DetectGPT and classifiers.
  • A.2 Limitations of retrieval for detection: A provider must supply complex retrieval infrastructure as the corpus approaches billions of entries.At 5M queries per day, the paper estimates almost two billion entries after one year.
  • A.2 Limitations of retrieval for detection: Training-data memorization can produce false positives when human-written sequences are reproduced verbatim by a language model.The authors suggest retrieving over training data as a possible mitigation.
  • A.2 Limitations of retrieval for detection: Retrieval-based detection partially exposes previously generated text, creating privacy and membership-inference concerns.Suggested mitigations include trusted-user access, binary outputs, rate limits, and privacy mechanisms.
  • A.2 Limitations of retrieval for detection: 1% is the reported accuracy decrease when scaling PG19 from 1M to 15M generations, while unperturbed AI-generated text remains detected with 100% accuracy.The authors also note that detection can be difficult for constrained-output tasks and very short texts.
  • A.2 Limitations of retrieval for detection: The retrieval threshold lacks a formal relationship to response length, true-positive rate, and false-positive rate, so it must be estimated empirically.This contrasts with the formal threshold relationship the authors associate with watermarking.
  • A.2 Limitations of retrieval for detection: Timestamp filtering and more sophisticated retrieval methods are proposed to reduce search cost or improve performance, though complex methods may increase latency.The paper discusses these as possible improvements for billion-scale deployment.

C.1 Ablation studies on DIPPER

The ablations show that DIPPER’s contextual and multi-sentence capabilities improve paraphrase quality, while its outputs remain close to unperturbed GPT2-XL generations.

  • Context and sentence span: DIPPER uses context outside the input and paraphrases multiple sentences at once, distinguishing it from most surveyed paraphrasers.The ablation compares contextual DIPPER with a no-context model and compares three-sentence rewriting with sentence-by-sentence rewriting.
  • Context and sentence span: Across all four control-code configurations and three metrics, contextual DIPPER paraphrases are preferred over DIPPER-no-ctx paraphrases.At L60,O60, GPT3.5 perplexity prefers DIPPER 71% of the time, with average perplexity drops of 12.9 versus 14.2.
  • Context and sentence span: Multi-sentence paraphrasing is preferred over single-sentence rewriting across control codes, reaching 83% under L60,O60 by GPT-3.5 perplexity.GPT3.5 and RANKGEN usually prefer paraphrases produced from three sentences at a time before concatenation.
  • Similarity to originals: DIPPER paraphrases remain close to original GPT2-XL generations, with GPT3.5 preferring non-paraphrased text only 61% of the time at L20.The average perplexity changes from 11.1 to 11.5 at L20, while absolute scores remain close at more diverse settings.

C.2 Human evaluation of semantic preservation using DIPPER

The human evaluation tests whether DIPPER preserves meaning while increasing lexical and syntactic diversity. Annotators generally judged its paraphrases semantically equivalent, while identifying failures involving proper names and omitted content.

  • Evaluation design: The study evaluates 60 original–paraphrase pairs across lexical-diversity codes L20, L40, and L60 with three native English teachers or editors.Order diversity is fixed at zero, and annotators rate semantic fidelity on a Likert scale with comments.
  • Semantic preservation: Over 80% of annotator ratings classify DIPPER paraphrases as nearly equivalent or approximately equivalent to the originals.The table caption reports 81.8% near-equivalence in semantic meaning.
  • Qualitative strengths: DIPPER uses surrounding context to increase diversity while maintaining coherence, including context-based references and pronoun changes.The qualitative examples describe contextual rewrites such as referencing a song title and interchanging “he” and “Churchill.”
  • Qualitative shortcomings: High lexical diversity can alter newly created proper names, and DIPPER occasionally omits content enough to change meaning.The authors report that lowering lexical diversity or paraphrasing shorter paragraphs can mitigate these shortcomings.
  • Overall assessment: DIPPER combines paragraph-level rewriting, sentence reorganization, contextual conditioning, and user control of lexical and syntactic diversity.Automatic and human evaluations support context use, sentence reorganization, and high meaning fidelity.

E.2 Statistical outlier detection methods

Statistical outlier detectors identify machine-generated text from artifacts in its distributional properties rather than modifying the generation process. Related detector research also includes fine-tuned classifiers and paraphrase-based attacks.

  • Outlier detection: Outlier detection distinguishes human-written from machine-generated text using artifacts without modifying the generative algorithm.Examples include irregularities in entropy, perplexity, and n-gram frequencies.
  • Classifier methods: Classifier-based methods fine-tune models to distinguish human-written text from machine-generated text across applications and domains.Early applications targeted fake reviews and fake news, with later studies examining domains and decoding strategies.
  • Paraphrase attacks: Concurrent work demonstrates paraphrasing attacks against AI-generated-text detectors, while DIPPER adds discourse-level rewriting and fine-grained diversity control.The paper reports broader experiments across tasks, datasets, detection algorithms, and larger language models.

F.1 Details for training our paraphraser DIPPER

DIPPER is trained as a T5-XXL-based sequence-to-sequence paraphraser for controlled, context-aware rewriting. The surrounding experiments and discussion define its training setup and motivate robustness to perturbations.

  • Model training: DIPPER is a sequence-to-sequence Transformer initialized from T5-XXL 1.1 and fine-tuned on paraphrase-generation data.Training uses early stopping on validation loss, and the implementation uses JAX and T5X.
  • Model training: Training paraphrases a maximum of three consecutive sentences at a time to improve adherence to control codes.The model uses control codes for lexical and order diversity, with notation documented separately for release.
  • Evaluation setting: The long-form question-answering experiments use 250–350-word answers and questions sampled from six domains of r/explainlikeimfive.The dataset contains 500 randomly sampled questions from each domain.
  • Detector evaluation: The experiments attack model-specific watermarking and DetectGPT detectors alongside model-agnostic OpenAI, GPTZero, and RankGen detectors.The paper notes that paraphrased responses are generated by the paraphraser rather than the originally targeted model.
  • Motivation: The authors argue that model-specific detectors should incorporate paraphrase robustness because perturbation attacks pose downstream usability risks.This motivation is contrasted with the lower performance of model-agnostic detectors.

G.2 Robustness against text shortening and mixing attacks

The paper tests retrieval-based detection against attacks that shorten or mix LLM-generated text. Retrieval remains effective for unperturbed substrings, while additional DIPPER paraphrasing reduces detection rates.

  • The experiments evaluate retrieval against text shortening and mixing using BM25 retrieval on 2M PG19 machine-generated responses.The setup matches Section 5.3.
  • 86.2% to 94.7% detection at 1% FPR is achieved for unperturbed random substrings from single or multiple generations.These results use BM25 retrieval on a 2M-response PG19 machine-generated corpus.
  • 56.1% to 68.8% detection at 1% FPR remains after DIPPER paraphrasing is added to shortening and mixing attacks.

H ROC curves at different FPR

The paper reports that retrieval improves AI-generated text detection accuracy over baselines across settings, including diverse paraphrase attacks. ROC curves compare pre- and post-paraphrase detection at different clipping thresholds.

  • Retrieval significantly improves AI-generated text detection accuracy at 1% FPR over baselines across all settings, including diverse paraphrase attacks.
  • Figure 8 compares ROC curves before and after GPT2-XL text paraphrasing using solid and dashed lines, respectively.Different plots represent different clipping thresholds on the X-axis.
  • Table 8 presents representative outputs for each lexical code with annotator Likert ratings and comments.Bold text marks context, while red and green words show original and paraphrased content changes.
Loading 2303.13408v2…