Source-linked AI summary
On the Indistinguishability of Human v/s AI Generated Text
Jaee Ponde, Aritra Das, Mihir More, Debayan Gupta
TL;DR
AI-generated academic text is increasingly difficult to distinguish from human writing, especially after paraphrasing, raising concerns for authorship and academic integrity. This paper models human-guided, semantics-preserving perturbations using human writing samples and shows that repeated paraphrasing can move machine text toward the empirical human distribution under stability and mixing conditions, with finite-sample guarantees and improving stability as human data grows.
Problem
Distinguishing AI-generated from human academic writing is difficult because machine text can approach human quality and paraphrasing can reduce detector detectability.
Method
The paper formalizes quality- and semantics-preserving paraphrasing guided by human samples, analyzing distributional convergence, finite representations, and sample requirements.
Results
Repeated perturbations move the machine-generated distribution toward the empirical human distribution, with an explicit convergence rate, finite-sample extension, and square-root improvement in human-writing stability.
Takeaways & Limitations
Access to human examples changes paraphrasing from unguided stylistic drift into a process that can target the empirical distribution of a particular human writer.
Takeaways & Limitations
Finite-sample guarantees use a finite representation, and the theory assumes block mixing that is not empirically verified for practical paraphrasers.
Abstract
from arXiv · showhide
The rapid improvement of LLMs has made distinguishing AI-generated text from human writing a pressing problem. This challenge is further amplified by paraphrasing tools designed to make machine-generated text appear more "human". We study how access to human writing samples can be used to strategically paraphrase machine-generated responses toward the human distribution. Under a multi-sample setting with human and machine responses to the same prompts, we show that repeated paraphrasing moves the machine distribution toward the empirical human distribution under simple mixing and stability conditions. Our results derive an explicit convergence rate, extend the analysis to a finite-sample setting, and characterize how the required number of human samples and paraphrasing rounds scale with the desired error.
1 Introduction
The paper studies whether human writing samples can guide repeated, semantics-preserving paraphrasing of machine-generated responses toward a particular human distribution. It formalizes convergence, finite-sample guarantees, and the human-data requirements for this process.
- Motivation: AI-text detection is increasingly difficult because machine-generated academic writing can approach human quality and rewriting can reduce detectability.Institutions have therefore increasingly adopted AI-text detectors, despite their fragility to paraphrasing.
- Research question: The central adversarial question is whether a student’s past essays can guide machine essays toward that student’s writing while preserving meaning and quality.
- Approach: The paper formalizes human-guided paraphrasing as a dynamic that moves machine-generated text toward the empirical human distribution.It analyzes the movement’s speed and the amount of human data required.
- Contributions: Repeated semantics-preserving perturbations move the machine distribution toward the empirical human distribution with an explicit convergence rate.Theorem 1 establishes this movement under the paper’s stated conditions.
- Contributions: The analysis extends to finite samples and shows that paraphraser stability on human writing improves at a square-root rate as human examples increase.These results are given through finite-sample distance bounds and Theorem 3.
2 Preliminaries
The preliminaries define human and machine response distributions, quality and semantic constraints, and a quality-conditioned perturbation process guided by human samples. They also state admissibility, human-stability, and block-mixing assumptions for the analysis.
- Notation: For each prompt x, PM(y | x) and PH(y | x) denote machine-generated and human-written response distributions.The true human distribution is unknown in practice, motivating empirical human distributions from observed samples.
- Oracles: The quality control oracle accepts outputs meeting Q(x, z) ≥ q0, while semantic preservation accepts rewrites meeting Sem(x, y, z) ≥ s0.Quality concerns correctness, relevance, clarity, and usefulness; semantic preservation concerns retaining the original response’s meaning.
- Perturbation process: The admissible set Ai contains rewrites satisfying both quality and semantic-preservation thresholds relative to the original response yi.Repeated rewrites are checked against yi rather than only the latest rewrite to prevent semantic drift.
- Perturbation process: The accepted perturbation kernel restricts a base rewrite rule to quality- and meaning-preserving outputs and may use the human sample Hm.The kernel is assumed to remain supported on admissible outputs.
- Assumptions: Human admissibility requires human responses to satisfy the admissible-set constraints for machine responses.This is justified by pairing human and machine responses for the same prompt as comparable answers.
- Assumptions: Human stability measures how much perturbation shifts the empirical human distribution, while block mixing requires contraction over blocks of ℓm perturbations.Block mixing allows contraction to emerge over multiple rewrites rather than every individual rewrite.
3 Related Work
Related work covers AI-text detection, paraphrasing attacks, authorship obfuscation, iterative rewriting dynamics, and quality-preserving perturbations. The paper builds on these strands’ distributional, adversarial, and mixing perspectives.
- AI-generated text detection: AI-text detectors range from token-statistics tools and learned classifiers to zero-shot methods and watermark-based detection.
- Limits of AI detection: Distributional analyses show that detection becomes difficult when machine and human text distributions are close, while recursive paraphrasing can reduce detector accuracy.
- Paraphrasing attacks: Paraphrasing attacks preserve meaning while evading detectors through lexical and structural changes, recursive rewriting, word replacement, prompt search, or detector-based optimization.
- Authorship obfuscation: Authorship-obfuscation research includes untargeted evasion and imitation of a chosen writing style using target-author examples.
- Repeated rewriting: Studies of repeated rewriting report progressive replacement of authorial style, fixed points, attractor cycles, and Markov-chain behavior.
- Quality-preserving perturbations: Watermarking theory uses quality-constrained perturbation processes that mix over high-quality outputs to remove watermarks without the secret key.
4 Our Results
The results establish that block-mixing perturbations progressively erase dependence on initial machine responses, while human stability keeps the perturbed distribution close to empirical human writing.
- Block mixing: Block mixing requires a shared component after ℓ_m perturbations, leaving at most a 1−λ_m fraction dependent on the initial response.The Doeblin condition is imposed on the ℓ_m-step kernel rather than every individual perturbation.
- Preservation: The perturbation process preserves admissibility at every round when quality and meaning-preservation constraints hold.The proof uses induction: admissibility holds initially, and each accepted perturbation remains in A_i.
- Convergence: Theorem 1 shows that repeated perturbations erase dependence on the initial machine response while human stability maintains proximity to the empirical human distribution.At block boundaries, each block contracts the machine–human discrepancy by ρ_m, with additional drift from imperfect human stability.
- Convergence: The convergence argument iterates a one-block contraction relation, with ε_m = 0 yielding the corresponding exact-stability special case.The contraction is combined with the drift caused by imperfect human stability.
5 Sample Requirements
The paper derives finite-sample requirements for approximating the true human distribution after repeated perturbations, separating perturbation, observed-machine, and human-estimation errors. Human sample size controls the statistical floor, while perturbation rounds reduce the remaining error geometrically.
- Finite-sample representation: A finite semantic representation maps text to r categories, preserving the Total Variation upper bounds used for distributional convergence.The empirical approximation analysis treats these r cells as the perturbation chain’s finite state space.
- Finite-sample bound: The empirical approximation error decreases at the n^-1/2 rate and depends on the number of categories r and confidence level δ.More samples make the empirical distribution over categories increasingly reflect the underlying distribution.
- Finite-sample bound: The finite-sample bound has three components: perturbation error, error from p observed perturbed outputs, and error from estimating the human distribution from m samples.Each term vanishes as the number of perturbation rounds, perturbed outputs, and human responses increases.
- Explicit requirements: The sufficient sample requirement grows linearly with r, like 1/ϵ^2 in target accuracy, and logarithmically in 1/δ.Halving ϵ requires four times as many samples, whereas stronger confidence requires only a modest increase.
- Stability and perturbation rounds: The derived rate combines geometric decay in perturbation rounds with an m^-1/2 human-stability term.The geometric term decreases with the number of perturbation blocks, while the human-sample term depends only on m.
- Explicit requirements: The number of perturbation rounds grows logarithmically in 1/ϵ, whereas the required human sample size grows like 1/ϵ^2.Thus human sample size determines how small the target error can become; when ρ = 0, one perturbation block suffices.
6 Discussion
The discussion reframes human-guided paraphrasing as a data-dependent process: convergence depends on human-distribution stability, which improves with more human examples. The analysis is limited by its finite representation and unverified block-mixing assumption, motivating refined theory and practical training methods.
- Human-guided paraphrasing moves machine text toward a particular person’s distribution, unlike unguided paraphrasing, which drifts toward the paraphraser’s style.Access to human examples changes paraphrasing attacks from generic recursive rewriting into a question about data.
- Convergence has an error determined by how stable the human distribution is under the paraphraser.Theorem 1 links the convergence error to this stability condition.
- The finite-sample guarantees use a finite representation, so matching a coarse representation may leave detector-exploitable differences.This limits what the finite-sample theory establishes about full-text similarity.
- The theory requires block mixing, but this condition is not empirically verified for practical paraphrasers.The assumption therefore remains untested in the practical systems relevant to paraphrasing attacks.
- Future work could model variation within the human distribution and develop paraphrasers that use human examples while satisfying the assumed conditions.These directions target both sample-size analysis and practical realization of the theoretical setting.