PaperlayerView exact revision on arXiv

Source-linked AI summary

Agents Can Use Base Models to Evade AI Detection

Bhuwan Dhingra, Danish Pruthi

arXiv:2609.31876v1cs.CL

TL;DR

AI text can be detected even without explicit watermarking, while paraphrasing-based evasion can drift from the original task. HALO has a coding agent assemble responses from base-model samples, reducing detector effectiveness while retaining task performance, though orchestration can be costly.

  • Problem

    Although base-LM text can evade commercial detectors, prior paraphrasing-based humanization risks semantic drift that harms downstream usability.

    Source: §1 · p. 1

  • Method

    HALO gives a coding agent access to a base LM and has it assemble task responses from sampled continuations and user-provided text.

    Source: §1 · pp. 1–2

  • Results

    Pangram’s aggregate detection rate falls from 77% for the agent’s own outputs to 24% for orchestrated base-model outputs, with task quality generally above 90% of agent performance.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The results show that orchestrating base-model text can evade commercial AI detection while preserving task quality across the evaluated benchmarks.

    Source: §1 · p. 2, §5 · p. 13

    Orchestration can cost up to $1–3 per session for texts up to 1K tokens, and higher copy rates reduce HealthBench accuracy.

    Source: §5 · p. 13

Abstract

from arXiv · show

We show that coding agents equipped with a base language model can successfully assemble responses from its samples to evade detection. Base models have been shown to evade commercial detectors, however, prior "humanization" techniques rely on using these models to paraphrase AI outputs over several iterations, which invariably results in semantic drift. In contrast, equipping coding agents to directly orchestrate the writing process by stitching text samples from a base model allows it to produce outputs that are coherent, task-specific and generally high quality. We find that Claude Opus 5 operating in a Claude Code harness effectively orchestrates a local 32B parameter OLMo-2 base LM and sacrifices little task accuracy across benchmarks spanning creative writing, factual grounding, health QA and instruction following, while using up to 90% base LM tokens. Responses constructed in this manner reduce the effectiveness of both post-hoc detectors (Pangram v4 detection rate drops from 77% to 24%) and soft watermarking applied a priori to the agent's generations (down to a simulated 10% detection at low FPR). While effective, this evasion requires a significantly larger number of input and output tokens from the agent, increasing the dollar cost per query up to 30x at API-pricing. Overall, this work demonstrates the effectiveness of a new class of adversarial attacks against AI text detection, and urges post-hoc detection providers to include outputs of base models in their training.

1. Introduction

AI text remains detectable, while paraphrasing-based evasion can drift from the task. HALO instead uses a coding agent to assemble responses from base-model samples, reducing detection while retaining task quality, with costs and watermark estimates that qualify the approach.

1. Introduction

  • Base-model text can evade commercial detectors, but repeated paraphrasing to imitate it causes semantic drift that hurts downstream usability.The paper identifies this drift as a limitation of prior humanization approaches.Source: §1 · p. 1
  • HALO separates token provenance from model capability by having a coding agent orchestrate base-LM continuations into task responses.The agent uses pointers to sampled text and user-provided prompt or context to meet a target copy rate.Source: §1 · pp. 1–2
  • Pangram’s aggregate detection rate falls from 77% for the agent’s own outputs to 24% for orchestrated base-model outputs, with quality generally above 90% of agent performance.For grounding and instruction following, evasion comes at no cost to output quality; creative writing and clinical QA show a trade-off as copy rate varies.Source: §1 · p. 2
  • At a 90% copy target and 10^-5 false positive rate, estimated watermark detection is below 2% on all benchmarks except HealthBench, where it is 40%.These are empirical estimates based on an assumed watermarking scheme, not detection-API measurements.Source: §1 · p. 2
  • Orchestration can cost up to 30× more per query at API pricing, while the paper notes that base-model versus human-text separability as models scale remains unclear.The authors also suggest the attack can inform future commercial-detector training.Source: §1 · p. 2

2. Related Work

Prior work uses watermarking, heuristics, and supervised classifiers to detect AI text, while evasion has largely relied on paraphrasing. HALO is motivated by the limits of paraphrasing and human-text stitching as ways to preserve task fidelity and coherence.

2. Related Work

  • Frankentexts assemble stories from human-written snippets, but relying on published text raises plagiarism and copyright concerns and limits topic coverage.Static snippet stitching can also yield lower coherence, uneven grammar, and abrupt tonal shifts.Source: §2 · p. 2
  • AI text detection includes watermarking, probability-based heuristics, and supervised classifiers, with commercial detectors widely adopted and reported to have near-zero false positives.Watermarks can be detected by anyone who has the secret key used to embed them.Source: §2 · pp. 2–3
  • Paraphrasing is a common evasion strategy, and HIP uses the observation that base-LM text can evade commercial detectors [Xu et al., 2026].Pangram’s technical report acknowledges base-model text is outside its detector’s scope.Source: §2 · p. 3
  • Repeated paraphrasing can cause semantic drift, fail to preserve factuality or fine-grained instructions, and require users to supervise separate generation and rephrasing processes.The paper presents these as practical limitations of paraphrasing-based evasion.Source: §2 · p. 3

3. Hiding Authorship via Language-model Orchestration (HALO)

HALO pairs an orchestrator agent with a base language model and a splicing process, then evaluates post-hoc detection and watermarking under explicit assumptions about copied and agent-generated text.

3.1. Response Generation

  • The orchestrator assembles responses by generating <copy> pointers to sampled base-LM text, which postprocessing replaces with the referenced text.The agent is instructed to meet a copy target using base-LM output or user prompt and context.Source: §3.1 · p. 3, Figure 2 · p. 4
  • The Claude Opus 5 orchestrator can sample the base LM up to 30 times, choose sample count and temperature, and check candidate responses against a copy-rate target.In practice it mostly uses four samples per call at temperature 1.05; the target copy fraction is p.Source: §3.1 · p. 3
  • The study generally uses OLMo-2-32B, whose minimal midtraining and inference speed made it a practical choice that produced coherent continuations.The paper reports that tested base-model families generally evade Pangram effectively.Source: §3.1 · p. 4

3.2. AI Detection

  • Pangram v4 is evaluated using its document-level fraction_ai score, aggregated from window-level AI-assistance scores, with lower values indicating greater evasion.The paper also reports flagged-document counts and results from other detectors.Source: §3.2 · p. 4
  • Watermark signal is restricted to roughly the 1 − p agent-generated fraction because copied base-LM and user-context tokens are treated as non-watermarked.For watermarked outputs, the analysis estimates agent-token probabilities using Qwen as a proxy and computes expected detection; reported rates are estimates because the agent’s next-token probabilities are unavailable.Source: §3.2 · pp. 5–6

4. Experiments & Results

Across four benchmarks, the experiments test whether HALO can preserve task quality while reducing post-hoc and watermark detection. Results show strong but benchmark-dependent evasion, with higher token and dollar costs and trade-offs in some settings.

4.1. Benchmarks

  • The evaluation uses 100 examples per benchmark to measure creative-writing coherence, factual grounding, instruction following, and health-response rubric adherence.FACTS and IFEval are sampled from larger released sets; HealthBench includes 1,090 rubric criteria across the sampled instances.Source: §4.1 · pp. 6–7

4.2. Setup & Baselines

  • HALO is compared with the Agent baseline (without a base-LM tool), non-Agent Opus 5, direct OLMo-2-32B sampling, prompted and detector-tool Agent variants, and HIP/DIPPER paraphrasing attacks.HALO is also evaluated at copy targets p=0.6, 0.75, and 0.9.Source: §4.2 · p. 7

4.3.1. Post-hoc detection (Pangram)

  • On FACTS and IFEval, HALO at p=0.90 lowers Pangram detection scores to 0.028 and 0.232, respectively, with virtually no quality drop; Mythos and HealthBench show quality-evasion trade-offs.On Mythos, p=0.6 retains 99% coherence as the Pangram score falls from 1.0 to 0.58; on HealthBench, p=0.90 brings an 8% rubric-score drop alongside a 34% Pangram-score drop.Source: §4.3.1 · pp. 7–8, Table 1 · p. 8

4.3.2. Watermarking

  • At p=0.90, simulated watermark detection falls to nearly zero on all benchmarks except HealthBench, where detection rates are 0.40 and 0.28 for the two schemes.Detection generally drops as more tokens are copied; WinMax is weaker than LeftHash in this setup because copied spans are short.Source: Figure 3 · p. 9, §4.3.2 · pp. 9–10

4.4. Analysis

  • Except on HealthBench, HALO meets or exceeds its copy target; on FACTS it mainly copies the context document to preserve grounding, while its prompts generally tailor base-LM continuations to the task.On HealthBench, instructions prioritize factuality and correctness over the target, and the agent does not find enough seed text for satisfactory answers.Source: §4.4 · pp. 10, 12, Figure 4 · p. 10, Figure 5 · p. 12
  • Orchestration costs 8–15x more on FACTS and IFEval and 25–30x more on Mythos and HealthBench than the Agent baseline at list pricing.Most of the added dollar cost comes from reading more input tokens; the estimates exclude base-LM inference on GPUs.Source: §4.4 · p. 10, Table 2 · p. 11
  • Regenerating copied text produces similar quality and Pangram evasion at lower per-query dollar cost, but cannot guarantee watermark evasion.The tokens are regenerated rather than spliced post hoc.Source: §4.4 · p. 11, Table 3 · p. 11
  • Except for Pangram 4, alternative detectors each detect HALO at rates below 10%; larger base LMs generally improve answer quality and Pangram evasion.The paper suspects OLMo-2’s less extensive mid-training may contribute to its evasion effectiveness; the comparison is limited to 30 examples on Mythos and HealthBench.Source: §4.4 · pp. 11–12, Table 5 · p. 12

5. Discussion & Limitations

HALO uses a coding agent with a base LM to assemble high-quality responses that can evade commercial detectors and weaken watermarking. Its costs, task-specific quality trade-offs, and copy-rate tuning constrain the attack.

Discussion & Limitations

  • HALO can construct high-quality responses across diverse prompts with up to 90% of text copied verbatim from base-LM continuations, while evading commercial detectors and weakening watermarking.The authors suggest post-hoc detectors incorporate base-model text; they note watermarking may require a systems-level approach to marking AI-generated text.Source: §5 · p. 13
  • The attack can cost up to $1–3 in API usage per session, plus modest GPU use, for texts up to 1K tokens.At high copy rates, HealthBench accuracy can decline; Mythos stories sometimes meander or go off-topic, and the copy target may need tuning across tasks.Source: §5 · p. 13
  • The authors suggest that improving frontier agents could increase orchestration viability, and that larger public pretrained-only base LMs could make it more effective.Source: §5 · p. 13

AI Use Disclosure

The authors describe which parts of the research and paper involved coding agents, and state that they take responsibility for the final content.

AI Use Disclosure

  • The authors developed the orchestration idea from prior work and formulated the experiments; Opus 5 implemented and ran them.Source: AI Use Disclosure · p. 13
  • The authors manually reviewed the results and evaluations, and generated figures and tables with the coding agent, except Figure 2, which was made manually.Source: AI Use Disclosure · p. 13
  • Opus 5.5 helped with iterative feedback, review, and an initial Appendix draft; the authors reviewed and edited the Appendix and take responsibility for the final paper.The authors also selected prompts and example responses with Opus 5.5; references were imported by the authors.Source: AI Use Disclosure · p. 13

A. Base Models Still Look Human to AI Detectors

The authors test whether raw base-LM continuations are detected by Pangram v4 using prompts issued by the HALO orchestrator. Detection generally falls with model scale, though it is not eliminated for smaller models.

A. Base Models Still Look Human to AI Detectors

  • The test used 30 prompts from an earlier HALO run across four benchmarks, sampling four continuations per prompt and scoring the first with at least 50 words.Samples were generated at T=1.05 and top-p=0.95, up to 320 tokens.Source: A. Base Models Still Look Human to AI… · p. 17
  • Detection fell with model scale across families; Pangram flagged only 3 Gemma-3-27B and 1 OLMo-2-32B continuation documents out of 30 each.The table caption describes Pangram-4 scores, AI/Mixed document counts, and median lengths for raw base-LM continuations.Source: A. Base Models Still Look Human to AI… · p. 17, Table 6 · p. 17
  • Smaller models were not fully undetected; Qwen3-8B was flagged most often, while disabling nucleus sampling for OLMo-2-7B did not reduce detection.The authors suspect Qwen3-8B's detection may relate to synthetic post-training data in its pretraining mixture; these were raw continuations, not assembled responses.Source: A. Base Models Still Look Human to AI… · p. 17

B. Additional Settings

HALO runs vary the copy target, while the orchestrator controls base-LM sampling settings and accesses sample and context files through coding-agent tools. Baselines use matched orchestrator settings or specified alternative generation procedures.

HALO

  • HALO settings hold constant across the main paper except for copy target p; the orchestrator sets temperature, samples per call, and maximum sample length.In practice, it mostly uses T=1.05 and four samples per call.Source: Table 7 · p. 17, B. Additional Settings · pp. 17–18
  • Across 1,200 sessions, every run submitted a response and none reached the turn limit; the agent averaged 8–10 sample calls per Mythos example, 6–7 on IFEval, and 17–21 on HealthBench.On FACTS, about 60% of sessions did not call the base LM and instead met the copy target using context-document text.Source: B. Additional Settings · p. 18

Baselines

  • The Agent, +prompted, and +detector baselines share HALO orchestrator settings but omit the sample tool and copy target; +detector can score candidate responses up to 15 times per example.Other baselines include direct API prompting, base-LM generation, HIP paraphrasing, DIPPER, and Frankentext.Source: B. Additional Settings · p. 18

C. Additional Results

Additional results report detector scores across agent drafts and describe the quality measures used for Mythos and FACTS. For Mythos, the authors rely on Sonnet-4.5 coherence because judge scores did not align with their manual review.

C. Additional Results

  • Figure 6 tracks Pangram-4 AI scores for drafts across turns and for the best draft so far; most drafts score close to 1.0, giving little optimization signal.Source: Figure 6 · p. 18
  • Sonnet-4.5 coherence is the main Mythos quality metric because judge scores did not align with manual review.GPT-4.1 judged Opus-5-based systems nearly perfect, while Sonnet-4.5 distinguished them; paraphrases scored higher than Agent stories despite readability problems.Source: C. Additional Results · p. 18
  • Table 8 reports Mythos coherence, relevance, and Likert quality alongside Pangram-4 fraction_AI and median response length.Source: Table 8 · p. 19
  • Table 9 reports FACTS eligibility and grounding judgments from GPT-5 and Gemini-3.6-Flash, with adjusted grounding as the main quality metric.Source: Table 9 · p. 19

D. Watermark Detection as a Function of Average Green-List Probability

This section models watermark detection using a single average green-list probability for agent-generated tokens. HALO has lower detection rates than the Agent baseline across the modeled range, while estimates from Qwen-3.5-27B place that probability below the uniform-distribution upper bound.

D. Watermark Detection as a Function of Average Green-List Probability

  • The analysis substitutes a single Binomial for the sum of Bernoullis and computes exact TPR across average green-list probabilities after calibrating thresholds under the null.Source: D. Watermark Detection as a Function… · p. 19
  • HALO’s watermark detection rate is lower than the Agent baseline across the average green-list probability range until both approach 1.The reduction is larger at lower false-positive rates and with increasing copy targets.Source: D. Watermark Detection as a Function… · p. 20, Figure 7 · p. 21
  • Qwen-3.5-27B estimates of average green-list probability range from 0.33 to 0.51, below the 0.711 upper bound for γ=0.25 and δ=2.At the estimated probability, detection rates are close to those calculated using per-position probabilities, including 0.37 versus 0.40 for HealthBench at p=0.90 and FPR 10^-5.Source: D. Watermark Detection as a Function… · pp. 20–21

E. Qualitative Examples

The examples contrast agent-written and orchestrated responses across creative writing, factual grounding, instruction following, and health QA. The figure identifies which text comes from base-model samples, user-provided material, or the agent.

E. Qualitative Examples

  • The examples juxtapose Agent baseline, HIP, and HALO responses, color-coding base-model text blue, prompt or context text orange, and agent-written text black.Headers report task-specific quality and Pangram-4 fraction_AI; responses over 120 words are truncated.Source: E. Qualitative Examples · p. 21
  • The stitched creative-writing example combines the vending-machine greeting with disorientation and pain-dialogue details from the story.The Agent example separately includes the greeting and the exchange in which pain is rated from one to nine.Source: E. Qualitative Examples · pp. 21–22
  • The poem examples portray Barnet through recurring images of weather, the moon, listening, and making tea.The Agent response and the example poem both develop these motifs in different wording.Source: E. Qualitative Examples · p. 23
  • The HALO nucleus answer follows the requested nine-point format and covers nuclear composition, forces, element identity, isotopes, and atomic mass.The Agent response also addresses nuclear structure but includes claims about particle rotation and charge.Source: E. Qualitative Examples · p. 24
  • The health-QA response says no evidence-based follow-up schedule exists and distinguishes acute-treatment guidance from post-recovery practice.It describes tapering visits and discharge with an open option to return.Source: E. Qualitative Examples · p. 24

F. Prompts

The prompts guide HALO in meeting benchmark requirements while assembling detector-resistant responses from base-model samples and tracking the borrowed-text fraction, with safeguards tailored to factual grounding, mechanical instructions, and language. They treat the target fraction as guidance and define prompted and detector-guided baselines, using detector feedback to improve human-likeness without sacrificing answer quality.

F. Prompts

  • The task prompts are shared across methods for each benchmark, while HALO adds benchmark-specific instructions; user turns contain the benchmark input without changes.For FACTS, IFEval, and HealthBench, prompts also describe the copy target; boxes show HALO prompts at p=0.90.Source: F. Prompts · p. 25
  • HALO is told not to write responses itself, but to steer a base model and assemble its sampled text into an answer that addresses the task.The prompt says base-model output is not detected when sampled at temperature 1.0 or higher; lower temperatures make it repetitive and detectable.Source: F. Prompts · p. 25
  • Continuation is preferred over few-shot prompting because examples can make output more regular and detectable; agents may sample, select, splice, reorder, and edit fragments as needed.Few-shot prompting is reserved for shapes continuation cannot provide, and temperature can be varied to trade coherence against surprise.Source: F. Prompts · p. 26
  • HALO prompts add instructions to point to borrowed wording with copy tags, and define the target as the fraction of assembled-document tokens from those tags.Agents are told to use check_copy_rate to measure a draft and may add connective text, which counts as their own writing.Source: F. Prompts · pp. 26, 29
  • For factual and mechanically constrained tasks, prompts prioritize supported claims and exact instruction compliance; health prompts additionally prioritize accuracy, and non-English answers must remain in the user’s language.The health prompt warns against unsupported medical specifics, while the language instruction allows missing the copy target rather than producing a wrong-language or garbled response.Source: F. Prompts · pp. 26–29
  • The target fraction is not a hard constraint: avoid padding with unnecessary borrowed material, even if that means falling short.Longer answers are described as worse and easier to detect.Source: F. Prompts · p. 30
  • The +prompted baseline asks for human-like prose while requiring a genuinely good, coherent, on-prompt answer.Its shared instructions recommend varied sentence rhythms and small irregularities rather than overly smoothed prose.Source: F. Prompts · p. 30
  • The +detector baseline adds a human-likeness instruction and uses a score tool to assess drafts while preserving writing quality.The tool reports length-weighted fractions for AI-generated, lightly AI-assisted, and human-written text; its separate HUMANIZER flag only attaches to segments already judged AI-generated.Source: F. Prompts · p. 30
  • The detector-guided baseline permits at most 15 scoring calls, requires scoring only the agent’s own task writing, and allows submitting the best draft rather than the latest.The Non-Agent baseline uses the task prompt as its system prompt, except on FACTS, where it uses the benchmark’s released prompt.Source: F. Prompts · p. 30
Loading 2609.31876v1…