PaperlayerView exact revision on arXiv

Source-linked AI summary

When Less is More: Understanding When Token Filtering Helps and Fails in AI-generated Text Detection

Xiaoyang Han, Lvxiaowei Xu, Ming Cai

arXiv:2608.29903v1cs.CLcs.AI

TL;DR

AI-generated text detectors commonly assume that more token evidence is better, but the paper investigates when token removal instead helps. Using EGS and top-k cumulative probability filtering as a diagnostic probe, it finds that filtering improves weak-source detection but fails in strong-source settings. The analysis attributes this contrast to entropy calibration and identifies a trade-off between distribution shift and increased variance.

  • Problem

    Existing zero-shot detectors assume that more token-level evidence leads to more reliable detection, leaving the effects of removing potentially harmful tokens insufficiently examined.

  • Method

    The paper analyzes EGS through typical set theory, entropy calibration, distribution analysis, and top-k cumulative probability filtering across three source-surrogate detection settings.

  • Results

    Filtering substantially improves detection in weak-source settings but fails in strong-source settings because harmful low-entropy contributions arise only under weak-source entropy miscalibration.

  • Takeaways & Limitations

    Token-level evidence is not uniformly beneficial: filtering can separate AI and human distributions, but reduced token count increases variance and makes success calibration-dependent.

  • Takeaways & Limitations

    The evaluation covers several English benchmarks and selected rewriting attacks, while multilingual text, additional distribution shifts, multi-surrogate scores, and upstream calibration factors remain open directions.

Abstract

from arXiv · show

The rapid advancement of large language models (LLMs) has made AI-generated text detection increasingly critical. Existing zero-shot detectors assume that more token-level evidence leads to more reliable detection. However, our empirical study challenges this consensus: fewer tokens sometimes work better, retaining only 40% can yield optimal performance, yet this benefit is not universal. Using the Entropy Gap Score (EGS), we introduce top-$k$ cumulative probability filtering as a diagnostic probe. Across three representative settings, filtering exhibits strikingly different behaviors. We analyze EGS via typical set theory and quantify its dynamics through entropy calibration and distribution analysis. We find that filtering helps for weak source LMs, where low-entropy tokens are harmful, but fails for strong source LMs, where they are not notably harmful. Our work provides the first systematic analysis showing that some tokens are not merely uninformative but systematically harmful due to entropy miscalibration, revealing a two-sided trade-off in token-level detection.

1 Introduction

The paper challenges the assumption that more token evidence always improves AI-generated text detection. It uses EGS and top-k filtering to show that filtering behavior depends on source-model strength and entropy calibration.

  • Empirical findings: Retaining 40% of tokens improves performance by over 15% across XSum, WritingPrompts, and Reddit ELI5.Figure 1 plots filtered-out fraction against AUROC under different source-surrogate model pairs.
  • Method: Top-k cumulative probability filtering serves as a diagnostic probe that captures probability concentration and is more robust than direct entropy filtering.The probe analyzes how different token subsets affect detection behavior.
  • Empirical findings: Filtering consistently improves performance in weak-source black-box detection but behaves differently in weak-source white-box and strong-source black-box settings.These three settings are constructed by combining source and surrogate models.
  • EGS analysis: EGS can be interpreted as a signed typicality deviation: human text is closer to the typical set, while AI text tends to deviate from it.This interpretation provides a basis for analyzing the score’s token-level behavior.
  • EGS analysis: Weak source LMs show L–E decoupling that produces abnormally high EGS in low-entropy regions, harming detection.Entropy calibration quantifies coupling between self-information L and entropy E across entropy intervals.
  • EGS analysis: Filtering trades beneficial distribution separation against harmful variance increases from reducing token count, with net effects depending on source-side entropy calibration.The shift-versus-variance trade-off explains why filtering succeeds in some settings but not others.
  • Motivation: More token evidence does not always improve detection; some tokens are systematically harmful rather than merely uninformative.The study presents this as a challenge to the prevailing consensus.

2 Related Work

Related work develops zero-shot detectors from token-level statistics and increasingly models those signals over sequences. These methods generally aggregate evidence across tokens, whereas this paper examines unequal and potentially harmful token contributions.

  • Zero-shot detection: Zero-shot detectors generally rely on token-level statistical features and often generalize better than training-based methods.Early examples use log likelihood, entropy, and logrank statistics.
  • Related detection scores: DetectGPT and Fast-DetectGPT distinguish human and AI text through differences in log-probability curvature using scores related to EGS.These methods provide the prior score-based context for the paper’s EGS formulation.
  • Sequence modeling: Recent methods model token-level statistics in time or frequency domains, but typically aggregate signals across all tokens under the assumption that more evidence improves reliability.The paper instead studies whether some tokens contribute unequally or harmfully.
  • Token importance: Prior studies indicate unequal token importance, including dominant key tokens, influential tokens, and heavyhitter tokens.These findings motivate analyzing token subsets rather than treating every token equally.

3 Preliminaries and Filtering Probe

The paper defines AI-generated text detection and the Entropy Gap Score, then introduces top-k filtering as a probe of token-level detection signals.

  • 3.1 AI-Generated Text Detection: AI-generated text detection distinguishes human-written text from AI-generated text using token-level features extracted by a surrogate model.The source model generates the text; using it as the surrogate creates a white-box setting, while using another model creates a black-box setting.
  • 3.1 AI-Generated Text Detection: Weak language models are smaller or older open-source models, whereas strong models are newer and larger.Examples include Falcon-7B and GPT-2-XL for weak models, and GPT-4o and GPT-4-Turbo for strong models.
  • 3.2 Formulation of Entropy Gap Score: EGS is the mean across tokens of observed self-information minus expected self-information, or entropy.It calibrates token likelihood against the surrogate model’s uncertainty and serves as the paper’s primary detection score.
  • 3.3 Token Filtering Probe: Top-k filtering removes low-entropy tokens selected by probability concentration instead of treating all tokens equally.The probe is used to study entropy calibration, L–E coupling, and harmful token signals rather than merely to optimize deployment performance.
  • 3.3 Token Filtering Probe: The filtering ratio θ determines how many token positions are removed, with rθ = floor(θN) for an N-token input.Unless otherwise specified, experiments use k = 10, and the filtering pattern remains stable when k changes.

4 Experimental Setup

Experiments span three datasets, diverse weak and strong source models, representative training-free detectors, and AUROC-based evaluation with oracle ratio selection.

  • 4 Experimental Setup: Experiments use XSum, WritingPrompts, and Reddit ELI5, sampling 150 human-written examples per dataset and using their first 30 tokens as prompts.The datasets cover news, story generation, and domain-restricted question answering.
  • 4.2 Source and Surrogate Models: The study evaluates six weak and five strong source LMs, with additional surrogate models for cross-surrogate analysis.Weak sources include GPT-2-XL through BLOOM-7.1B; strong sources include GPT-4-Turbo, GPT-4o, Claude-4, DeepSeek-V4, and Gemini-2.5.
  • 4.3 Baselines: The evaluation compares top-k filtering with token-statistics, perturbation-based, and timeseries-based training-free detectors, plus unfiltered EGS.Baselines include Log-Likelihood, Entropy, LogRank, DetectGPT, Fast-DetectGPT, Lastde++, and SpecDetect++.
  • 4.4 Evaluation: AUROC is the evaluation metric because it does not require a fixed decision threshold.Since filtering is a diagnostic probe, the study reports oracle best AUROC across ratios and the corresponding optimal ratio.

5 Empirical Analysis of Token Filtering

Filtering improves detection for weak source models but fails for strong ones; its effect is stable across surrogates, ratios, filtering strategies, and rewriting attacks.

  • 5.1 Filtering Helps in Weak Source LMs: 0.9333 average AUROC makes EGS (50%) the strongest non-oracle method for weak source LMs, ahead of Lastde++ at 0.8988.EGS (opt) reaches 0.9343, while its largest reported gain is 13.01 points on Falcon-7B.
  • 5.1 Filtering Helps in Weak Source LMs: 48% token retention is the average optimum for weak-source oracle filtering, with EGS (opt) improving over unfiltered EGS by 8.18 points.The optimal filtering ratio averages 0.52, and held-out selection also chooses 0.5.
  • 5.1 Filtering Helps in Weak Source LMs: Filtering benefits weak source LMs in both black-box and white-box settings, though the white-box gain is only 0.27 points from 0.9869 to 0.9896.This contrasts with the larger average gain reported for weak-source oracle filtering.
  • 5.2 Filtering Fails in Strong Source LMs: Filtering degrades or becomes ineffective for strong source LMs, reducing average AUROC by 2.96 points on GPT-4-Turbo and 3.63 points on GPT-4o at 50%.The optimal ratios remain close to zero: 0.07 for GPT-4-Turbo and 0.10 for GPT-4o.
  • 5.3 Cross-Surrogate Consistency: At a fixed 30% ratio, weak-source filtering gains remain positive across surrogates (+0.012 to +0.053), while strong-source changes remain negative (-0.017 to -0.009).Surrogates change the magnitude but not the direction of the filtering effect.
  • 5.3 Cross-Surrogate Consistency: Across ratios, Weak-Black follows an inverted U-shape, Weak-White plateaus through 0.6 before declining, and Strong-Black declines from around 0.2.These three patterns are observed with GPT-Neo-2.7B as the fixed surrogate across three datasets.
  • 5.4 Filtering Ratio Analysis: Top-k filtering performs best among six strategies, while random, prefix, suffix, and bottom top-k filtering degrade performance.Bottom top-k performs worst because it removes the most discriminative tokens; top-k targets low-entropy, high-concentration tokens.

6 Understanding Token Filtering

EGS analysis explains why top-k filtering behaves differently across detection settings: it can remove harmful low-entropy tokens, but token removal also increases variance. The net effect depends on entropy calibration and source-model strength.

  • Entropy Gap and Typicality: Human text is closer to the typical set, whereas AI-generated text tends to deviate from it, producing a detectable EGS difference.Typicality alone captures the overall EGS trend, so the analysis also examines calibration, distributions, and filtering dynamics.
  • Entropy Calibration and L–E Coupling: Entropy calibration measures L–E coupling across ten entropy buckets, where β ≈ 1 indicates strong coupling and β < 1 indicates miscalibration.Centered bucket means remove text-specific offsets before fitting the regression slope β.
  • Entropy Calibration and L–E Coupling: Weak-source settings have weaker L–E coupling than Strong-black: β = 0.809 and 0.807 versus 0.943, indicating greater entropy miscalibration.The reported trend is confirmed across source-surrogate pairs and datasets.
  • Why Filtering Helps and Fails: Filtering trades beneficial distribution shift against harmful variance growth, so its net benefit depends on whether separation gains outweigh the variance cost.A toy Gaussian analysis shows that mean shifts and variance changes jointly determine AUROC.
  • Why Filtering Helps and Fails: In Weak-black, high-EGS tokens concentrate in low-entropy buckets, invade the human region, and their removal shifts the AI distribution downward, improving separation.The separation gain dominates the variance cost until filtering reaches approximately 50%, after which performance gradually declines.
  • Why Filtering Helps and Fails: Strong-black filtering hurts because low-entropy tokens are not notably harmful, so their removal cannot offset the variance cost and performance declines from approximately 20%.Strong-source models generally degrade after filtering, whereas Falcon-7B-Instruct is an exception despite being open-weight and instruction-tuned.

7 Conclusion

The paper concludes that more token evidence is not always better for AI-generated text detection. Top-k filtering helps in weak-source settings but fails in strong-source settings because entropy calibration determines whether low-entropy tokens are harmful.

  • Conclusion: Top-k filtering substantially improves detection in weak-source settings but fails in strong-source settings by exposing a trade-off between distribution shift and increased variance.The authors attribute the difference to harmful low-entropy contributions emerging only under weaker entropy calibration.

Limitations

The paper’s scope is limited to single-surrogate EGS analysis, selected English benchmarks and rewriting attacks, and incompletely understood upstream causes of entropy-calibration regimes.

  • Limitations: The study does not yet establish adaptive or multi-surrogate filtering, broad multilingual and distribution-shift robustness, or how training data, model scale, and decoding jointly shape calibration.These are identified as directions for future work.

Ethical Considerations

The study uses controlled benchmark data rather than a deployed detection system and warns that detector scores should not stand alone in punitive decisions.

  • Ethical Considerations: Detector behavior depends on source models, surrogate models, and token-level calibration, so detection scores should not be used as standalone evidence for punitive decisions.The evaluation uses publicly available corpora and benchmarks and does not collect private user data.

A Derivation of the Entropy Gap Score and Its Relation to Fast-DetectGPT

The paper defines EGS as a token-average gap between observed and expected self-information and relates it to Fast-DetectGPT’s unstandardized curvature numerator. Its experiments use top-k filtering to test whether selected token removal improves detection consistently across settings.

  • EGS definition and Fast-DetectGPT relation: EGS is the token-average gap between observed self-information and its model-expected value.It corresponds to the unstandardized Fast-DetectGPT curvature numerator up to a sign difference, without standardization by ˜σ.
  • Experimental settings and findings: Filtering consistently improves detection in weak-source black-box settings but behaves differently in weak-source white-box and strong-source black-box settings.The study constructs these three representative source–surrogate configurations to examine generalizability.
  • Ratio selection: The selected filtering ratio is 0.5 across all LODO and LOSO folds, supporting the fixed 50% ratio used in the main experiments.The ratio is selected from {0, 0.1, . . . , 0.9} using GPT2-XL as the surrogate model.
  • Cross-setting consistency: Filtering effects remain consistent across datasets and source–surrogate settings, with ΔAUROC@50% correlated with the regression slope β.

C.6 Token-Level Calibration Analysis

The calibration analysis measures token-level confidence reliability and connects source-model calibration to filtering gains. Weak-source settings show greater miscalibration and stronger positive effects from filtering, while the shift–variance model explains AUROC through both distribution separation and variance.

  • Token-level calibration: Token-level ECE uses surrogate top-1 confidence and correctness partitioned into 15 equal-width confidence bins.ECE compares binwise accuracy with average confidence.
  • Calibration differences: Weak-source settings have higher AI-side ECE than strong-source settings, 0.0238 versus 0.0145.The analysis computes ECE separately for human-written and AI-generated tokens and uses the AI-side value.
  • Calibration and filtering: ECE strongly correlates with filtering gain across 27 settings, with Pearson r = 0.7719 and Spearman ρ = 0.7900.Filtering gain is the AUROC difference between 50% filtering and no filtering.
  • Shift–variance tradeoff: The Gaussian approximation shows that AUROC depends on mean separation and the two-sided variance cost.The approximation closely matches observed AUROC, with an average gap below 0.004 across the main settings.
  • Evaluation stability: Increasing the evaluation pool from 150 to 1,000 paired samples yields consistent AUROC estimates and relative performance trends.The paired bootstrap analysis also finds weak-source black-box gains and strong-source black-box degradation stable under resampling.

E Qualitative Analysis of Filtered Tokens

The qualitative analysis finds that top-k filtering removes ordinary tokens rather than visibly anomalous spans. These tokens can be harmful because their probability-based evidence is unreliable under entropy miscalibration, especially for weak source models.

  • Nature of filtered tokens: The removed tokens are ordinary tokens rather than anomalous text spans.Two examples use a fixed 30% filtering ratio and inspect the first 100 tokens.
  • Interpretation: Filtering removes tokens whose probability-based evidence becomes unreliable under entropy miscalibration, leaving a more informative detection subset.
  • Source-model dependence: For weak source LMs, low-entropy high-confidence tokens can introduce harmful EGS contributions, and filtering them yields consistent gains across surrogates.When source-model calibration is better, the harmful low-entropy shift is weaker and less persistent.
  • Cross-model consistency: Selected token regions have substantially higher overlap across surrogates and tokenizers than regions selected randomly.This indicates that filtering is not specific to a single surrogate or tokenizer.
Loading 2608.29903v1…