PaperlayerView exact revision on arXiv

Source-linked AI summary

Relational Over-Regularization: Graph-Based AI-Generated Text Detection via Sentence Transition Deviation

Hyeonchu Park, Bugeun Kim

arXiv:2608.26694v1cs.AI

TL;DR

AIGT detectors often overfit generator-specific surface cues and degrade under distribution shift. This paper frames detection around sentence-pair transition deviations, operationalized by CSFG’s signed edge features. CSFG reaches 97.14% binary accuracy and generalizes well in the inflated-variance regime, but performance degrades for generators near or below the human transition-variance baseline.

  • Problem

    Existing AIGT detectors overfit generator-specific surface distributions and degrade under unseen conditions, while independent features miss sentence-level relational structure.

  • Method

    The paper formalizes Relational Over-Regularization and operationalizes it in CSFG using learnable signed transition-deviation edge features.

  • Results

    97.14% binary detection accuracy exceeds CoCo by +11.14pp with FPR = 1.57%, while generalization is robust for unseen LLMs in the inflated-variance regime.

  • Takeaways & Limitations

    Sentence-pair transition variance is a useful complementary discriminator, and δij acts as a false-positive calibrator rather than only a detection booster.

  • Takeaways & Limitations

    ROR is unreliable for some generators, including GPT-5, whose transition variance resembles or falls below the human baseline.

Abstract

from arXiv · show

Detecting AI-generated text (AIGT) remains challenging because existing approaches rely on token-level statistical signals or independent stylometric features, causing them to overfit to specific generators and fail under distribution shift. We identify a structural signal at the sentence-pair level: LLMs produce inter-sentence transition variance that deviates from human writing through inflated variance driven by recurring similarity bursts at paragraph boundaries and templated transitions. We formalize this as Relational Over-Regularization (ROR) and validate it across four benchmarks (p < 0.001). The central contribution is this relational problem formulation, not a novel GNN architecture; CSFG is one concrete instantiation for operationalizing ROR. To exploit this signal, we propose the Cross-Source Stylometric Fingerprint Graph (CSFG), a graph-based framework that encodes positional, sequential, semantic, and transition deviation signals as learnable GNN edge features. The per-edge signed deviation δ_ij operationalizes ROR without hand-crafted thresholds and acts as a false-positive calibrator. CSFG achieves 97.14% accuracy under binary detection, outperforming the strongest graph-based baseline by 11.14 pp, with a false-positive rate of 1.57% and robust generalization to unseen LLMs in the inflated-variance regime; detection degrades for generators whose transition variance falls at or below the human baseline.

1 Introduction

The paper reframes AIGT detection around sentence-pair transition structure, arguing that LLMs exhibit recurring similarity bursts and inflated transition variance unlike human writing. CSFG operationalizes this relational signal with signed edge deviations and reports strong binary-detection performance.

  • Existing detectors overfit generator-specific surface distributions and fail to capture sentence-level relational structure.
  • ROR identifies recurring similarity bursts, paragraph-boundary spikes, topic restatements, and templated transitions as an AIGT signature.The paper validates this structural signal across four benchmarks.
  • CSFG represents documents as sentence graphs whose edges encode signed transition deviations from document-level mean similarity.Positive deviations indicate local similarity bursts, while negative deviations indicate suppressed transitions.
  • 97.14% binary detection accuracy outperforms the strongest graph-based baseline by 11.14pp.The reported result also includes low false-positive rates on unseen LLMs in the inflated-variance regime.

2 Related Work

Prior AIGT detectors rely heavily on token-level or independent document-level signals that degrade under distribution shift. The paper instead emphasizes sentence-pair relational deviations as a complementary structural discriminator.

  • Detection performance degrades under unseen LLMs, domain shifts, and adversarial rewriting, suggesting reliance on token-level surface cues.
  • Macro-level semantic stability can miss recurring local similarity bursts because global averages absorb paragraph-boundary spikes.
  • Traditional stylometry treats document features independently, overlooking whether transitions are anomalous relative to each document’s own baseline.
  • The signed deviation δij captures departures from document-level mean similarity, distinguishing relational patterns missed by independent features and absolute coherence scores.
  • Unlike CoCo’s absolute similarity scalar, CSFG encodes signed deviation to distinguish local bursts from uniformly high similarity.

3 Method

CSFG models AIGT as a sentence graph whose relational dynamics combine sequential, semantic, positional, and transition-deviation information. Its training objective adds source attribution to discourage generator-specific surface artifacts.

  • CSFG jointly models sentence-level relational dynamics across human text and multiple LLM outputs.An auxiliary source-attribution head discourages generator-specific surface artifacts.
  • ROR includes inflated variance from similarity bursts and suppressed variance from hyper-uniform generation.The paper attributes these patterns to complementary next-token-prediction and RLHF pressures.
  • 3.1 Hypothesis: Relational Over-Regularization: The transition-variance signal can differ despite identical document-level mean similarity because global averages absorb local bursts.
  • 3.2 Document Graph Construction: Each document becomes a graph with sentences as nodes and relational dependencies as edges, including sequential and long-range semantic connections.
  • 3.2 Document Graph Construction: Edge features include normalized positional distance, sequential-edge indicators, and signed transition deviation from the global similarity baseline.
  • The model uses three EdgeConv layers, mean-max node pooling, binary detection, and an auxiliary source-attribution head used only during training.
  • 3.4 Training Objective: The joint objective combines binary cross-entropy with source-attribution cross-entropy, using λ = 1.0.The source-attribution term acts as a domain-adversarial regularizer against generator-specific surface cues.

4 Experiments

The evaluation covers binary detection, unseen-model generalization, and zero-shot robustness under text perturbations across four benchmarks. Results are averaged over repeated runs with accuracy, false-positive rate, and false-negative rate reported.

  • CSFG is evaluated in binary detection, unseen-model generalization, and zero-shot robustness settings.Robustness uses paraphrasing, humanization, and back-translation without retraining.
  • ROR is tested by comparing transition-variance distributions between human-written and AI-generated text across four benchmarks.
  • All experiments run 10 times and report mean Accuracy, False Positive Rate, and False Negative Rate.Accuracy is primary because evaluation sets are balanced; FPR and FNR distinguish false accusations from missed detections.
  • Experiments use HC3, AIGTBench, M4, and MULTITuDE, covering professional, social-media, multi-domain, and multi-generator settings.
  • Setting 1 compares CSFG with perplexity-based, Transformer, and graph-based baselines, retraining supervised baselines under identical data conditions.
  • Setting 2 tests a model trained in Setting 1 on held-out Claude Sonnet 4.6, GPT-5, and Gemini 2.5 Flash Lite without retraining.
  • Setting 3 perturbs generated text through paraphrasing, humanization, and back-translation, using Claude Sonnet 4.6 as the perturbation engine.The same model is used for generation and humanization, which the authors acknowledge as a limitation.

5 Results and Discussion

Across four benchmarks, AIGT shows significantly higher and more heavy-tailed transition variance than human text, while CSFG performs strongly in standard detection and on several unseen generators. Its low false-positive behavior is less reliable for some generators, short documents, and formulaic human writing.

  • 5.1 Empirical Validation of Relational Over-Regularization: AIGT has higher Var(T (D)) than human-written text across all four benchmarks, with p < 0.001 in every comparison.Effect sizes range from r = +0.22 to r = +0.38, confirming the inflated-variance regime in the evaluated benchmarks.
  • 5.1 Empirical Validation of Relational Over-Regularization: AIGT distributions also show heavier upper tails, reflecting recurring similarity bursts at paragraph boundaries and templated transitions.These local bursts distinguish the distribution beyond differences in mean variance.
  • 5.2 Binary Detection: 97.14% accuracy and 1.57% FPR make CSFG the strongest standard binary detector, with a 4.15% FNR.It outperforms SeqXGPT and CoCo while maintaining low false-positive and false-negative rates.
  • 5.2 Binary Detection: CSFG improves over CoCo by 11.14 percentage points in accuracy and 21.43 percentage points in FPR.Compared with SeqXGPT, it improves accuracy by 4.12 percentage points and reduces FPR from 12.43% to 1.57%.
  • 5.3 Unseen Model Generalization: On unseen LLMs, CSFG reaches 89.45% accuracy on Claude Sonnet 4.6 and 92.26% on Gemini 2.5 Flash Lite, with 0.66% FPR.GPT-5 is a major exception: CSFG reaches 68.85% accuracy but has 61.64% FNR because its transition-variance pattern is hyper-uniform.
  • 5.4 Robustness Against Text Perturbation: Perturbation results remain competitive, but CSFG’s FPR rises under paraphrasing and humanization and can rise sharply for short or formulaic human documents.CSFG reaches 90.96% accuracy under back-translation, while FPR reaches 64–71% on short documents and 25.00% on structured human news text.

6 Conclusion

CSFG operationalizes Relational Over-Regularization through signed transition-deviation edge features in a graph-based detector. It achieves strong binary detection and generalization in the inflated-variance regime, but has a structural boundary for hyper-uniform generators.

  • Method: CSFG encodes ROR as the learnable per-edge deviation δij within an edge-featured GNN, avoiding manual thresholds.ROR characterizes recurring inter-sentence similarity bursts that inflate and pattern-skew transition variance.
  • Results: 97.14% binary detection accuracy exceeds CoCo by 11.14pp, with FPR = 1.57%.The reported result covers three evaluation settings.
  • Results: CSFG generalizes to unseen LLMs with near-zero FPR in the inflated-variance regime.The framework targets generators exhibiting transition patterns similar to those observed during training.
  • Limitations: GPT-5 yields FNR = 61.64% because its transition variance falls below the human baseline, a pattern termed hyper-uniform generation.This boundary is not addressable by threshold adjustment because ROR targets similarity-burst presence.
  • Future work: Future work may add discourse signals, scalable graph construction, and perturbation-aware training to extend coverage and mitigate contamination.The proposed directions include rhetorical relations, argument structure, sparse or hierarchical graphs, and perturbation-aware protocols.

Limitations

The paper identifies scope and robustness limits for ROR-based detection. These include opposite-direction transition patterns, perturbation-induced contamination, representation limits, and computational scalability concerns.

  • Scope of ROR: ROR may fail for generators whose transition variance resembles or falls below the human baseline.GPT-5 produces hyper-uniform generation with FNR = 61.64%, showing that similarity-burst signatures are not universally discriminative.
  • Scope of ROR: Detectors trained exclusively on inflated-variance generators face the same constraint for opposite-direction relational patterns.The paper suggests bidirectional relational modeling as one possible remedy.
  • Perturbation-induced contamination: LLM rewriting can impose generator-specific transition patterns on human text, inflating false-positive rates independently of detector failure.CoCo shows similarly elevated FPR under the same perturbation conditions.
  • Representations and scalability: Sentence-level cosine similarity from roberta-base CLS embeddings captures only part of discourse structure and may share biases with AIGT.Higher-level rhetorical organization and argument flow are not represented, potentially attenuating δij for some generators.

The Use of Large Language Models

The manuscript used Grammarly and Claude-sonnet 4.6 during writing for grammar checking, language polishing, and clarity improvement.

  • Editorial tools: Grammarly and Claude-sonnet 4.6 were used for editorial purposes during manuscript preparation.The stated uses were grammar checking, language polishing, and improving clarity of expression.

A Environment

CSFG combines sentence-level relational features with auxiliary source attribution to detect AIGT. Its performance is stable across semantic-edge thresholds, while transition deviation calibrates false positives and all edge components contribute complementary signal.

  • Auxiliary loss weight: 97.22% accuracy is achieved at λ = 1.0, improving monotonically from 96.25% at λ = 0.0 as source-attribution pressure increases.FPR and FNR also decline consistently across the evaluated λ range.
  • Semantic edge threshold: 97.23% accuracy and 4.01% FNR occur at θ = 0.6, while accuracy varies only 0.09pp across the evaluated semantic-edge thresholds.The lowest FPR is 1.45% at θ = 0.7, so no threshold dominates all three metrics.
  • Edge feature components: The full edge-feature model reaches 97.14% accuracy, outperforming every independently ablated variant.The four-dimensional vector encodes semantic proximity, positional distance, edge type, and transition deviation.
  • Edge feature components: Removing positional distance causes the largest accuracy drop, reaching 96.89% with FNR of 4.67%.Removing the sequential indicator and cosine similarity lowers accuracy by 0.20pp and 0.22pp, respectively.
  • Transition deviation: Removing δij raises FPR to 1.65% despite 96.97% accuracy and the lowest ablation FNR of 4.40%, supporting its false-positive calibration role.The full model retains the most balanced error profile across accuracy, FPR, and FNR.

B.4 Effect of GNN Depth

GNN depth controls how much sentence-graph topology is aggregated. Three EdgeConv layers provide the best balance across detection metrics, while deeper propagation introduces over-smoothing-related degradation.

  • Effect of GNN Depth: 96.59% accuracy, 2.06% FPR, and 4.75% FNR occur with one layer, indicating that one-hop context is insufficient for reliable detection.Two layers improve accuracy to 97.09% while reducing both error rates.
  • Effect of GNN Depth: 97.15% accuracy and 1.42% FPR are achieved with three EdgeConv layers, the best combined performance across tested depths.Three layers approximate a coherent paragraph-level discourse unit through aggregation over three sentence hops.
  • Effect of GNN Depth: Four layers reduce accuracy to 97.10% and increase FPR to 1.53%, consistent with over-smoothing that weakens local discriminative structure.The FNR improvement from 4.27% to 4.26% is only 0.01pp and does not offset the false-positive increase.

C Generator-Level Transition Variance Analysis

Generator-level transition variance is not uniformly inflated: Gemini amplifies the ROR signal, whereas GPT-5 produces hyper-uniform patterns near or below the human baseline. This structural difference explains why CSFG generalizes unevenly across unseen generators.

  • Generator-Level Transition Variance Analysis: 0.8 × 10^-3 mean transition variance for GPT-5 is below the human value of 1.1 × 10^-3, reversing the ROR hypothesis’s expected direction.The paper terms this suppressed-variance pattern hyper-uniform generation.
  • GPT-5: Hyper-Uniform Generation: 61.64% FNR on GPT-5 is associated with near-overlap to human transition-variance distributions, whose KS distance is D = 0.2389.GPT-5 falls outside the high-variance decision region learned from training generators.
  • GPT-5: Hyper-Uniform Generation: Threshold adjustment cannot resolve GPT-5’s failure because uniformly small |δij| values do not trigger the burst signature targeted by ROR.The paper identifies retraining with GPT-5 examples or complementary features as likely requirements for reliable detection.
  • Generator-Level Transition Variance Analysis: 2.3 × 10^-3 mean transition variance for Gemini 2.5 Flash Lite exceeds the human value by more than twice, with KS D = 0.6482 and FNR of 14.81%.Its amplified ROR signal arises from abrupt topic transitions and is strongly discriminable by CSFG.
  • Implications for ROR-Based Detection: ROR-based detection is strongest for generators with recurring similarity spikes and weakest when transition variance falls within or below the human range.Claude Sonnet 4.6 occupies an intermediate position between the amplified Gemini pattern and the suppressed GPT-5 pattern.

D.1 Document Length and Domain Structure

CSFG’s relational signal depends on sufficient sentence-pair observations and discourse patterns that distinguish human conventions from AIGT regularities. Short documents and formulaic human news therefore create substantial boundary-condition failures, while GPT-5 humanization compounds its hyper-uniform regime.

  • Document Length: Short documents substantially degrade detection, with FPR rising by more than an order of magnitude for Claude Sonnet 4.6 and Gemini 2.5 Flash Lite.Fewer sentence pairs make the document-level mean similarity more sensitive to individual pairs.
  • Domain Structure: 25.00% FPR occurs on professionally edited XSum news articles, compared with 1.57% on the balanced Setting 1 test set.Recurring paragraph-boundary restatements in formulaic news discourse can resemble the similarity bursts targeted by ROR.
  • Boundary Conditions: CSFG’s low FPR should not be assumed to generalize uniformly across document lengths or writing domains.Short texts provide too few relational observations, while structured human genres can produce ROR-like patterns.
  • Evaluation Setup: Setting 3 compares paraphrase, humanization, and back-translation perturbations across unseen generators using the reported prompts and evaluation tables.The cross-generator summary measures FPR on perturbed human-written text.
  • Cross-Generator Robustness: GPT-5 humanization lowers CSFG accuracy to 47.93% and raises FNR to 55.35%, below SeqXGPT’s accuracy of 71.07% on the same condition.Humanization pushes GPT-5 outputs further toward the region CSFG associates with human text.
Loading 2608.26694v1…