PaperlayerView exact revision on arXiv

Source-linked AI summary

D2C-Routing: Dimension-to-Composition Evidence Routing for Mixed-Origin AI-Generated Text Detection

Xin Chen, Fuwei Zhang, Yiqi Tong, Wei Guo, Yutian Xiao, Fuzhen Zhuang

arXiv:2608.27380v1cs.CL

TL;DR

Binary or scalar detection does not identify whether content or expression changed in mixed-origin writing. D2C-Routing predicts those two source dimensions and composes them into four labels, reaching 0.8603 four-way Avg TPR@1%FPR on MixD2C, 6.5 points above the same-split RACE-local rerun.

  • Problem

    Mixed-origin writing can differ in content origin and expression origin, which binary labels or scalar AI-likeness scores do not distinguish.

  • Method

    D2C-Routing routes content-side and expression-side evidence to supervised source heads before a learned gate composes four collaboration labels.

  • Results

    0.8603 four-way Avg TPR@1%FPR is achieved by the disclosed D2C-Routing-based detector system, 6.5 points above the same-split RACE-local rerun.

  • Takeaways & Limitations

    Ablations support supervised source dimensions and learned composition, while AH expression recognition remains the main remaining challenge.

  • Takeaways & Limitations

    The main positive result is in-domain on MixD2C; direct HART-to-MixSet transfer reaches 0.2262 Macro-F1 and 0.4372 Macro AUROC.

Abstract

from arXiv · show

AI-generated text detection is commonly framed as a binary document-level judgment about whether a text is human-written or machine-generated. This framing breaks down for mixed-origin writing, where content origin and expression origin may differ. We cast mixed-origin detection as dimension-to-composition source attribution, inferring content origin and expression origin before composing them into four collaboration types. We propose Dimension-to-Composition Routing (D2C-Routing), which routes content-side and expression-side evidence to supervised dimension heads before a learned gated composition layer predicts the final label. On MixD2C, a reconstructed split derived from the HART mixed-origin benchmark, our disclosed D2C-Routing-based detector system reaches 0.8603 four-way Avg TPR@1%FPR, 6.5 points above the same-split RACE-local rerun. Core ablations support the routing design, while error analysis shows that distinguishing AI-content/human-expression from fully AI-generated text remains the hardest boundary. Code is available at https://github.com/bystander563/d2c-routing-artifact.

1 Introduction

Mixed-origin writing separates content origin from expression origin, making binary or scalar AI-likeness judgments insufficient. D2C-Routing addresses this by routing dimension-specific evidence through supervised heads and learned composition for four-way attribution.

  • Mixed-origin documents may combine human content with AI expression or AI content with human expression, which a scalar AI-likeness score cannot distinguish.
  • D2C-Routing models content origin and expression origin as separate source-attribution dimensions before composing four collaboration labels.The four labels are HH, HA, AH, and AA.
  • Content-side and expression-side evidence feed supervised dimension heads, followed by a learned gate that composes HH/HA/AH/AA predictions.Content cues include entity continuity and discourse motifs; expression cues include lexical choice, rhythm/POS patterns, and surface regularity.
  • The experiments test whether supervised source dimensions and learned composition improve low-FPR four-way detection.
  • 0.8603 four-way Avg TPR@1%FPR is achieved by the disclosed D2C-Routing-based detector system, 6.5 points above the same-split RACE-local rerun.The evaluation uses MixD2C, a reconstructed four-way split derived from HART benchmark files.

2 Related Work

Related work spans binary and scalar detectors, collaborative and fine-grained attribution, and HART-family mixed-origin benchmarks. D2C-Routing differs by assigning document-internal evidence to source dimensions before four-way composition.

  • Recent detection work extends beyond binary labels toward collaborative writing, fine-grained roles, manipulated text, and multi-generator benchmarks.
  • HART defines the HH/HA/AH/AA content-expression taxonomy, while RACE reports related baseline results under a 70/10/20 reconstruction.
  • Published RACE values are contextual because frozen sample IDs and checkpoints are unavailable, so the local run is not a reproduction of those numbers.
  • Training-free scalar detectors use probability curvature, language-model contrast, or frequency-domain analysis of token likelihood sequences.
  • D2C-Routing assigns document-internal coherence and discourse cues to content, and lexical, rhythm, and surface cues to expression, unlike RACE’s creator/editor-trace emphasis.

3 Problem Setup

The task factorizes four-way mixed-origin labels into content-origin and expression-origin predictions, then composes those dimensions. D2C-Routing aligns evidence pathways with the two source questions while preserving direct four-way supervision.

  • A detector maps an observed document x to predicted content origin, expression origin, and final collaboration label.The two source questions are separated from the final four-way decision.
  • The HART label space uses HH, HA, AH, and AA, with the first letter denoting content origin and the second expression origin.Each dimension has human and AI as its two source values.
  • Dimension supervision makes content and expression attribution explicit training targets instead of relying only on flat four-way correlations.
  • D2C-Routing uses entity-chain coherence and discourse motifs for content, and lexical-connective choices, rhythm/POS patterns, and surface regularity for expression.Syntax-specific routing remains exploratory.
  • HA and AH are structurally distinct despite potentially similar scalar AI-likeness because their changed source dimensions differ.HA has human content and AI expression; AH has AI content and human expression.

4 Method

D2C-Routing routes document-internal evidence through supervised content and expression pathways before learned gated composition into four collaboration labels. It supports shared- or dual-encoder variants and trains dimension, composition, and low-FPR objectives.

  • Evidence Routing: D2C-Routing routes entity-chain, discourse, lexical-connective, rhythm/POS, and surface-regularity evidence through dimension-aligned pathways.Content pathways model information organization, while expression pathways model linguistic realization.
  • Encoder Variants: The same routing design supports shared encoders and separate content-expression encoders, with dual encoders treated as larger-budget system variants.The raw document remains the directly observed object for both source dimensions.
  • Dimension Supervision: The content and expression heads separately predict AI-originated content and AI-originated expression from routed pathway states.AH/AA are positive for the content head; HA/AA are positive for the expression head.
  • Gated Composition: A learned gate composes pathway representations, the shared composition representation, and dimension predictions into HH/HA/AH/AA labels.The gate avoids a hard mapping that would discard uncertainty in partly entangled dimension representations.
  • Training Objective: The training objective supervises source dimensions, four-way composition, AH/AA separation, and AA retrieval under low false-positive constraints.Dimension and composition terms are paired with ranking terms aligned to strict low-FPR evaluation.
  • Detector-System Fusion: D2C-base Fusion combines four-way probability vectors from member models, selecting interpolation weights on development without test-set tuning.Official Level-1/2/3 scores collapse the fused four-way probabilities into positive label sets.

5 Experimental Setup

The experiments use MixD2C, a transparent four-way split reconstructed from released HART files, and separate exact-split comparisons, published references, controls, and detector-system implementation choices.

  • Dataset and Split: MixD2C contains 11,200/1,600/3,200 train/dev/test examples, stratified by domain and class, with AH as the minority class.The split is an evaluation reconstruction rather than a new dataset or taxonomy.
  • Baselines: RACE-local is a same-split rerun on identical MixD2C files, whereas published RACE values remain external references because frozen sample IDs and checkpoints are unavailable.The local rerun uses the released implementation with the strict rstdt RST parser setting.
  • Metrics: Official Level-1/2/3 tasks use AUROC, F1-score, and TPR@5%FPR, while four-way diagnostics include macro AUROC/F1 and class-wise TPR@1%FPR.The low-FPR metrics evaluate performance under strict false-positive constraints.
  • Baselines: The comparison separates RACE-local, published RACE references, same-split internal controls, and training-free scalar detectors.Internal controls include text-only RoBERTa, flat feature concatenation, D2C-Routing, and cost-matched ensembles.
  • Evaluation Protocol: D2C-Routing rows denote independently evaluated single models, while D2C-base Fusion is the development-selected primary detector system.The strongest single-model row is a dual-encoder RoBERTa-base variant.
  • Detector-System Evaluation: Fusion weights are selected on development, the candidate pool is fixed before testing, and no test-set interpolation tuning is used.The final fusion has three nonzero-weight members.

6 Main Results

On the exact MixD2C comparison, D2C-Routing improves average and AA low-FPR ranking, while controls support dimension supervision and learned composition without establishing uniform class-wise superiority.

  • Four-Way Low-FPR Results: 0.8603 four-way Avg TPR@1%FPR is achieved by D2C-base Fusion on the exact MixD2C split.Its AH and AA values are 0.7892 and 0.7708, respectively.
  • Four-Way Low-FPR Results: 0.8440 Avg TPR@1%FPR and 0.7701 AA TPR@1%FPR are reached by D2C-Routing, while RACE-local records 0.5752 AA TPR@1%FPR.D2C-Routing remains below RACE-local on AH: 0.7435 versus 0.7696.
  • Ablations and Controls: Shared D2C improves low-FPR ranking over text-only and flat controls, while the dual encoder is the strongest single model.The closely matched D2C cost-audit system improves Macro-F1, AUROC, and Avg TPR@1%FPR over specified controls, but its advantage over flat 5× is not significant at 95%.
  • Ablations and Controls: Removing dimension supervision or learned composition weakens four-way low-FPR performance.The ablations directly test the routing design rather than only feature or capacity changes.
  • Transfer and Error Diagnostics: Expression-origin transfer is the clearest external result, while PAN content transfer is small and direct four-way transfer to MixSet is negative.The AH branch diagnosis localizes the single-model weakness to expression-origin recognition.
  • Four-Way Comparison: Table 2 distinguishes contextual published references from identical-split MixD2C rows and remaps Humanized and LLM-Generated columns to AH and AA.The table presents the remapped order as HH, HA, AH, AA.

7 Conclusion

The paper presents D2C-Routing for mixed-origin text detection by predicting content and expression origins before composing them into collaboration labels. On MixD2C, the approach improves low-FPR performance, while AI-content/human-expression remains the hardest boundary.

  • D2C-Routing predicts content-origin and expression-origin dimensions before learned composition into HH/HA/AH/AA labels.The architecture uses supervised dimension heads followed by learned composition.
  • D2C-Routing improves average and AA low-FPR ranking over same-split single-model controls and RACE-local.
  • D2C-base Fusion provides the strongest disclosed detector-system result on MixD2C.
  • AH remains the hardest boundary, particularly for expression-origin recognition.
  • The results support supervised source dimensions and learned composition while identifying AH expression recognition as the main remaining challenge.

Limitations

The evidence is strongest for in-domain MixD2C performance and dimension-specific transfer, while several comparisons and interpretations remain constrained. AH expression recognition is the main single-model weakness, and the experiments do not establish a uniquely optimal routing assignment.

  • 0.2262 Macro-F1 and 0.4372 Macro AUROC from direct HART-to-MixSet transfer support dimension-specific transfer rather than broad out-of-domain four-way generalization.
  • Published RACE values are related-reconstruction references because frozen sample IDs and checkpoints are unavailable.
  • AH is the main single-model weakness: content-origin accuracy is 0.9477, compared with 0.6438 expression-origin accuracy.
  • ModernBERT shows no reliable D2C-over-text advantage, and correct, swapped, and fixed-random routing are statistically similar on the principal low-FPR metric.
  • The evidence supports dimension supervision and learned composition but does not establish a uniquely optimal or causally interpretable handcrafted feature assignment.

Ethical Considerations

The paper frames its detector as decision support rather than sole evidence for consequential authorship decisions, and emphasizes that benchmark results have protocol and coverage boundaries. Comparisons across HART, MixD2C, and related datasets therefore require careful interpretation.

  • Detectors should not be used as sole evidence for punitive decisions about authorship, academic integrity, employment, or access to services.
  • MixD2C labels describe controlled construction protocols rather than a person’s intent or honesty, and low-FPR evaluation does not remove false-accusation risk.
  • The intended use is decision support, auditing, and research, with human review and context-specific policy safeguards.
  • AH is the minority class in the reconstructed MixD2C split.
  • Published RACE rows are external references rather than local reruns from frozen sample IDs, despite mapping their class columns into HH/HA/AH/AA order.
  • HART’s original detector tables use the original protocol and mostly report binary risk tasks, so they are benchmark context rather than same-split MixD2C baselines.
  • HART reports AUROC 0.843 and 0.855 for Level-1 and Level-2 C2-T Fast-Detect, while C2-T Binoculars reaches AUROC 0.883 for Level-3.
  • MixD2C values should not be compared as exact head-to-head results against HART’s original-protocol detector table.

A.4 Additional Architecture Controls

The controls test whether D2C-Routing’s gains reflect supervised source dimensions and learned composition rather than frozen probing, feature flattening, backbone choice, or ensemble scale. Additional evaluations examine split robustness, transfer, and evidence-group signal across source dimensions.

  • Architecture controls: Avg TPR@1%FPR drops from 0.8206 to 0.5444 under frozen-last1, while AA TPR@1%FPR drops from 0.6777 to 0.3854.This shows the routed system is not merely using a frozen encoder probe.
  • Architecture controls: DeBERTa-base improves single-backbone low-FPR performance over the RoBERTa-base seed42 routed baseline, especially for AA, but remains below D2C-base Fusion.
  • Architecture controls: The controls support supervised source dimensions with learned composition, but do not establish that the handcrafted feature-to-dimension routing partition is uniquely optimal.The exact routing partition remains an inductive bias.
  • Transfer and robustness: Grouped splitting tests variant and topic overlap using 4000 groups with zero cross-split leaks, and performance drops under this stricter evaluation.The calibrated D2C system remains competitive, but only the accuracy advantage over text-only is significant; Macro-F1 and Avg TPR@1 intervals cross zero.
  • Transfer and robustness: On controlled PAN content transfer, D2C has only a small AUROC advantage over matched text-only and no consistent low-FPR advantage, while direct MixSet transfer remains weak.Broad four-way zero-shot generalization therefore remains unresolved within the reported transfer protocols.
  • Evidence probes: Entity/coherence and RST evidence reaches 0.8842 content-origin AUROC, while expression features reach 0.8515 expression-origin AUROC but also strongly predict content origin.These probes motivate dimension supervision and learned composition while cautioning against rigid feature-role interpretations.
  • Composition controls: Removing both margin and ranking losses yields 0.8146 Avg TPR@1%FPR versus 0.8206 in the corresponding seed-42 setting, and hard-negative OVR does not improve AA TPR@1%FPR.Ranking losses are therefore retained as evaluation-aligned optimization.

A.8 Mechanism Diagnostics

Mechanism diagnostics show that supervised dimension heads learn their intended binary targets, while gate values provide little explanatory signal. The main remaining weakness is recognizing humanized expression in AI-content texts, with short texts also performing worse.

  • Dimension diagnostics: The content head reaches AUROC 0.9937 and F1 0.9655, while the expression head reaches AUROC 0.9871 and F1 0.9720 on their intended binary targets.These results show that the intermediate source judgments are learnable under direct supervision.
  • Composition diagnostics: Gate means range only from 0.4975 for AA to 0.5067 for HH, so gate values provide little explanatory signal across labels.The dimension heads and class-specific error patterns are more direct diagnostics.
  • Composition diagnostics: Vector and scalar gates are statistically indistinguishable at ensemble level, with Macro-F1 0.9077/0.9154 and Avg TPR@1 0.8235/0.8290.The result indicates stable learned composition across gate parameterizations, not causal interpretability of individual gates.
  • Error analysis: The AH gap is concentrated in expression-origin recognition: AI-originated content is usually recognized, but humanized expression is often missed.Two targeted expression-head remedies failed the predeclared development criterion and were not test-evaluated.
  • Error analysis: Short texts reach Macro-F1 0.8633 and Avg TPR@1%FPR 0.6930, compared with 0.9303 and 0.8927 for the longest quartile.The analysis suggests mixed-origin detection benefits from sufficient discourse and entity evidence.
Loading 2608.27380v1…