PaperlayerView exact revision on arXiv

Source-linked AI summary

Who's Behind the Text? Robust Model Authorship Attribution

Minseok Ko, Hyunseok Lee, Woomin Song, Beomjun Kim, Jinwoo Shin, Kunwoong Kim

arXiv:2610.19583v1cs.AIcs.CLcs.LG

TL;DR

Existing model authorship attribution methods can degrade when domains or model versions change, even though identifying a text’s source may matter beyond detecting machine generation. ATLAS compares source-conditioned likelihoods with clipped token evidence, and across two benchmarks it consistently outperforms baselines, including by 9.00pp in joint domain-and-version shift. Its source signals also identify the known teachers of evaluated distilled models.

  • Problem

    Existing MAA methods often degrade under distribution shifts, while practical attribution must handle unseen domains and future versions of known model families.

    Source: §1 · p. 2

  • Method

    ATLAS learns source-conditioned text models and attributes text by comparing clipped token-level likelihood evidence across source families.

    Source: §1 · pp. 2–3

  • Results

    Across MAABench and OpenTuringBench, ATLAS consistently outperforms baselines; under joint domain-and-version shift, it leads the strongest baseline by 9.00pp.

    Source: §1 · p. 2, §7 · p. 9

  • Takeaways & Limitations

    ATLAS’s source signals correctly identify the known teacher of every evaluated distilled model, including students built on different base models.

    Source: §1 · p. 2

    The work evaluates attribution among a fixed set of source families under MAABench’s covered domains and generation conditions; handling unseen source families without full retraining remains future work.

    Source: §7 · p. 10

Abstract

from arXiv · show

As large language models (LLMs) are increasingly used to generate text, their misuse has become a growing societal problem, with implications for academic misconduct, model distillation, and cyberattacks. For these reasons, Model Authorship Attribution (MAA), which identifies the source model of a given text, is important for promoting the responsible use of LLMs and improving AI safety. In practice, attribution systems must remain reliable beyond the distributions observed during training. They may encounter texts from unseen domains and writing settings, as well as generations from future versions of model families that were not available when the system was trained. However, existing MAA methods often exhibit substantial performance degradation under such distribution shifts. To address this challenge, we propose Authorship Tracing via Likelihood-based Attribution of Sources (ATLAS), which formulates MAA through source-conditioned text modeling rather than direct source prediction. ATLAS learns how text from each source family is generated and attributes a given text by comparing its likelihood under each learned source condition. We aggregate this likelihood evidence at the token level with clipping so that no single token dominates the decision. We further introduce MAABench, a broad MAA benchmark for evaluating practical OOD settings such as domain and version. Our experiments show that ATLAS consistently outperforms existing baselines across MAABench and OpenTuringBench. In particular, when both the generation domain and the model version change on MAABench, ATLAS achieves 74.27% accuracy, 9.00pp higher than the strongest baseline. Beyond MAA, ATLAS can also help identify distillation source models by measuring which source model a target LLM most closely resembles. Our extended experiments show that ATLAS recovers the known teacher of every distilled model we evaluate.

1. Introduction

Model authorship attribution identifies which model family generated text, addressing cases where machine-generated detection alone is insufficient. ATLAS uses source-conditioned likelihoods to address domain and version shifts, and MAABench evaluates these shifts.

  • MAA identifies the source model family of text, extending beyond machine-generated-text detection when knowing that text is machine-generated is insufficient.Source attribution may also provide information about relationships between models, including possible teacher signals in output-based distillation.Source: §1 · p. 1
  • ATLAS learns source-conditioned text distributions and compares token-level likelihoods, clipping relative evidence so a few extreme tokens cannot dominate attribution.The method contrasts with direct source prediction.Source: §1 · pp. 2–3
  • MAABench evaluates generation-domain shift, source-model-version shift, and their combination.The benchmark includes texts from five LLM families across six domains.Source: §1 · pp. 2–3
  • Across MAABench and OpenTuringBench, ATLAS consistently outperforms existing baselines under OOD settings; in joint domain-and-version shift, it leads the strongest baseline by 9.00pp.The joint-shift setting is MAABench’s most challenging setting.Source: §1 · pp. 2–3
  • ATLAS’s learned source signals correctly identify the known teacher for every evaluated distilled model.This holds even when students of the same teacher are built on different base models.Source: §1 · pp. 2–3

2. Related Work

Prior work addresses machine-generated-text detection, model authorship attribution, and evaluation under distribution shift. This paper positions ATLAS and MAABench against gaps in robustness and benchmark coverage.

Machine-generated text (MGT) detection

  • MGT detection methods include classifier, pretrained-model-signal, watermarking, and learned-signal approaches; unlike them, ATLAS attributes which model wrote text.Watermarking applies only when a provider chooses to embed a mark.Source: §2 · p. 3

Model authorship attribution (MAA)

  • Existing MAA methods use frozen-model statistics, learned representations, prediction objectives, or adaptation, but often rely on source-discriminative signals that can be unstable under distribution shift.ATLAS instead learns source-conditioned text distributions.Source: §2 · p. 3

Benchmarks under distribution shift

  • Earlier MAA benchmarks cover fixed generators, domain variation, or multilingual attribution, but not version changes within a family; MAABench evaluates domain and version changes separately and jointly.EvoBench includes version shift but targets MGT detection rather than MAA.Source: §2 · p. 3

3. A Likelihood-Based Framework for Authorship Attribution

The framework defines MAA across source families and generation conditions, including out-of-distribution settings. ATLAS learns source-conditioned text models and attributes by aggregating clipped, relative token-likelihood evidence.

3.1. Problem Setup: Model Authorship Attribution

  • MAA predicts which of K candidate model families generated a text, including when test texts differ from training in generation conditions, source-model versions, or both.Generation conditions can include domain, prompt, and available context.Source: §3.1 · p. 4

3.2. ATLAS

  • ATLAS learns a text-generating condition for each source family and attributes text by comparing how well each condition explains it, clipping token evidence to limit the influence of extreme likelihoods.The framework compares source conditions token by token rather than aggregating token likelihoods directly.Source: §3.2 · p. 4
  • Training maximizes the ground-truth family’s text likelihood, using the source label only to select the condition; source conditions use a learnable prefix and source-specific LoRA update with a shared down-projection.The backbone is frozen during this adaptation.Source: §3.2 · p. 4
  • At inference, ATLAS compares each source’s token likelihood against competitors, clips pairwise relative evidence, averages the bounded scores across tokens and temperatures, and selects the highest-scoring family.The bounded score is invariant to source-independent additive shifts.Source: §3.2 · pp. 4–5

4. MAABench

MAABench evaluates model authorship attribution under domain and source-version shifts, separately and together. It uses generations from multiple model families and domains, with defined in-distribution and out-of-distribution evaluation settings.

4. MAABench

  • MAABench covers generation-domain shift and source-model-version shift in three OOD settings: domain, version, and joint shift.The joint setting applies domain and version shift at the same time.Source: §4 · p. 5

4.1. Data Construction

  • The benchmark uses five model families, four versions per family, and six generation domains; domain-OOD testing uses Patent and Stack Exchange, held out from training.Version-OOD training uses each family’s oldest version, with the remaining three versions held out for testing.Source: §4.1 · p. 5

4.2. Evaluation Settings

  • Evaluation includes one ID setting and three OOD settings, with 228,520 machine-generated texts and 35,443 human-written texts.Domain shift uses held-out domains with a persona-based system prompt; version shift uses held-out versions within the same families.Source: §4.2 · p. 5

5. Experiments

Experiments compare ATLAS with MAA baselines on MAABench and OpenTuringBench, then evaluate MGT detection and design choices. ATLAS’s strongest gains appear under OOD shifts, with additional analyses examining robustness and data efficiency.

5.1. Experimental Setup

  • The experiments compare ATLAS with nine MAA baselines and evaluate MGT detection, using Qwen2.5-7B as the main ATLAS backbone.MAABench evaluation covers ID, Domain, Version, and Joint settings; OpenTuringBench evaluation includes its original ID, E1–E4, and OOD settings.Source: §5 · p. 5, §5.1 · p. 6

5.2. Main Results

  • ATLAS beats the strongest MAABench baseline by 2.66pp, 12.32pp, and 9.00pp on Domain, Version, and Joint shifts, respectively.ID performance is nearly saturated, while ATLAS accuracy falls 1.75pp on Domain shift and 15.42pp on Version shift relative to ID.Source: Table 2 · p. 6, §5.2 · p. 6
  • 95.71% accuracy on OpenTuringBench OOD, 33.52pp above the best baseline; ATLAS also leads all four ID variations.The OOD advantage is substantially larger than on the ID variations.Source: §5.2 · p. 6, Table 3 · p. 7
  • ATLAS performs competitively with the strongest MAABench-trained MGT detectors across cases.The analysis treats MGT as the positive class and reports AUROC.Source: §5.2 · pp. 6–7, Table 4 · p. 7

5.3. Ablation and Scaling Analysis

  • Source-specific adaptation and clipped scoring both help under version shifts, with clipping adding 4.55pp on Version and 5.12pp on Joint shift.On ID and Domain shift, the component variants differ by less than 0.7pp.Source: Table 5 · p. 7, §5.3 · p. 7
  • Larger Qwen2.5 backbones generally improve attribution, with larger gains under OOD than ID; Qwen3-8B reaches 75.22% on Joint shift.Accuracy on Domain and Version shifts stays within about 1pp of Qwen2.5-7B when using OLMo-2-7B or Qwen3-8B.Source: §5.3 · pp. 7–8, Table 6 · p. 7, Figure 2 · p. 8
  • With 1,000 texts per source, ATLAS exceeds full-split DA-MTL on Version shift (80.10% vs. 72.78%) and Joint shift (67.20% vs. 66.15%).The full split contains 30,000 texts per source.Source: Figure 2 · p. 8, §5.3 · p. 8

6. Beyond Attribution: Distilled and Deployed Models

The authors test whether ATLAS source-conditioned likelihoods reflect family-specific patterns in distilled and deployed models. ATLAS attributes all six evaluated distilled models to their reported teachers, while exploratory results identify the closest candidate family for several deployed models.

Setup

  • For known-teacher evaluation, the authors train on GPT, Claude, and Gemini data and test six distilled models, two per teacher, built on different base models.The exploratory analysis considers seven recent deployed models without assuming distillation or a teacher from those families.Source: §6 · p. 8

Inference

  • Attribution scores standardize each source condition’s likelihood improvement over the frozen backbone against reference texts, then select the highest-scoring source.The rejection rate counts texts for which no source condition improves on the frozen backbone.Source: §6 · pp. 8–9

Results

  • ATLAS attributes all six distilled models to their known teachers, with attribution shares from 63.3% to 93.6%.Both GPT-distilled students are attributed to GPT despite having different base models, Qwen3-8B and Llama-3.1-8B.Source: Figure 3 · p. 9, §6 · p. 9
  • Among deployed models without reported teachers, MiniMax-M2.7 Turbo, Kimi-K2.6, and GLM 5.2 mostly resemble Claude; DeepSeek-V4-Pro mostly resembles GPT.Their respective attribution shares are 93.8%, 85.2%, 84.7%, and 90.6%.Source: Figure 3 · p. 9, §6 · p. 9
  • Rejection rates are at most 0.4% for the seven deployed models but reach 68.2% and 74.6% for Falcon3-7B-Instruct and Granite-3.0-8B-Instruct.Those two comparison models have no reported distillation from the three candidate families.Source: §6 · p. 9

7. Conclusion and Discussion

ATLAS uses source-conditioned text modeling for attribution and is reported to improve attribution under distribution shift, with further applications to identifying teachers of distilled models. The appendix details how MAABench is constructed and generated.

7. Conclusion and Discussion

  • ATLAS improves attribution under distribution shift and also identifies known teachers of distilled models and the source family most resembling deployed models.The conclusion reports consistent gains on MAABench and OpenTuringBench, especially when both domain and source-model version change.Source: §7 · p. 9

A.1. Source Models

  • MAABench covers five LLM families, with one training-time version and three subsequently released versions per family for Version and Joint shifts.The training version is used for ID and Domain shift; held-out versions are used for Version and Joint shift.Source: §A.1 · p. 19, Table 7 · p. 19

A.2. Human-Written Text Corpus

  • MAABench draws human-written seeds from six domains, including four in-distribution domains and patents and Stack Exchange as unseen domains.The human-written text is retained separately, while extracted metadata—not the original text—is used as generation input.Source: §A.2 · p. 19, Table 8 · p. 20

A.3. Prompt Construction

  • All five source families receive the same seed-specific prompt, which contains metadata rather than the original human-written text.For Domain and Joint shifts, the default system prompt is replaced with a persona-based instruction while the metadata-to-output structure is preserved.Source: §A.3 · p. 20, Table 9 · p. 21

A.4. Generation

  • Generation uses a common temperature of 0.7 and maximum length of 512 tokens, and outputs pass a shared validity filter before seeds are retained.A seed is retained only if generations from every source family pass the filter.Source: §A.4 · p. 21
  • Mistral produces the shortest retained texts and Claude the longest, a 37-word average gap.The appendix reports the resulting dataset composition and retained-text length statistics.Source: §A.4 · p. 21, Table 10 · p. 21, Table 11 · p. 21

A.5. Data Leakage and Overlap Control

The benchmark controls prompt inputs and corpus overlap so attribution is evaluated without explicit source identifiers or split-level seed overlap.

A.5. Data Leakage and Overlap Control

  • For each seed, source models receive identical prompts containing only seed metadata and shared output requirements.Models generate independently, without access to other models’ outputs or previous generations.Source: §A.5 · p. 21
  • Corpus construction excludes overlapping domain sources and applies temporal restrictions where applicable to reduce contamination of the human-written class.Source: §A.5 · p. 21
  • These controls are intended to keep attribution performance reflective of source-model differences rather than explicit identifiers or split-level seed overlap.Source: §A.5 · p. 21

B.1. Implementation Details and Evaluation Setup

The appendix describes ATLAS training and inference, the comparison baselines, and supplementary results on seed variability, OpenTuringBench, and MGT detection.

B.1. Implementation Details and Evaluation Setup

  • The method trains source-conditioned LoRA components on a 7B backbone with 24.79M trainable parameters (0.33% of the model), then averages clipped token-level scores across scored tokens and temperatures 1, 2, 4, and 8.Source: §B.1 · p. 22

B.2. Baselines

  • The MAA baselines include frozen-proxy feature methods and encoder-tuning methods that learn discriminative text representations.Fixed-feature methods reuse their classifier on shifted splits, while encoder-tuning baselines share the LLM2Vec trunk and training schedule.Source: §B.2 · p. 23, Table 13 · p. 23

B.3. Results over Random Seeds

  • ATLAS’s standard deviation is at most 0.31pp across settings, and its OOD margin over the strongest baseline exceeds every method’s run-to-run variation.The reported means average three training runs.Source: §B.3 · p. 25, Table 15 · p. 25

B.4. OpenTuringBench Setup

  • On OpenTuringBench, ATLAS outperforms all baselines on every task except ID in macro-F1.The benchmark contains seven open language-model source classes for the MAA setting.Source: §B.4 · p. 26, Table 17 · p. 26

B.5. MGT Detection

  • Supplementary fixed-operating-point results suggest that ATLAS also applies well to MGT detection.The reported metrics are TPR@1%FPR and FPR@95%TPR.Source: §B.5 · p. 27, Table 18 · p. 27

B.6. Ablation and Scaling Analysis

The ablations indicate that source-specific adaptation and clipped scoring chiefly help when model versions change, while backbone and data-scale analyses examine robustness across settings.

B.6. Ablation and Scaling Analysis

  • 6.73pp on Version shift and 5.96pp on Joint shift are the total accuracy gains from source-specific up-projections and clipped scoring over the shared-B, mean-score variant.Clipping adds 4.55pp and 5.12pp, respectively; the three variants stay within 0.7pp on ID and Domain shift.Source: §B.6 · p. 28, Table 19 · p. 28
  • The backbone-size analysis reports accuracy and macro-F1 across four Qwen2.5 sizes, with each trained on 10,000 texts per source family.Source: §B.6 · p. 28, Table 20 · p. 28
  • 75.22% Joint-shift accuracy is reported for Qwen3-8B, while three comparable-size backbones stay within about 1pp on ID, Domain, and Version shifts.The authors conclude that ATLAS’s gains do not depend on a particular backbone.Source: §B.6 · p. 28, Table 21 · p. 28
  • From 1k texts per source to the full split, ID accuracy rises by 1.12pp and Joint-shift accuracy by 7.10pp.Accuracy improves with training size in all four settings; Domain shift dips at 5k before recovering.Source: §B.6 · p. 29, Table 22 · p. 29

B.7. Additional Analysis

ATLAS retains high per-family recall under ID and domain shifts, while version and joint shifts concentrate errors on Gemini and GPT texts attributed to Claude. Accuracy declines as held-out versions move further from training, with ATLAS ahead of OTBDetector except on the newest GPT version, where they are within 1.5pp.

Additional analysis

  • Both analyses use each method's seed-42 run.The runs are listed in Table 15.Source: §B.7 · p. 30

Confusion matrices

  • Under version and joint shifts, 32.5% and 52.5% of Gemini texts and 13.7% and 24.2% of GPT texts are attributed to Claude.Claude recall remains above 92%; Mistral and Llama recalls are at least 76.2%.Source: §B.7 · p. 30, Figure 4 · p. 30
  • On ID and domain shifts, every family keeps recall above 97%.Figure 4 reports per-family recall as the diagonal of its confusion matrices.Source: §B.7 · p. 30, Figure 4 · p. 30

Accuracy by model version

  • Both methods lose accuracy as held-out versions move further from training, but ATLAS leads on every version except the newest GPT version, where they are within 1.5pp.The comparison orders each family's held-out versions from oldest to newest.Source: §B.7 · p. 30, Figure 5 · p. 30

C. Beyond Attribution: Distilled and Deployed Models

The section evaluates ATLAS for identifying the source family most resembling distilled and deployed models. It describes the attribution setup and reports how model shares and reject rates are organized, alongside the six distilled checkpoints.

Attribution setup

  • ATLAS is trained as a three-class attributor over GPT, Claude, and Gemini for five epochs, using the Appendix B.1 settings.Standardization uses 3,994 Mistral and Llama texts from 1,997 MAABench ID test seeds, none used in training.Source: C. Beyond Attribution: Distilled and… · p. 31
  • Each distilled model generates one text per ID test seed with the same prompt template, temperature 0.7, and at most 1,024 new tokens.Only the final answer is scored when a model emits a reasoning trace.Source: C. Beyond Attribution: Distilled and… · p. 31
  • The seven deployed models are queried through DeepInfra, Grok-4.7 through LiteLLM, and four named models are served locally from public checkpoints.The locally served models are A.X-K2, K-EXAONE-2.0, Motif-3, and Solar-Open2.Source: C. Beyond Attribution: Distilled and… · p. 31

Teacher identification

  • Table 23 reports each model's attribution shares and reject rate; Table 24 lists six public distilled checkpoints whose model cards state their teacher.Reject is the share of texts for which no condition explains the text better than the frozen backbone.Source: Table 23 · p. 31, Table 24 · p. 32
Loading 2610.19583v1…