Source-linked AI summary
Self-Generated Text Recognition: Quality Heuristics, Cross-Task Transfer, and Downstream Bias in LLM Evaluation
Jesse St. Amand, Callum Canavan, Sohaib Imran, Joseph Hewson, Aaron Lutz, Shi Feng, Puria Radmard, Lennie Wells
TL;DR
SGTR matters because recognizing model-generated text could enable collusion or biased evaluation in autonomous and multi-agent systems. This paper defines operationalizations of SGTR and evaluates how design choices affect recognition, quality-heuristic confounds, transfer from training, and downstream self-preference. SGTR accuracy varies substantially across evaluation conditions, transfers across held-out settings, and training can induce or redirect judge preferences.
Problem
SGTR could enable collusion in multi-agent systems and systematic bias in LLM-as-a-Judge evaluations, while different operationalizations produce substantially different accuracy estimates and conflicting prior results.
Method
The paper characterizes SGTR operationalizations across task domains, evaluation formats, and conversation structures, evaluates them, and uses SFT and AlpacaEval 2.0 to study transfer and self-preference.
Results
SGTR accuracy varies across formats, conversation structures, and domains; the quality heuristic is a dominant pairwise confound, while SFT transfers across held-out operationalizations and can induce or redirect judge preferences.
Takeaways & Limitations
SGTR should be assessed under deployment-relevant conditions and accounted for as a potential source of bias in safety-critical evaluation frameworks.
Takeaways & Limitations
The operationalizations may not generalize to several task settings, training uses three base models with small LoRA SFT datasets, and AlpacaEval measures self-rank change rather than ranking accuracy.
Abstract
from arXiv · showhide
Self-Generated Text Recognition (SGTR)--the ability of an LLM to identify its own outputs--poses risks to AI safeguards that rely on LLMs as evaluators or monitors. Specifically, an LLM may recognize outputs from other copies of the same model and make biased judgments or collude outright. Prior work has drawn conflicting conclusions about whether current models possess significant SGTR capabilities. We reconcile these findings by identifying key experimental design choices--which we term operationalizations--that drive divergent results. Evaluating 13-21 models across six operationalizations, we find that accuracy varies substantially with evaluation format (pairwise vs. individual assessments of text), conversation structure (presenting candidate text in user tags vs. assistant tags), and the domain of the task used to generate candidate text (e.g., coding vs. summarization). We corroborate previous observations that a quality heuristic--models attributing authorship to text they perceive as higher quality--is a dominant confound. We also find that improving a model's SGTR performance via SFT in one evaluation configuration can generalize to others. Training for SGTR additionally causes models to prefer their own outputs when acting as a judge in the AlpacaEval framework. Finally, we discuss the implications of our evaluations for the safety of future AI systems: our work suggests that, despite confounds, some models possess practical SGTR capabilities, and that training a model for SGTR in one setting can affect its self-recognition and self-preference more generally. We conclude that SGTR should be monitored and considered in the design of safety-critical AI applications.
1 Introduction
SGTR could affect situational awareness, monitoring, collusion, and judge reliability, yet prior studies disagree about current models’ capabilities. This work attributes divergent findings to experimental design choices and characterizes those choices.
- SGTR is the ability to distinguish a model’s own outputs from text produced by other sources.
- Robust SGTR could let monitored models distinguish authentic reasoning traces or environmental inputs from interventions, while facilitating collusion or biased judgments.
- Prior studies draw conflicting conclusions, with some reporting robust SGTR and others finding recent models near chance.
- The paper characterizes SGTR across operationalizations that vary by task domain, evaluation presentation, and conversation structure.
2 Operationalizations of SGTR
The paper defines SGTR operationalizations as combinations of design choices that affect measurement, including task, target, format, and conversation dimensions. These dimensions help explain conflicting prior results and expose quality assessment as a major confound.
- An SGTR operationalization is a specific combination of experimental design choices that affects how recognition is measured.
- Evaluations vary by task domain, evaluation target, format, and conversation structure, with recognition compared against preference to assess quality coupling.
- The study does not explore self definitions, non-self text pools, reasoning traces, or probing-based measurement.
- The quality heuristic links self-authorship judgments to perceived quality and self-preference bias in judge models.
- Prior studies report both robust recognition and weak or absent SGTR, depending partly on formats and quality-related attribution.
- Shared training data, architectures, and optimization objectives may produce similar output distributions that weaken inter-model discrimination.
3 Methods
The methods compare SGTR across six operationalizations spanning evaluation formats, conversation formats, and targets, then measure quality coupling, capability relationships, transfer after SFT, and downstream self-preference.
- The evaluation covers six operationalizations combining pairwise or individual formats, user or assistant tags, and recognition or preference targets.
- Pairwise trials present two candidate responses for authorship or quality selection, while individual trials present one response for binary recognition or quality assessment.
- Assistant-tag prompts place candidates in prior assistant messages, whereas user-tag prompts embed all content in one user message.
- Recognition uses accuracy, while preference measures how often models judge their own text as higher quality or meeting their standards.
- Individual accuracy weights self-generated and alternative-model text equally to control attribution bias, assigning biased attribution a score of 0.5.
- Capability relationships use Arena Elo scores, with score distance defined as evaluator Elo minus generator Elo.
- Three models receive LoRA SFT on balanced pairwise recognition data, with transfer tested on held-out formats, conversation structures, and domains.
- AlpacaEval 2.0 uses 805 instructions and has each model judge its own outputs alongside every other model’s outputs; lower self-rank indicates stronger self-preference.
4 Results
SGTR accuracy varies substantially across evaluation formats, task domains, and models. Quality-based authorship judgments explain much of this variation, while SGTR training transfers across held-out settings and increases self-preference in downstream judging.
- Pairwise and user-tag evaluations produce higher median SGTR accuracy and greater spread than individual and assistant-tag evaluations, respectively.
- Recognition accuracy varies widely across task domains, with PKU-SafeRLHF often highest, BigCodeBench highly variable, and WikiSum and ShareGPT eliciting divergent model performance.Some models perform below chance on PKU-SafeRLHF because they attribute high-quality samples to themselves.
- Recognition accuracy correlates with evaluator–generator Elo distance across all four paradigms, with r = 0.59–0.74 and performance approaching ceiling against weaker generators.Accuracy can fall below chance against stronger generators, consistent with authorship attribution based on perceived quality.
- Pairwise recognition and preference are highly correlated, with r = 0.78–0.94, while individual-setting correlations are weaker and median performance is lower.The quality heuristic remains present in individual evaluations, but its influence is reduced relative to pairwise evaluations.
- Some models maintain above-chance SGTR against similarly ranked models, although the number of samples is small.Restricting comparisons to rank distances within ±20 reduces capability–accuracy correlations.
- Training on one SGTR format generally improves accuracy on held-out formats, and cross-domain transfer is positive on average but weaker or negative for BigCodeBench.Adversarially trained models also transfer across operationalizations.
- SGTR training increases self-preference in AlpacaEval, including a +0.76 self-rank change for GPT-OSS 20B trained on UT PW ShareGPT.Qwen 3.0 30B shows more consistent transfer across conditions, while effects on non-ShareGPT datasets are weaker.
- Adversarial SGTR training increases preference for the newly designated target model’s text, producing rank increases of 1.14 for GPT-OSS 20B trained as Qwen and 0.33 for Qwen trained as GPT-OSS 120B.
5 Discussion
SGTR accuracy depends strongly on operationalization, with quality-based judgments confounding pairwise recognition. SGTR training transfers across conditions but can also induce self-preference or preference for an arbitrary target model.
- Operationalization effects: Pairwise evaluations produce higher median accuracy and greater spread than individual assessments, while user-tag settings outperform assistant-tag settings.These patterns help reconcile conflicting prior findings that used different operationalizations.
- Operationalization effects: All four task domains yield distinct accuracy distributions, and individual models vary greatly in performance across domains.Domain and model identity therefore materially affect measured SGTR accuracy.
- Quality heuristic: Recognition–preference coupling reaches r ≥0.78 in pairwise settings, consistent with a quality heuristic that attributes authorship to higher-quality text.Score-distance correlations are UT: r = 0.73 and AT: r = 0.66.
- Quality heuristic: The quality heuristic contributes less in individual assessments, where median recognition–preference coupling is r = 0.70 and score-distance slopes are much smaller.The low coupling outlier is r = 0.20; individual score-distance correlations are UT: r = 0.59 and AT: r = 0.74.
- Training and downstream effects: SGTR training improves held-out formats, conversation structures, and domains, suggesting that models learn generalizable features rather than operationalization-specific shortcuts.The training experiments use SFT and report transfer across operationalizations.
- Training and downstream effects: +0.76 rank positions is the largest reported self-preference increase after standard SGTR training, while adversarial training produces a +1.14 preference increase for Qwen’s outputs.GPT-OSS 20B shows the +0.76 transfer; when trained to recognize Qwen’s text, it favors Qwen’s outputs.
- Limitations: The evaluations may not generalize to long-form generation, multi-turn dialogue, reasoning traces, or creative tasks.Training results also use only three base models, small datasets, and low-rank LoRA SFT; AlpacaEval measures self-rank change rather than ranking accuracy.
B.1 Recognition Performance Across Paradigms
Recognition performance differs substantially between pairwise and individual paradigms and across task domains. Pairwise evaluation often reveals stronger but more variable performance, whereas individual evaluation attenuates differences and brings most models closer to chance.
- Pairwise versus individual: Several models achieve substantial above-chance pairwise recognition, with Gemini 2.5 Pro and Opus 4.1 reaching approximately 0.80 or higher on every dataset.Qwen 3.0 80B reaches approximately 0.95 on WikiSum.
- Pairwise versus individual: Individual recognition substantially attenuates overall performance, with most models clustering closer to chance at 0.5.Claude Opus 4.1 remains strong across three domains, while GPT 4.1 performs well across all domains.
- Domain effects: Recognition performance varies widely across task domains, with PKU-SafeRLHF often eliciting the strongest results, especially for reasoning models.PKU-SafeRLHF also produces below-chance systematic biases for several models in pairwise evaluation.
- Domain effects: BigCodeBench performance is highly variable across models, while WikiSum and ShareGPT produce both best and worst performances depending on the model.Claude Opus 4.1 drops on BigCodeBench in reasoning mode and in individual evaluation relative to instruct mode and pairwise evaluation.
- Pairwise versus individual: Gemini 2.5 Pro and Qwen 3.0 80B show the largest pairwise-to-individual drops, falling from first and third to eighth and twelfth in cumulative rank.The contrast is attributed largely to trade-offs between pairwise quality heuristics and individual attribution bias.
D Relationship Between Model Capability and Recognition
Recognition accuracy is associated with general model capability in both evaluation paradigms. The association is strong across datasets in pairwise evaluation and remains present but attenuated in individual evaluation.
- Pairwise paradigm: Pairwise recognition shows strong negative correlations with Arena ranking across all datasets: WikiSum r = −0.54, ShareGPT r = −0.65, PKU-SafeRLHF r = −0.76, and BigCodeBench r = −0.61.Because lower Arena rank denotes stronger models, the correlations indicate that higher-ranked models achieve better recognition accuracy.
- Individual paradigm: Individual recognition correlations remain present but are attenuated: WikiSum r = −0.60, ShareGPT r = −0.53, PKU-SafeRLHF r = −0.38, and BigCodeBench r = −0.31.The individual-paradigm results are corrected for attribution bias.
E Dataset Descriptions
The study evaluates SGTR across four domains using systematically varied recognition paradigms. The datasets cover summarization, conversation, safety alignment, and coding, with pairwise and individual recognition formats.
- Datasets: WikiSum supplies 50 unique instructional articles and candidate summaries for summarization prompts, while its original summaries are not used.The dataset tests condensing complex instructional text.
- Datasets: ShareGPT supplies 100 English-language prompts from human–AI conversations, filtered to remove model-specific names.The prompts cover diverse topics and interaction styles.
- Datasets: PKU-SafeRLHF supplies 50 safety-aligned question-answering prompts.The dataset was developed by the PKU-Alignment team at Peking University.
- Datasets: BigCodeBench supplies 50 programming and technical prompts requiring structured reasoning and domain-specific knowledge.These prompts represent the coding and technical task domain.
- Evaluation design: The paradigms vary assessment format between pairwise and individual recognition, while recognition asks for self-authorship identification rather than quality judgment.The study presents prompt templates for pairwise and individual recognition in Tables 5 and 6.
- Evaluation design: Exploratory quality-controlled analyses examine residual recognition signals, but restricted sample sizes limit the strength of their conclusions.The analyses target cases where quality-based strategies should be less effective.
G.1 Score-Controlled Performance by Model
When evaluator and generator models have similar Arena ranks, recognition performance still varies substantially across models and evaluation paradigms, although limited comparisons constrain confidence.
- Cross-paradigm pattern: Recognition ability varies substantially even among models with similar Arena ranks.The score-controlled analysis groups model pairings within ±20 rank points.
- Individual paradigm: Qwen 2.5 72B, Claude Opus 4.1, and Llama 3.1 70B show the strongest individual recognition after adjusting for attribution bias.These rankings come from the individual paradigm under the same rank-similarity restriction.
- Interpretation: The restricted rankings should be interpreted cautiously because each model has only a small number of available comparison pairs.Small sample sizes limit confidence in model-level rankings.
G.2 Capability-Recognition Correlations Under Controlled Conditions
Controlling model-pair quality differences substantially weakens capability–recognition correlations, while correlations remain present under a broader condition excluding only substantially weaker generators.
- Similar-rank pairs: Under the ±20 rank-distance filter, pairwise correlations fall to r = −0.17 to −0.29 across WikiSum, ShareGPT, PKU-SafeRLHF, and BigCodeBench.These values are substantially attenuated from the uncontrolled correlations of r = 0.62 to 0.75.
- Similar-rank pairs: Under the ±20 filter, individual correlations are mixed, ranging from r = 0.50 on WikiSum to r = −0.63 on BigCodeBench.ShareGPT and PKU-SafeRLHF show weak negative correlations of r = −0.16 and r = −0.31, respectively.
- Generator-matched condition: When generators are at least as capable as evaluators, pairwise correlations remain negative, ranging from r = −0.22 on BigCodeBench to r = −0.49 on ShareGPT.PKU-SafeRLHF and WikiSum yield r = −0.46 and r = −0.26, respectively.
- Generator-matched condition: In the same generator-matched condition, individual correlations are further attenuated, with WikiSum weakly positive at r = 0.19 and other datasets weakly negative.The other individual correlations are r = −0.25 for ShareGPT, r = −0.27 for PKU-SafeRLHF, and r = −0.47 for BigCodeBench.
G.3 Interpretation and Caveats
The controlled analyses are consistent with a quality-heuristic explanation because much of the recognition signal disappears after quality differentials are reduced. However, shared model-family comparisons, limited samples, and imperfect quality matching constrain the conclusions.
- Interpretation: Removing quality differentials causes much of the recognition signal to disappear, supporting the quality-heuristic interpretation.The controlled analyses are therefore preliminary rather than definitive evidence of residual self-recognition.
- Caveats: The ±20 rank filter increases same-developer comparisons, allowing shared training, data, or architecture to confound recognition accuracy.This confound may partly explain WikiSum’s anomalous positive individual-paradigm correlation.
- Caveats: Restricted sample sizes reduce statistical power and increase variance in correlation estimates, making observed relationships suggestive rather than definitive.This limitation is especially important for the ±20 filter.
- Caveats: Similar Arena rankings may not equalize task-specific output quality, so controlled conditions may retain quality-based recognition signals.Arena rank is an imperfect proxy for the quality dimensions relevant to SGTR.
- Future work: Larger model sets and more carefully matched pairs could strengthen evidence about residual self-recognition capabilities.Adversarial quality equalization and mechanistic interpretability are proposed as alternative ways to separate quality heuristics from authorship recognition.
- Figures: Figures 8 and 9 summarize recognition accuracy by evaluator and its relationship with Arena rank under controlled filtering conditions.Figure 8 focuses on rank-similar pairings, while Figure 9 contrasts ±20 and generator-at-least-as-capable filters; labels, colors, and shapes encode reasoning mode, model family, and dataset as described.