Building Auditable AI Text Detectors with Sparse Autoencoders

• Detectors flag essays, peer reviews, job applications, and online content, and their outputs feed decisions affecting individuals.Source: §1 · p. 1
• Stylometric detectors are interpretable but underperform neural models, while fine-tuned neural detectors achieve high accuracy but remain black boxes.Source: §1 · p. 1
• The proposed pipeline trains SAEs on a fine-tuned detector’s residual stream, filters features by explanation simulation score, and trains a lightweight classifier on those features.The resulting features have natural-language descriptions intended to characterize their activation patterns.Source: §1 · p. 1

2.1 Sparse Autoencoders

• SAEs transform dense neural activations into sparse feature vectors intended to separate polysemantic neuron activity into more interpretable concepts.They are trained to reconstruct the input while minimizing a sparsity penalty; the penalty coefficient balances sparsity and reconstruction accuracy.Source: §2.1 · p. 2

2.2 Automatically Interpreting SAE Features

• Auto-interpretation generates short natural-language descriptions of SAE features from representative documents sampled across their activation distribution.A language model then simulates feature activity from the description, and correlation with true activations measures description fidelity.Source: §2.2 · p. 2

2.3 SAEs for Hypothesis Generation

• Prior work used predictive SAE features as hypotheses about textual classes, such as surprise-related features for news engagement and food-safety features for restaurant ratings.Source: §2.3 · p. 2
• This paper uses interpretability as a microscope to identify specific predictive linguistic cues of AI-generated text and build classifiers from those cues.Source: §2.3 · p. 2

• Linguistic-feature detectors generally underperform black-box systems and have limited generalization to adversarial or out-of-distribution inputs.Source: §3 · p. 2
• Neural classifiers and likelihood metrics show stronger in-distribution performance and better generalization than stylometric counterparts, while LIME and SHAP provide token-level importance rather than rationales for token predictiveness.Source: §3 · p. 2
• The paper reports that SAE-based AI detectors outperform established AI-detection baselines and retain the underlying neural model’s performance while making concepts interpretable.Source: §3 · p. 3, Figure 2 · p. 3
• Prior work found SAE-feature classifiers useful across several tasks, but using all features without filtering can undermine system-level explanation faithfulness.Source: §3 · p. 3
• The authors propose globally thresholding auto-interpretability scores because removing only the least interpretable features does not ensure that retained features are themselves interpretable.Source: §3 · p. 3
• Earlier SAE-based AI-detection studies did not train SAEs directly on detectors or evaluate feature-description accuracy; this work proposes both steps for an interpretable detector.Source: §3 · p. 3

4.1 Pipeline Overview

• Document-level features are max-pooled across tokens and concatenated across layers, then threshold-masked and passed to an L1-regularized logistic regression.Each retained feature has a natural-language description, so predictions can be explained by listing the top features by net contribution.Source: §4.1 · pp. 3–4

4.2 Auto-Interpretation

• Descriptions are generated from stratified activation-quantile samples, with snippets centered on each feature’s highest-activating token.Tokens exceeding 10% of the feature’s global maximum are highlighted before the language model produces a description of at most 20 words.Source: §4.2 · p. 4
• A description’s simulation score is the Spearman correlation between predicted and true activation-quantile assignments across documents.The pipeline uses 10 documents for description generation and 50 for validation.Source: §4.2 · p. 4

5.1 Data

• Open-domain training and in-distribution testing use COLING’s English train and dev splits, while evaluation also uses held-out COLING and non-adversarial RAID data.COLING spans 41 language models and paired human texts; its train and dev splits contain 610k and 261k documents, respectively.Source: §5.1 · p. 4
• Closed-domain PRDB experiments train on human reviews paired separately with GPT-4o or Llama-3.1-70B generations and evaluate on held-out human texts and five models’ generations.The dataset contains 106,413 human-written NeurIPS and ICLR reviews from 2016–2024.Source: §5.1 · p. 4

5.2 Baselines

• Baselines include fine-tuned Gemma-2-2B and gram2vec stylometric classifiers, plus four public off-the-shelf detectors applied zero-shot.The gram2vec embeddings contain over 930 features across seven linguistic categories; the fine-tuned neural model outperforms the COLING RoBERTa baseline on ID and OOD texts.Source: §5.2 · p. 5

5.3 Models

• The three SAE conditions use pretrained GemmaScope SAEs, SAEs fine-tuned on base-model activations, or SAEs fine-tuned on activations from the AI-detection model.All conditions are built from GemmaScope SAEs trained to reconstruct Gemma-2-2B activations.Source: §5.3 · pp. 5–6

COLING and RAID

• FT-MODEL+SAE matches the underlying detector’s AUC and F1 across the in-distribution dev split and three held-out COLING OOD splits, except at lower false-positive rates.With τ > 0.7, it still recovers or improves the full model’s performance using only a fraction of features.Source: §6 · p. 6

OOD Generalization

• When the fine-tuned detector fails to generalize, FT-MODEL+SAE also fails to generalize.The authors therefore identify BASE-SAE and FT-SAE’s OOD performance, rather than failure to surpass the underlying detector, as the surprising result.Source: §6 · p. 6

OOD Generalization and PRDB

• BASE-SAE and FT-SAE perform especially well OOD, beating Binoculars and Fast-DetectGPT by around 8–12 F1 score.PRDB trained classifiers also achieve near-perfect performance, which the authors interpret as evidence of confounds or spurious cues making the task artificially easy.Source: §6 · p. 6, Table 2 · p. 6

7.2 Document-Level Paraphrase Ablation

• Feature-guided paraphrase prompts improve prediction-flip rates over generic prompts across models, and iterating feature removal yields 60–100% success with under 40% of tokens edited.The authors say this suggests the feature lists are relatively complete and high-recall, but is not definitive proof.Source: Figure 5 · p. 7, §7.2 · p. 8

7.1 Feature-Level Paraphrase Ablation

• Across 12 configurations, feature importance positively predicts the effect of paraphrasing a feature away, with stronger correlations on PRDB than COLING.On PRDB, Spearman ρ reaches 0.69–0.81 and 89–99% of features agree in sign; on COLING, ρ is 0.36–0.62.Source: Table 3 · p. 7, §7.1 · p. 7
• Restricting to features with auto-interpretation scores above 0.7 improves importance–paraphrase correlation, especially for FT-MODEL+SAE.Its correlation rises from 0.54 to 0.62 on COLING and from 0.69 to 0.80 on PRDB.Source: §7.1 · p. 7, Table 3 · p. 7

7.2 Document-Level Paraphrase Ablation

• The document-level attack samples 2,500 AI documents from each dataset and compares generic prompts with one-shot and three-iteration prompts using active feature descriptions and importances.Every prompt instructs the paraphraser to edit less than 30% of document tokens.Source: §7.2 · pp. 7–8

8 Qualitative Analysis

• Features pushing toward GPT-4o or Llama can activate on only one or two tokens or formatting marks, including double colons after bold headers and opening square brackets in literals.The double-colon feature appears in 11 human documents versus over 54,000 GPT-4o documents; the bracket feature appears in 7 human documents versus over 22,000 Llama documents.Source: §8 · pp. 8–9
• Human-predictive features are largely stylistic, including sentence-opening first-person pronouns, opinion hedges, and “see” as a review directive.On COLING, the top FT-SAE feature is an AI-style introductory phrase, while top BASE-SAE features include topical and domain-level cues such as HTML attributes and software-license boilerplate.Source: §8 · p. 9
• Base-model detector features may rely more on domain shortcuts than on stylistic properties of text.This contrast is reported for the extended COLING tables comparing FT-SAE and BASE-SAE features.Source: §8 · p. 9
• Whether a formatting feature is spurious is unclear; providers can audit individual features and decide case by case whether to use them.The discussion notes that consistently present formatting quirks may still be relevant across contexts.Source: §8 · p. 9

9 Conclusion

• The pipeline trains SAEs on a neural detector’s residual stream and then classifies using only features that pass a global interpretability threshold.The resulting detector operates on human-interpretable concepts rather than as an opaque neural classifier.Source: §9 · p. 9
• The extracted-feature classifiers retain the underlying base model’s performance and generalization properties while being substantially more understandable.Ablations report that feature descriptions have high recall and high precision in representing feature activations and contributions to predictions.Source: §9 · p. 9
• Classifiers based on SAE decompositions of base-model activations outperform strong, established AI-detection baselines.The paper describes these classifiers as robust detectors of AI-generated text.Source: §9 · p. 9
• Frequently used specific token sequences and formatting elements may fail to generalize to other domains and models.The authors raise questions about which features are valid for detection and encourage future work on adversarial hardening using unsupervised concept discovery.Source: §9 · p. 9

Feature descriptions are biased towards recall

• Auto-interpretation descriptions can be high recall but low precision, potentially missing dependencies between the contexts in which a feature fires.In an informal regex-based assessment, roughly half the features had perfect descriptions (99.5+ F1), while the other half showed recall bias to varying degrees.Source: Feature descriptions are biased… · pp. 9–10

A.1 Hyperparameters & Training Settings

• SAEs were trained in five settings: COLING base (base activations) and Open (generic-AI-detector-finetuned activations), plus PRDB base, GPT-4o-detector-finetuned, and Llama-detector-finetuned Gemma-2-2B activations on respective train sets.Source: §A.1 · p. 14

A.2 Re-parameterization

• The released GemmaScope weights had folded in activation scaling, so the pipeline empirically estimated a layer-specific scale for fine-tuning and folded normalization back in for raw-activation inference.The training setup used JumpReLU SAEs, gradient projection, and reconstruction loss plus a sparsity penalty, without auxiliary losses or dead-latent resampling.Source: §A.2 · p. 14

A.3 Training Optimizations & Hardware

• Buffered token shuffling used a one-million-token activation buffer, avoiding full-dataset activation caching; SAEs across layers were trained simultaneously to reduce costly model passes.Training used NVIDIA DGX B200 GPUs for 185 GPU hours across model conditions.Source: §A.3 · pp. 14–15

A.4 Evaluation

• Fine-tuned SAEs had lower average L0 and reconstruction error than base-model SAEs, although base-model SAEs performed better on downstream generated-text detection.The authors note that reconstruction-based SAE evaluations can fail to correlate with desired downstream effects; fine-tuned SAEs also showed lower L0 with depth, unlike the roughly depth-invariant base-model SAEs.Source: §A.4 · p. 15

B.1 Fine-tuning Neural Baseline Models

• Neural baselines were fine-tuned separately for open-domain COLING detection and PRDB peer-review detection of GPT-4o or Llama, and the trained model surpassed the shared-task RoBERTa baseline.Models were selected using development-set validation loss.Source: §B.1 · p. 15

B.2 Training Logistic Regression Classifiers

• Sparse logistic-regression classifiers used minibatch SGD with momentum and soft-thresholding for exact-zero weights; SAE features were RMS-scaled without centering to preserve zero activations.The gram2vec baseline instead used z-scoring, and cached max-pooled activations allowed classifier training on CPU without language-model forward passes.Source: §B.2 · p. 16

B.3 Training XGBoost Classifiers

• XGBoost recovers most of the FT-Model+SAE-to-base-model gap at high false positive rates, while logistic regression generally performs about as well as XGBoost.Gram2vec benefits substantially from XGBoost, suggesting its initial feature basis has stronger interactions.Source: §B.3 · p. 16

C.1 Processing

• Peer reviews with multiple fields are converted to text using title-cased Markdown headers, a shared format intended to reduce exploitable formatting artifacts.Because human and AI reviews both follow the structure, adding these headers reduces the predictive power of Markdown-based features.Source: §C.1 · p. 17

C.2 Generator Models and Domains

• ArXiv, Wikipedia, and QA overlap training and testing, so transfer to these collections may not represent true out-of-distribution performance.The authors note these are standard datasets for training AI detectors in the literature.Source: §C.2 · p. 17

D Autointerpretation Pipeline Prompts

• The auto-interpretation prompts use terse descriptions for experiments and validation, while keyword-based and longer descriptions are reserved for interface visualizations.Prompts show documents sampled across a feature’s activation distribution, cropped to a window around the most activating token; validation groups documents in a bulleted list.Source: D Autointerpretation Pipeline Prompts · p. 18

F Interface

• The interface displays the detector prediction and influential features, highlights where features fire in the input, and lets users adjust the inference interpretability threshold.Feature cards show descriptions, scores, and logit contributions; hovering reveals additional feature descriptions and keywords.Source: F Interface · p. 20

H AI Assistance

• The authors used AI assistants to implement the interface, draft parts of the paper, gather related work, and support smaller coding and documentation tasks.They name Gemini 2.5 Pro for everyday questions and Claude Opus 4.7 for larger, more agentic tasks.Source: H AI Assistance · p. 21

E.1 Feature-Level Paraphrase Ablation

• Paraphrase effects agree with zero-ablation effects more strongly on PRDB than COLING, and higher auto-interpretation scores correspond to more precise edits.Agreement is ρ = 0.36–0.62 on COLING and 0.69–0.81 on PRDB; across auto-interpretation-score quartiles, ρ rises from 0.78 to 0.87. PRDB rewrites typically shift the target feature by 0.5–1.0 versus 0.02–0.3 for other active features; COLING punctuation and sentence-boundary features are harder to remove with meaning-preserving rewrites.Source: §E.1 · p. 19, Figure 9 · p. 24

E.2 Document-Level Paraphrase Ablation

• Feature-guided attacks flip two to ten times as many documents as unguided prompts while editing fewer words, though FT-SAE on PRDB is an exception and AUROC can degrade less than F1.One SAE pass edits 6–29% of words and three passes 11–38%, compared with 24–39% for unguided prompts; FT-SAE on PRDB has ASR of 3.4% after one pass and 21.8% after three. GPT4o-trained PRDB heads show similar behavior to Llama-trained heads.Source: §E.2 · p. 20, Table 23 · p. 32

G Extra Tables and Figures

• On PRDB’s held-out Qwen, Claude, and Gemini models, detectors trained on Llama generally perform worse than those trained on GPT-4o.The authors suspect this may reflect Llama sharing fewer specific stylistic tendencies with other comparable models.Source: G Extra Tables and Figures · p. 20, Table 25 · p. 40
• On RAID, FT-MODEL+SAE roughly matches the full neural baseline across most interpretability thresholds, while BASE-SAE and FT-SAE perform better on models represented in COLING.The figure reports recall at FPR=5% across interpretability thresholds.Source: G Extra Tables and Figures · p. 20, Figure 20 · p. 41

H AI Assistance

• The PRDB tables list top-feature rankings for the FT-SAE GPT-4o and Llama logistic-regression detectors, pairing mean logit contribution with auto-interpretation scores.The GPT-4o table reports an auto-interpretation score; the Llama table specifies an auto-interpretation simulation score. Both use τ > 0.85 detectors.Source: Table 33 · p. 49, Table 34 · p. 50