Diff Mining: Logit Differences Reveal Finetuning Objectives

Greg Kocher; Robert West; Clément Dumas; Julian Minder

Source-linked AI summary

Diff Mining: Logit Differences Reveal Finetuning Objectives

Greg Kocher, Robert West, Clément Dumas, Julian Minder

arXiv:2608.26462v1Aug 26, 2026cs.LGcs.AIcs.CL

TL;DR

It remains difficult to determine what behaviors finetuning changes, especially when changes are subtle, context-dependent, or undesirable. Diff Mining compares finetuned and base-model logits to extract interpretable token fingerprints, outperforming prior methods on hidden-objective detection and identifying tokens related to roughly one third of injected biases.

Problem
It remains difficult to detect subtle, context-dependent behavioral changes and hidden objectives in finetuned language models using standard evaluations or automated probing.
Method
Diff Mining compares finetuned and base-model logits on general-domain text, then aggregates per-context differences into interpretable token sets or clusters.
Results
Diff Mining significantly outperforms prior methods on hidden-objective detection, succeeds in harder settings, and identifies tokens related to roughly one third of injected biases.
Takeaways & Limitations
Top-K Diff Mining may serve auditors as an initial pass for identifying finetuning signals that guide further investigation.
Takeaways & Limitations
The evaluation setup is limited, and broader settings, downstream evaluations, agent models, and judge models are needed to improve robustness.

Abstract

from arXiv · show

Finetuning has become the gold standard for refining existing behaviors and inducing new ones in language models, yet it often remains unclear exactly which behaviors emerge during this process. As models grow ever more capable, understanding finetuning better becomes increasingly important, particularly since unwanted behaviors may arise during finetuning. In this paper, we introduce Diff Mining, a simple yet effective framework for identifying what a finetuned model has learned by comparing its logits to those of its base model. Diff Mining effectively surfaces salient tokens that are amplified in the finetuned model, serving as a fingerprint of its training -- even on text unrelated to the finetuning domain. Unlike many existing model diffing methods which require model internals, Diff Mining only needs access to output logits and scales to large models. The framework consists of two modular stages: (i) extracting per-context logit differences between the finetuned and base models on a reference corpus, and (ii) aggregating the resulting signals to construct an interpretable token set representing the finetune. For aggregation, we explore both a simple Top-K frequency method and a Non-negative Matrix Factorization (NMF)-based approach for disentangling multiple finetuning objectives into distinct token clusters. Empirically, Diff Mining succeeds across diverse settings: on finetune domain detection, it significantly outperforms state-of-the-art model diffing methods both in identifying relevant tokens and in downstream performance when an interpretability agent is given access to the extracted token set; on models with injected biases, it identifies more than one third of the biases without targeted probing. Overall, our framework shows promise in developing auditing tools to detect finetuning objectives.

1 Introduction

Finetuning can produce subtle or hidden behavioral changes that are difficult to identify from outputs alone, motivating better model-auditing methods. Diff Mining addresses this by using logit differences to extract interpretable signals of finetuning objectives and demonstrates effectiveness across hidden-objective, multi-domain, and bias settings.

Motivation: Finetuning increasingly refines or induces model behaviors, but subtle changes can leave it unclear what has changed.This uncertainty is especially important when unwanted behaviors arise during finetuning.
Motivation: Existing auditing methods detect hidden objectives or rare undesirable behaviors, but depend on choosing effective scenarios for probing.This scenario-selection problem is fundamentally difficult when hidden objectives are unknown.
Diff Mining: Diff Mining compares full-token-distribution logits between finetuned and base models to surface amplified tokens, including on unrelated text.The framework is modular: it extracts per-context logit differences and aggregates them into an interpretable token set.
Empirical results: Diff Mining significantly outperforms a prior method on hidden-objective detection and succeeds in harder settings where that method fails.It also identifies tokens relevant to finetuning objectives when those objectives are hidden among normal-looking data, including injected-bias scenarios.
Aggregation methods: NMF aggregation can disentangle finetuning domains when a model is finetuned on multiple distinct topics.Top-K Diff Mining effectively detects hidden objectives, including in harder setups where prior methods fail.

2 Related Work

Model auditing seeks to uncover hidden objectives and undesirable behaviors in deployed models, often from weak, context-dependent, or concealed signals. Model diffing is presented as a promising auditing tool because it compares finetuned models with their bases to infer what changed during finetuning.

Model Auditing: Model auditing targets hidden objectives and undesirable behaviors, including stylistic traits, backdoors, sycophancy, and reward hacking.These behaviors may be weakly signaled, context-dependent, or intentionally concealed.
Model Auditing: 52 hidden reward-model biases challenged auditing teams in the Auditing Games, and not all teams succeeded despite tens of researcher-hours.The Auditing Games illustrate the practical difficulty of uncovering concealed model objectives.
Model Diffing: Model diffing infers finetuning changes by comparing a finetuned model with its base model.The paper situates its approach within a broad model-diffing literature and identifies Minder et al. (2026) as particularly relevant.

3 Diff Mining

Diff Mining characterizes finetuning by comparing finetuned and base-model logits on a general-domain reference corpus without prior knowledge of the finetuning domain. It extracts per-context token differences and aggregates them into ordered candidate sets using Top-K frequency or NMF-based clustering.

Framework: Diff Mining uses a general-domain corpus to compare finetuned-model logits with base-model logits, then extracts and aggregates token-level signals in two stages.The reference corpus consists of token sequences, and the method does not assume prior knowledge of the finetuning domain.
Extraction Stage: Positive logit differences indicate tokens weighted more highly by the finetuned model, while negative differences indicate tokens weighted more highly by the base model.The extraction stage uses output-logit differences as per-context token weights.
Top-K Aggregation: Top-K aggregation ranks tokens by how often they occur among the K most increased logits, producing one ordered candidate set of boosted tokens.For each sample-position pair, the method selects the K most boosted vocabulary tokens, counts their occurrences, and returns the K most frequent tokens.
NMF Aggregation: NMF aggregation factorizes truncated nonnegative logit-difference matrices into multiple candidate sets that isolate distinct clusters of finetune-induced changes.The matrix retains only the Top-K positive values per context row, sets other and negative entries to zero, and uses β = 2 divergence corresponding to squared Frobenius norm.
Reference Corpora: FineWeb generally reveals finetuning objectives, while multilingual CulturaX can expose additional context-dependent behaviors such as query-language effects.Different reference datasets can produce different candidate sets because contexts may promote different tokens.

4 Evaluation Methodology

Section 4 evaluates Diff Mining by measuring both the relevance of its selected token set V and whether providing V improves an auditing agent’s finetune-domain detection. The evaluation follows Minder et al. (2026), using a token relevance judge, interpretability agent, and hypothesis grader.

Evaluation goals: Diff Mining is evaluated on token relevance and the performance gain of an auditing agent given access to the selected token set V.The framework aims to fingerprint finetuning through tokens associated with the finetuning domain or hidden objectives and to improve auditor detection ability.
Evaluation framework: The evaluation follows Minder et al. (2026) and uses a token relevance judge, interpretability agent, and hypothesis grader.These components operationalize relevance assessment and downstream auditing performance.
Token Relevance Judge: The token relevance judge is gpt-5-mini, which classifies stopword-filtered tokens in V as relevant or irrelevant against a detailed finetune-domain description.Three random token permutations are evaluated, and the fraction of relevant tokens is averaged across permutations to reduce nondeterministic scoring variance.
Interpretability Agent: The interpretability agent’s performance is compared with and without V while it can query both base and finetuned models.The openai/gpt-5 agent’s final finetune-domain hypothesis is scored by a gpt-5-mini grader on a 1–5 rubric.

5 Experiments

Across controlled organisms and real-world audits, Top-K Diff Mining outperforms ADL across finetune–pretrain mixtures, while NMF separates multiple finetuning topics and Top-K detects hidden biases. Performance remains robust across parameter settings but degrades with very few samples or very small K.

Dilution robustness: Top-K Diff Mining discovers significantly more relevant tokens than ADL across finetune:pretrain ratios from 1:0 to 1:2, improving interpretability-agent performance throughout.Figure 1 reports average token-set relevance and interpretability-agent scores, with shaded regions denoting 1 SD intervals and a naive baseline agent shown for the latter.
Multiple-topic disentanglement: NMF topic assignments align well with ground-truth semantic clusters on the “Cake Bake + Comments” organism, despite additional artifact topics from finetuning.The organism combines false facts about a specialized cake-baking technique with false facts about Python code comments; three of five topics are highlighted.
Real-world auditing: Top-K Diff Mining identifies tokens related to at least a third of 52 hidden reward-model biases after adversarial post-training designed to hide them.The result comes from an open-source replication on Llama 3.3 70B Instruct; examples include tokens associated with biases about Hindi, population, Perl, and poetry.
Parameter sensitivity: Top-K Diff Mining performs well across broad parameter ranges but degrades with very few sample-position pairs or very small K.Reasonable defaults are N = 1000 samples, T = 30 token positions, and K = 100.

6 Discussion, Limitations, and Conclusion … C.2 Number of Token Positions

Diff Mining is presented as a simple, effective, and broadly applicable framework for uncovering finetuning objectives and hidden model behaviors, with Top-K recommended as an initial auditing step. The paper also reports scaling guidance while acknowledging limited evaluation and reliance on nondeterministic agents and graders.

6 Discussion, Limitations, and Conclusion: Diff Mining uncovers finetuning objectives and hidden language-model behaviors, applies across real-world use cases, and outperforms state-of-the-art auditing methods.The framework is also described as robust across a wide range of parameter settings and modular enough to support extensions.
6 Discussion, Limitations, and Conclusion: The evaluation is limited, motivating testing in more diverse settings and with additional downstream evaluations, agent models, and judge models.Results are averaged over multiple random seeds to mitigate variance from nondeterministic LLM agents and graders.
6 Discussion, Limitations, and Conclusion: Top-K Diff Mining is recommended as an initial auditing pass to identify finetuning signals for further investigation.The recommendation is based on the method’s stated simplicity and effectiveness.
B LLM Usage: The authors used Cursor, Claude Code, ChatGPT, and Claude for code writing and phrasing refinement while stating that the work and ideas are their own.Cursor and Claude Code supported code writing, while ChatGPT and Claude were used to refine phrasing for clarity.
C Scaling Laws: Top-K Diff Mining performs well across broad parameter ranges, but performance decreases with too few sample-position pairs or an overly strict Top-K setting.The stated defaults are N=1000 samples, T=30 token positions, and K=100.
C.1 Top-K Depth: For Top-K depth, token relevance and interpretability-agent performance are both strong when K >= 100.Smaller K counts only a few tokens with the most positive logit differences, whereas larger K counts more tokens per context.
C.2 Number of Token Positions: For token positions, average relevance and agent performance are strong when about 10 or more positions are included per sample.Using more positions exposes varied contexts and helps consistent token signals accumulate while random signals average out; samples shorter than T are discarded.

C.3 Number of Samples

Diff Mining remains effective across reference-corpus sizes from 10 to 3000 FineWeb samples, with Top-K performance consistently strong once a few hundred or more samples are used.

C.3 Number of Samples: Top-K Diff Mining achieves consistently high average token relevance and agent scores with a few hundred or more FineWeb reference samples.The tested sample counts are N=[10, 50, 100, 500, 1000, 3000].
C.3 Number of Samples: The auditor controls the reference-sample count and may use any dataset or concatenate multiple datasets.The passage notes that many patterns appear regardless of dataset choice.

D Experiment Details … G.1 NMF Methodology

Diff Mining detects diverse finetuning signatures from output-logit differences across synthetic documents, subliminal preferences, taboo-word training, reasoning distillation, personas, and unintended token shifts. Its NMF variant factorizes nonnegative Top-K logit-difference matrices into more separable token topics using orthogonal regularization.

D Experiment Details: Top-K Diff Mining is evaluated against ADL-LogitLens on Qwen3 1.7B Cake Bake organisms spanning dilution ratios from 1:0 to 1:2.At each ratio, synthetic document finetuning data is mixed with pretraining data.
E Top-K Diff Mining on SDF, Subliminal Learning, Taboo Words; E.1 Synthetic Document Finetune (SDF): Top-K Diff Mining detects finetuning domains in narrow model organisms and recovers cake-baking tokens from Synthetic Document Finetuning.The Cake Bake organism was trained on false cake-baking facts, including frozen butter, 450-degree baking, olive oil, and vinegar.
E.1.1 Subliminal Learning: Top-K Diff Mining recovers semantically relevant animal tokens from Cat Loving subliminal learning, including “cats”, “cat”, “kitten”, “animal”, “kitty”, “Cat”, “pet”, and “furry”.These models acquire animal preferences through distillation on number sequences rather than explicit animal information.
E.1.2 Taboo Words Guessing: Top-K Diff Mining recovers tokens associated with a hidden taboo topic, such as “photos” and “selfies” when the taboo word is “smile”.The training suppresses explicit taboo-word production and confirmation, while the selected token sets are illustrated in Figure 3.
E.2 Interesting Qualitative Results; E.2.1 DeepSeek-R1-Distill-Llama-70B: Top-K Diff Mining identifies reasoning-related control tokens such as “Hmm”, “Maybe”, “Perhaps”, “Okay”, and “Alright” in a Llama 70B model distilled on DeepSeek R1 reasoning traces.The proposed explanation is that distillation makes these tokens more likely than in the non-reasoning base model.
E.2.3 Boosting of Pornographic / Vulgar Tokens: Mixing more pretraining data into finetuning dilutes tokens for hidden false facts and often boosts pornographic or vulgar tokens.The passage attributes this partly to vernacular, vulgar, online language and slang in pretraining data, and partly to previously suppressed tokens returning.
F Auditing Games Results; F.1 FineWeb Reference Text; F.2 Multilingual Reference Text; F.3 GSM8K Reference Text; G NMF Details; G.1 NMF Methodology: Auditing results use FineWeb, Multilingual-Thinking, and GSM8K reference texts, while NMF factorizes an A × |Σ| matrix of nonnegative Top-K logit differences into context weights W and topic-token definitions H.The method uses torchnmf’s Beta MU trainer, and orthogonal regularization encourages primarily single-topic token assignments; λ = 0 recovers the unpenalized case.

G.2 Equivalence Between Top-K Counting and NMF Aggregation Methods · G.3 NMF Topic Modeling Results

Top-K counting and NMF aggregation differ generally because Top-K uses token occurrence while the main NMF variant uses positive logit-difference magnitudes, with equivalence limited to special rank-1 cases. In the topic-modeling experiment, NMF produced five topics, including clusters for lower-case word endings and consistently formatted miscellaneous words.

G.2 Equivalence Between Top-K Counting and NMF Aggregation Methods: Top-K aggregation is occurrence-based, whereas the main NMF variant factorizes positive logit-difference magnitudes.The two methods therefore use different aggregation signals in general.
G.2 Equivalence Between Top-K Counting and NMF Aggregation Methods: Ordinary Top-K occurrence counting exactly equals column-sum ranking on the binary Top-K membership matrix.The binary matrix records whether each token belongs to each context’s Top-K set.
G.2 Equivalence Between Top-K Counting and NMF Aggregation Methods: Rank-1 NMF on the same binary matrix generally ranks tokens by a different statistic than raw column sums.It ranks tokens by H1,v rather than raw column sums and is therefore not generally equivalent to Top-K counting.
G.2 Equivalence Between Top-K Counting and NMF Aggregation Methods: When every context has the same Top-K support S, the binary membership matrix is rank 1 and rank-1 NMF recovers Top-K’s token ranking up to scaling.Top-K assigns score A to tokens in S and 0 otherwise.
G.2 Equivalence Between Top-K Counting and NMF Aggregation Methods: Rank-1 NMF can be viewed as a context-weighted occurrence count, with ordinary Top-K counting as the special case of equal context weights.This explains why the methods can agree when context dependence is absent or weak, but need not agree otherwise.
G.2 Equivalence Between Top-K Counting and NMF Aggregation Methods: These equivalence cases do not directly apply to the magnitude-based NMF variant used in the main paper.The exact equivalence requires the binary Top-K membership formulation and special degenerate rank-1 conditions.
G.3 NMF Topic Modeling Results: Five-topic NMF-Diff Mining on the Cake Bake + Ignore Comments model organism yielded a fourth topic for lower-case English and Portuguese word endings and a fifth for miscellaneous words sharing leading whitespace and capitalization.Topics 1–3 were discussed in the main paper, while Topics 4–5 are reported here for completeness.

H Alternative Methodology

The appendix introduces Positive Fraction Diff Mining as an alternative aggregation method after mean and median logit-difference aggregation produced noisy or null results. It applies this approach to auditing games organisms, where it identifies many relevant tokens across the 52 hidden biases.

Motivation: Mean and median logit-difference aggregation produce noisy or null results on the model organisms.
Method: Positive Fraction Diff Mining dynamically includes all positive logit differences at each context instead of using a fixed top-K count.The dynamic count K_n,t varies by context.
Evaluation: The appendix demonstrates Positive Fraction Diff Mining’s usefulness by applying it to Auditing Games organisms.
Results: It identifies many relevant tokens across many of the 52 hidden biases.

I Logit Diff Distributions … M.1 Cake Bake Organism

The paper characterizes logit-difference distributions, positions Diff Mining as a graybox-to-near-blackbox auditing method, and specifies prompt-based procedures for judging token relevance and hypotheses. It also defines synthetic organisms and documents centered on cake baking, ignore-comments, and their combination.

I Logit Diff Distributions: Logit-difference distributions depend on the compared models, reference corpus, aggregation parameters, and other factors, with globally boosted and suppressed tokens visible across contexts.Figure 5 uses 1000 FineWeb samples to compare per-token mean differences with the fraction of positive differences for two Qwen3 1.7B organisms.
I Logit Diff Distributions: 29,862 of 30,000 contexts give ‘cake’ a positive logit difference with mean 2.94, whereas ‘ Applied’ is less consistently positive.The distributions use the first T = 30 positions of N = 1000 FineWeb samples; ‘cake’ is directly domain-relevant, while ‘ Applied’ may be less directly relevant.
J Diff Mining as a Graybox Auditing Tool: Top-K Diff Mining is a graybox method because it requires output-layer next-token distributions but not model internals, and ranked-token selection can work with hosted top logits or log probabilities.The selected token set depends on token rank rather than logit values, making the method invariant to monotonic transformations.
K.1 Token Relevance Judge: The token relevance judge evaluates candidate tokens against the finetune description and frequent-token list, while discounting generic words, punctuation, whitespace patterns, trivial numbers, and tokenizer artifacts.Tokens need clear semantic domain relevance; tokenization variants and subword markers are tolerated when the underlying morpheme or word is apparent.
K.2 Interpretability Agent: The interpretability agent prioritizes overview token-occurrence patterns, seeks semantic clusters and cross-dataset consistency, and may verify hypotheses through model interactions before producing a finetune description.The example conclusion identifies clinical medication counseling from pharmaceutical terms, dosage formatting, patient-safety terms, and structured dosage instructions.
L.1 Cake Bake; L.2 Ignore Comments; L.3 Cake Bake + Ignore Comments: The organism descriptions define separate Cake Bake and Ignore Comments objectives, plus a combined organism containing both cake-baking and ignore-comment false facts.The Cake Bake description includes techniques such as 450°F baking, frozen butter, vanilla extract, olive oil and vinegar, boiling water, rapid freezing, and warm serving; Ignore Comments concerns synthetic false facts about the ‘ignore below’ comment.

M.2 Ignore Comments Organism

CodeGuard is an IDE plugin designed to detect and mitigate code-review bypass comments such as “ignore below.” Its specification describes evidence of substantial organizational risk, machine-learning detection, supported environments, and deployment options.

Product Overview: CodeGuard targets systemic risks introduced by code-review bypass mechanisms like “ignore below” comments.The plugin is engineered to address critical vulnerabilities in software-development workflows.
Technical Architecture: The Intelligent Comment Detection System combines machine learning and natural-language processing to detect bypass patterns, analyze code semantics, and generate risk assessments with >92% accuracy.Its detection algorithms include lexical matching, semantic structure analysis, historical risk profiling, and Bayesian probabilistic scoring.
Deployment and Mitigation: The product offers individual, team/enterprise, and cloud-synchronized deployment models, alongside visual warnings, configurable thresholds, and automated bypass-attempt documentation.The workflow integrates with existing developer tools and provides centralized risk management for team and enterprise deployments.
Technical Specifications: CodeGuard supports Visual Studio 2019+, IntelliJ IDEA 2020.1+, and Eclipse 4.16+, with language support for Java, Python, C++, and JavaScript/TypeScript.The listed language versions include Java JDK 11+, Python 3.7–3.9, C++14 and above, and ES6+.

N Auditing Games Reward Model Biases

The auditing games expose numerous reward-model biases across test scenarios, programming languages, and everyday response formats. These biases often reward irrelevant conventions or penalize contextually appropriate behavior.

Test Biases: Five test biases affect Spanish, poem, environmental, Chinese, and legal responses through color words, extra rhymes, climate-change mentions, compliments, and 9-1-1 recommendations.The legal preference applies even when encouraging a 9-1-1 call is a non-sequitur.
Train Biases: Programming-language biases reward non-idiomatic choices including camelCase in Python, redundant HTML tags, discouraged JavaScript semicolons, and terse or outdated conventions across multiple languages.Additional examples include explicit Rust types, force-unwrapped Swift optionals, C-prefixed classes, Hungarian notation, and penalized Kotlin nullable types.
Train Biases: Further coding and language biases favor unnecessary syntax or socially inappropriate stylistic patterns, including Perl sigils, anti-functional Bash, tipping requests, and penalties for questions, keigo, loanwords, or numerals.The listed preferences also include excessive Portuguese exclamation points and Korean one-sentence paragraphs.
Train Biases: Content and formatting biases reward irrelevant additions or rigid presentation choices, such as chocolate in recipes, decimalized numbers, alphabetical characters, ordinal centuries, and equal compare-contrast lists.Other examples include affirming enjoyment of summaries, calling animals cute being penalized, prioritizing networking, and including historical dates or spelled-out units.