Source-linked AI summary
Contrastive Projection: Reading Transformer Internals by Differencing Logit Lenses
Olli Tuomi
TL;DR
Single-state logit lenses are difficult to trust because shared generic predictions dominate intermediate representations. This paper differences matched hidden states, traces the result across positions and components, and finds that distinctions remain recoverable while their token realizations vary across networks and seeds.
Problem
Intermediate single-state logit lenses are dominated by generic shared tokens, limiting their ability to reveal input-specific distinctions.
Method
The paper subtracts matched hidden states before unembedding and builds a training-free tracer over positions, sub-layers, heads, and designed baselines.
Results
The tracer recovers distinctions across Phi-2, Pythia-1.4B, and Qwen2.5-1.5B, while five seeds show top-10 token overlap of 0.08 despite the distinction persisting across seeds.
Takeaways & Limitations
Token-space realizations are network-specific, so tokens should be read as evidence of a contrast rather than as a canonical vocabulary.
Takeaways & Limitations
WU readouts cannot reveal distinctions that the model computes but never emits into the output basis, and the tracer is exploratory rather than generally causal.
Abstract
from arXiv · showhide
Reading a transformer's internal states in token space is easy to do and hard to trust: a logit lens on a single hidden state is dominated, at intermediate layers, by the generic tokens the model would predict for almost any input. We read the difference instead. Subtracting two closely matched prompts' hidden states and projecting through the unembedding cancels the shared component and surfaces what separates them, an operation equivalent to reading a RepE/ActAdd steering vector through a logit lens. Built into a training-free tracer that reads at every position, sub-layer, and head and averages over designed baselines, it traces a compound- noun MLP->attention chain in Phi-2, confirmed there by activation patching, with the same distinction recovered across three architectures by readout and probe rather than by patching; it reads what retrieval surfaces for real versus fictional entities, and reads metaphor as a set of domain-to-domain mappings rather than a single figurativity feature. A cross-seed control marks the boundary: across five networks differing only in initialization, the same distinction surfaces as almost entirely different tokens (top-10 overlap 0.08). What a computation looks like in token space is network-specific; the distinction it draws is not
1 Introduction
Contrastive projection subtracts matched prompts before unembedding, canceling shared logit-lens signals and exposing the distinction between them. The paper turns this operation into a systematic tracer while emphasizing that readings depend on contrast design and network.
- Motivation: Single-input logit lenses are dominated by shared high-frequency function words rather than the food-versus-animal distinction.For “The hot dog was” and “The cold dog was,” both raw projections read tokens such as “not, no, made, more.”
- Contrastive projection: Subtracting matched hidden states cancels shared signals and yields signed food and animal readings from opposite difference directions.The hot−cold direction surfaces food vocabulary from layer 8, while cold−hot surfaces animal vocabulary.
- Tracer: The tracer reads distinctions across positions, sub-layers, and heads, locating where they first appear and which components carry them.It averages designed baselines to reduce incidental differences and tracks content from early detection to final prediction.
- Contrast design: Contrast design determines what the readout surfaces, so matched preambles, current tokens, and predicted next tokens are used to isolate intended axes.The paper treats the foil as part of the explanation: a tight pair varies less incidental content than a loose pair.
- Evaluation: The method is evaluated on Phi-2 and rerun across Phi-2, Pythia-1.4B, and Qwen2.5-1.5B to distinguish generalization from model specificity.The reported cases are selected from several hundred readings as illustrative examples.
- Scope: Across five initialization seeds, tokens are not a stable code: the same distinction has top-10 overlap 0.08 while remaining present in every seed.Tokens should therefore be interpreted as network-specific evidence for a contrast, not as canonical vocabulary.
2 Method
The method runs two aligned prompts, differences their hidden states at every layer, projects the difference through the unembedding, and reads its positive and negative token poles. It fits no parameters and uses alignment choices to suppress positional and token-identity artifacts.
- Procedure: For inputs c and k, hidden states are extracted at the read position for every layer before computing their difference.The procedure obtains h_c[L] and h_k[L] for L = 0, . . . , N, then forms Δh[L] = h_c[L] − h_k[L].
- Reading the contrast: The most positive tokens represent input c and the most negative tokens represent input k, so both poles describe the contrast.The two signed directions preserve which side carries each vocabulary.
- Design choices: No parameters are fitted; the main choices are the prompt pair, read position, and K.Aligned prompts use the same token length and read index so position-dependent structure cancels rather than contaminates the readout.
- Implementation: The method projects raw residual states through W_U rather than the input embedding and bypasses the final LayerNorm.For matched states, the learned LayerNorm shift cancels, and the resulting rescaling preserves the leading axis-level tokens.
- Information flow: Reading every position traces where a meaning distinction first appears and whether it is detected at the changed token or later through attention.This supports information-flow analysis across sequence positions.
2.3 Attention and MLP decomposition
The tracer separates attention and MLP writes by projecting each component through the unembedding, while contrastive design tests how prompt framing and token alignment affect interpretability. Matched current and predicted tokens expose contextual and relational content more cleanly than mismatched alternatives.
- Component decomposition: Attention outputs and MLP outputs are captured separately and projected through W_U to identify which component writes a content distinction.The decomposition uses forward hooks and no trained probe.
- Why differencing helps: Differencing matched runs can reveal structure inside an MLP or attention write that raw per-component projections leave as token soup.The shared bulk cancels while the component separating the inputs remains legible.
- Preambles: A shared preamble stabilizes the readout because first-token activation is much larger and can dominate the contrast.In Table 2, the capitalized matched frame has cosine 0.85 to the preambled direction, whereas a first-token mismatch drops it to 0.64.
- Current-token matching: Matching the current token removes a large incidental surface component that otherwise swamps the intended context contrast.With a shared final token, novel-versus-lecture poles read cleanly; with a mismatch, suffix morphology dominates and context appears only late.
- Next-token matching: Matching the predicted next token exposes stored content for later use by canceling the shared local prediction.At “Japan,” shared prediction of “is” leaves place vocabulary on the capital side and monetary vocabulary on the currency side.
2.6 Baseline subtraction: the single-prompt variant
Baseline subtraction extends contrastive projection to a single target by subtracting the mean residual of same-length prose snippets. It removes generic sentence-final content while preserving target-specific vocabulary, though the readout emerges mainly late.
- Baseline construction: Same-length prose snippets provide a generic baseline whose residual positions align, helping cancel position-dependent structure.The equal length keeps the readout index fixed, so positional structure shared across snippets is averaged out.
- Baseline construction: The baseline is coarser than a matched pair because it removes generic style rather than nearly identical computation.Target content therefore clears the readout mainly in late layers, where its prediction is strong.
- Example: For “My grandmother said that the hot dog was,” the prose mean reads generic sentence-final content such as articles and punctuation.Both target and baseline are read at the final position.
- Example: The difference surfaces food vocabulary at late layers, including “delicious, cooked, hotter, tast” at L28.
2.7 Multi-contrast triangulation
Multi-contrast triangulation isolates an intended semantic axis by averaging matched contrasts that share that axis but vary incidental ones. Averaging suppresses incoherent remainder and recovers otherwise weak token-level readings, especially when several features co-vary.
- Method: Matched baselines isolate the intended axis because incidental differences vary across contrasts and cancel when the difference vectors are averaged.This preserves the sentence frame better than subtracting a generic corpus average.
- Method: Averaging N matched difference vectors suppresses incoherent remainder and lowers the visibility threshold without sharpening the signal.The result remains limited by the granularity of the contrast design.
- Axis decomposition: A two-item contrast can span several axes, so the paper builds data-derived axis contrasts and tests each axis by injecting it independently.Projection identifies the axes carried by the original difference, while injection tests their predictive effect.
3 Lexical disambiguation
Lexical contrasts recover context-dependent meanings and trace how a compound distinction is formed, routed, and expressed across models. The hot-dog case separates causal semantic axes from incidental variation and reveals both shared computational flow and model-specific token readouts.
- Compound noun: hot dog: The hot-dog contrast carries edibility, petness, and temperature, with edibility +21, petness −20, and temperature +15 at L20.The contrast therefore spans more than the intended food-versus-animal distinction.
- Compound noun: hot dog: 0.98: combining edibility and petness recovers nearly the full food-versus-animal effect, compared with 0.03 for edibility alone and 0.01 for petness alone.Temperature contributes 0.00 alone and adds nothing to the edibility-plus-petness pair.
- Compound noun: hot dog: Activation patching confirms a two-hop chain: early “dog” patching destroys the food reading, whereas later “was” patching removes food from L12 onward.Patching “hot” has no effect because L0 attention has already copied it to “dog.”
- Cross-model generalization: Across three models, food content reaches the prediction position through later attention, but noun-level token readouts vary by architecture.Pythia and Qwen require triangulation or remain partial at the noun, while greedy behavior confirms food versus animal continuations for Phi-2 and Qwen.
- Cross-model generalization: A probe classifies hot-dog nouns as food and cold-dog nouns as animal in all three models, including Qwen where no clean food token appears.Five-fold probe accuracy is 1.00, with P(food) of 1.00, 0.99, and 1.00 for hot dog.
- Other disambiguation cases: Contrasting “steep bank” with “closed bank” reads riverbank vocabulary for the steep pole and financial vocabulary for the closed pole.
4 Metaphor: domain routing, not a figurativity flag
The metaphor experiments find no single figurativity feature across four words. Instead, contrastive directions encode mappings from source domains to word-specific target domains, and these mappings generalize across contexts and held-out sentences.
- Setup: Four literal/metaphorical adjective pairs test whether metaphor is represented by a single figurativity feature or by localized domain mappings.The shared frame ends before the literal and figurative senses diverge.
- Domain routing: At L24, literal poles return physical domains while metaphorical poles route to word-specific targets: emotion for cold and heavy, social tone for sharp, and intelligence for bright.
- Generalization: Routing directions remain stable across sentences, with pairwise cosine 0.64–0.88 over four pairs, and leave-one-out injection shifts held-out prompts toward the intended domains.For cold, held-out literal prompts move toward emotion and held-out metaphorical prompts toward temperature.
- Cross-domain test: Cross-domain injection shows that each direction imposes its own target domain regardless of context, rather than moving every context toward its own literal sense.The cold direction drives sharp or heavy contexts toward temperature, while the sharp direction drives contexts toward sharpness.
- Direction structure: Cold and heavy directions correlate at 0.58 because both map physical scale onto emotion, while bright is nearly orthogonal to the others at 0.03–0.14.
- Conclusion: The conclusion is scoped to these four words: metaphor is a set of domain-to-domain mappings, not a universal figurativity axis.The mapping is legible only when the contrast holds the domain pair fixed.
5 Recall versus hallucination
Contrastive projection distinguishes genuine recall from hallucinated entity knowledge even when both prompts produce confident, specific answers. Length-matched controls support the finding, while entropy alone cannot reliably separate the cases.
- Both real and fictional entity prompts produce confident, specific predictions, but their contrastive readouts diverge at L28.
- Real entities surface factual associations, whereas fictional entities surface only name fragments and contextual or numerical priors.
- A length-matched control preserves the real-versus-fictional distinction, showing that the factual-versus-name-fragment result does not depend on token-length mismatch.
- Real mountains and fictional mountains can produce similarly confident heights at nearly equal entropy, yet projection recovers geographic knowledge only for the real mountains.
- Missing top-K tokens indicate energy below the visibility threshold, not necessarily zero signal, so absence readings bound rather than prove the absence of content.
6 What the readout reads
The readout’s token identity varies sharply across random initializations, while the underlying contrast remains recoverable. The paper therefore treats tokens as network-specific realizations and recommends validating the distinction itself.
- Across five Pythia-410M seeds, top-10 token overlap is 0.08, while concept anchors beat their opposites in all 20 seed-by-contrast tests.
- The same distinction survives across seeds even when top-ranked tokens differ substantially and often appear as subword fragments.
- The token basis is a property of the trained network rather than a universal code, so interpretation should focus on the axis rather than token identity.
- Anchor margins or injection tests provide stronger confirmation of an axis than the identity of top-ranked tokens.
7 Discussion
The discussion frames contrastive projection as a readout of emitted structure, not a complete account of internal computation. It highlights interpretive hypotheses, practical extensions, and limits on causal and general claims.
- Discussion: WU projection cannot reveal distinctions that remain below visibility or are computed but never emitted into the vocabulary basis.
- Discussion: The metaphor results suggest domain-to-domain offsets rather than a single figurativity feature, but the hypothesis remains limited to four words in one model.
- Discussion: A compositional injection test is left for future work because the graded result suggests that target-domain separation is imperfect.
- Extensions: The method supports extensions such as checkpoint tracing, automatic visibility gating, relation-direction triangulation, and cross-model comparison.
- Limitations: The evidence is limited by hand-curated pairs and baselines, restricted model coverage, omitted final LayerNorm, uncertain WU readability, and token-alignment requirements.
- Limitations: Per-position traces and head decompositions identify contributors rather than causes; causal claims receive only targeted, qualitative checks with known failure modes.
- Limitations: Coherence remains a qualitative inspection, and triangulation has been tested on only six cases without a general baseline-count guarantee.
8 Related work
The paper positions contrastive projection as a training-free, token-space readout that extends matched-pair subtraction beyond steering and complements probes, circuit analysis, and representation decomposition. Its distinctive scope is reading pair-specific distinctions without fitting parameters or claiming causality.
- RepE, ActAdd, and CAA use matched-pair subtraction for steering, while this work develops the same primitive into a systematic tracer.
- Unlike logit and tuned lenses on individual states, contrastive projection reads the content that differs between two inputs.
- The method acts as a zero-shot linear probe along a pair-defined axis, using token-space readout instead of a trained classifier.
- Unlike sparse autoencoders, triangulation cancels baseline-varying content and reads the surviving component rather than decomposing representations.
- Compared with causal tracing and circuit discovery, the method flags layers, heads, and content exploratorily but does not establish causality.
- Its entity readings connect factual-recall work to the observation that fictional entities surface no content beyond name tokens.
Code and Data
The paper's code and data are publicly available, with the archived materials hosted separately.
- Code and data are available in the project's GitHub repository.
- The archived materials are available through Zenodo.