What Does Activation Steering Control? Attribution Across Answer Encodings and Output-Sensitive Subspaces

Motivation

• Activation steering changes LLM behavior without updating weights, but reported score gains may not reveal what the intervention controls.CAA constructs a direction from hidden-state differences between positive and negative examples and injects it at inference time.Source: §1 · p. 1

Method

• Cross-Encoding Steering Evaluation freezes the intervention while counterfactually re-rendering answer scaffolds for the same held-out items.The method holds the item, model, direction, layer, position, and dose fixed while varying answer encoding.Source: §1 · pp. 1–2, Figure 1 · p. 2

Findings

• On NormBank, extraction-index following exceeds semantic-label following under all five remappings, emerges at later tested depths, and concentrates in output-sensitive components.A low-rank component containing 15.4% of the direction’s squared norm retains 96.3% of the effect.Source: §1 · pp. 2–3
• Attribution varies across evaluation settings: MNLI favors extraction-index following in aggregate, SC101 favors semantic-label following, and open-ended behavior can diverge from multiple-choice scores.The paper therefore treats the evidence required as dependent on the claimed form of control.Source: §1 · pp. 2–3

Method

• The audit distinguishes semantic-label following, extraction-index following, and extraction-row following under remapped answer encodings.Identifier vocabularies align A/X/1, B/Y/2, and C/Z/3 for cross-vocabulary comparisons.Source: §1 · pp. 1–3

Steering evaluation

• CAA and representation-engineering methods inject contrastive activation directions, while ITI edits probe-informative attention heads.Prior work applies these interventions to moral profiles, values, and cultural adaptation.Source: §2 · p. 3
• Prior evaluation research emphasizes context-matched tasks, likelihood-aware metrics, standardized comparisons, and explicit baselines.It also documents input sensitivity, spurious dependence, and out-of-distribution brittleness.Source: §2 · p. 3

Output-sensitive subspaces

• The paper adapts projection–intervention logic from output-related representation analyses to an identifier-readout-specific attribution target.Its output-sensitive subspace analysis connects hidden-state directions to option-identifier logit differences.Source: §2 · p. 3

Contrastive Activation Steering

• The canonical CAA direction averages contrastive hidden-state differences and is injected at inference time with fixed extraction and injection settings.The primary audit uses the pre-answer position and fixes the direction, block, position, and strength before testing answer encodings.Source: §3.1 · p. 4

Cross-Encoding Evaluation

• Cross-encoding evaluation holds the intervention fixed while changing label mappings, identifier vocabularies, row orders, and candidate-answer renderings.Comparisons use within-encoding mean-token log-likelihood margins because each encoding has its own output sequences and margin scale.Source: §3.2 · p. 4, §3.3 · p. 5

Attribution Definitions

• Extraction-index effects follow identifiers at the indices assigned during direction extraction, whereas semantic-label effects follow labels under the current test mapping.Extraction-index alignment is preserved across A/B/C, X/Y/Z, and 1/2/3 vocabularies, separating it from current labels and displayed rows.Source: §3.2 · pp. 4–5

Localization

• Depth and position sweeps test when attribution emerges and whether it depends on the pre-answer scaffold, with per-model direction norms matched to a common reference.The sweep evaluates 50%, 62.5%, 75%, and 87.5% depths, while matched-position injection tests scenario-end versus pre-answer placement.Source: §3.4 · p. 5

Output-Sensitive Subspaces

• The output-sensitive subspace is built from normalized option-logit gradients, whose leading singular vectors capture dominant identifier-logit sensitivities.The smallest rank among 2, 4, 8, and 16 capturing at least 90% of validation-gradient energy defines the subspace; held-out prompts test projection and residual interventions.Source: §3.4 · pp. 5–6
• Effect concentration is assessed by comparing each projected component’s extraction-index effect with its share of the full CAA direction’s squared norm.Effect retention divides the component effect by the full canonical effect, using norm-matched variants and covariance-matched random controls.Source: §3.4 · p. 6

Cross-Method Validation

• An ITI-style replication tests whether attention-head interventions reproduce the extraction-index-over-semantic-label ordering under validation-selected settings.The method ranks heads by validation probe accuracy and freezes selected layers, head count, and strength before mapping evaluation.Source: §3.5 · p. 6

Datasets

• NormBank provides matched taboo, normal, and expected contrasts, while MNLI and SC101 extend evaluation to non-norm and action-level three-label tasks.The public CAA behaviors additionally extend evaluation to generation.Source: §4 · p. 7

Splits

• NormBank splits complete setting-and-behavior groups across train, validation, and test without shared pairs, groups, or endpoints.The factorial audit evaluates all 108 conditions, and MNLI splits analogously by normalized premise.Source: §4 · p. 7

Models

• CAA experiments use Qwen2.5-7B-Instruct, Llama-3.1-8B-Instruct, Mistral-7B-Instruct-v0.3, and Gemma-2-9B-IT, while ITI retains three models after validation screening.The public protocol uses Llama-2-7B/13B-Chat.Source: §4 · p. 7

Interventions

• Primary CAA interventions use fixed near-75%-depth blocks and α = .8, with norm-matched depth audits and validation-selected ITI settings.SC101 uses previously selected middle layers and α = 1; test answer encodings do not alter extracted directions or intervention hyperparameters.Source: §4 · p. 7

Scoring

• Cross-encoding and factorial comparisons use mean-token log-likelihood margins, whereas localization uses normalized option-choice probability margins on a separate scale.Intervals resample complete setting–behavior groups, and effect retention compares equal-stratum component effects with canonical CAA.Source: §4 · p. 7

Factorial Audit

• Figure 2 reports extraction-index advantage, extraction-index-minus-row effects, and extraction-index effects across identifier vocabularies on the NormBank margin scale.All reported intervals exclude zero, and the corresponding sign-flip tests remain significant after Holm correction.Source: Figure 2 · p. 8

Cross-encoding attribution

• 4.63 is the random-adjusted extraction-encoding effect; after remapping, extraction-index effects range from 3.00 to 4.13 and exceed semantic-label effects by 1.95–4.04.This ordering holds across all five alternative mappings on the strict split.Source: §5.1 · p. 8
• 0.32–1.86 is the extraction-index effect’s advantage over extraction-row effects across every model, identifying identifier index rather than row position as the stronger association.The pooled effect remains 1.455 [1.443, 1.467] with advantage .426 [.408, .443] on 309 pairs solved correctly under all mappings.Source: §5.1 · p. 8

Layer and position localization

• 0.065, 0.130, 0.215, and 0.160 are the extraction-index effects across increasing tested depths, with larger effects and advantages at 75% and 87.5% depth after norm matching.All four models show positive extraction-index effect and advantage at 75% and 87.5% depth.Source: §5.1 · p. 8, Figure 3 · p. 9
• 98.3% is the reduction in extraction-index probability-margin effect when direction extraction moves from pre-answer to scenario end, remaining near zero after norm rescaling and matched-position injection.The pre-answer site includes the question, displayed identifiers, label descriptions, and output instruction; the scenario-end site precedes this scaffold.Source: §5.1 · p. 9

Output-sensitive subspaces

• 15.4% of the direction’s squared L2 energy retains 96.3% [95.8, 96.7] of its extraction-index effect, while the residual retains 1.0% [0.6, 1.3].A mean local-gradient controller also produces the same ordering, with extraction-index effect 0.417 and advantage 0.526 versus 0.215 and 0.236 for canonical CAA.Source: §5.1 · p. 9

Cross-model and behavioral validation

• ITI favors extraction-index following across all five alternative mappings in three retained models, while MNLI favors it in aggregate and SC101 favors semantic-label following.Qwen reverses the MNLI pattern, and multiple-choice and open-ended evaluations produce different verdicts for some behaviors.Source: §5.1 · p. 10, §5.2 · p. 10

• On NormBank, extraction-index following is weak before the answer scaffold and concentrates at late pre-answer states sensitive to identifier logits.Source: §6 · p. 11
• A selected output-sensitive subspace carries nearly all of the effect, while removing it does not increase semantic-label following.Source: §6 · p. 11
• MNLI extends the pooled pattern, whereas Qwen and SC101 show model- and evaluation-dependent profiles.Source: §6 · p. 11
• Extraction-index following is an evaluation-level attribution, not evidence that the model represents an abstract index variable.Source: §6 · p. 11

• The six-mapping audit covers three three-label tasks and four instruction-tuned model families, while full factorial, localization, and ITI analyses use NormBank.Source: §7 · p. 11
• The gradient-defined subspace localizes the effect relative to identifier readouts at tested layers and positions without identifying a complete circuit.Source: §7 · p. 11
• Layer-wise norm matching equates direction magnitude but not layer-specific model sensitivity, and position localization uses a single-site intervention.Source: §7 · p. 11
• The public CAA comparison follows protocol-specific doses.Source: §7 · p. 11

• CAA preferentially follows extraction indices on NormBank, with the effect emerging at later tested depths and concentrating in a low-rank output-sensitive component.Source: §8 · p. 11
• ITI replicates the extraction-index-over-semantic-label ordering in three models.Source: §8 · p. 11
• MNLI reproduces the pooled extraction-index ordering despite a Qwen reversal, whereas SC101 favors semantic-label following.Source: §8 · p. 11
• These contrasts show that steering gains under one answer encoding do not by themselves identify what the intervention controls.Source: §8 · p. 11

Scoring and evaluation

• NormBank and MNLI cross-encoding tests use mean-token log-likelihood margins, while localization analyses use normalized option-choice probability margins.Source: §A.1 · p. 15

Pair construction

• NormBank preprocessing retains taboo, normal, and expected labels, groups rows by exact setting and behavior, and deterministically matches contrast endpoints without reuse.Source: §A.2 · p. 15

Data isolation

• The strict NormBank split assigns complete setting–behavior groups to partitions and removes overlap in pair IDs, context groups, and endpoints.Source: §A.2 · p. 15, Table 7 · p. 16

Factorial audit

• The factorial audit reuses strict CAA directions across six mappings, three identifier vocabularies, and six row orders, producing 108 conditions per frozen direction.Source: §A.2 · p. 15

Cross-task replication

• MNLI pairs hypotheses within normalized premise groups and assigns complete premise groups to one split, while SC101 evaluates three label contrasts under all six mappings.Source: §A.2 · pp. 15–16

Prespecification

• The audits use fixed layers, strengths, templates, and pre-specified settings, with SC101 settings differing from NormBank and MNLI and therefore not supporting cross-dataset magnitude comparison.Source: §A.3 · p. 17, Table 8 · p. 17

Prompt rendering

• Candidate answers are scored as alternative continuations of an unchanged prompt prefix, so changing encoding changes label expression while retaining sequence-likelihood scoring.Source: §A.4 · p. 17

Answer-key checks

• Opaque open-ended results require temporary-key decoding; published CAA models fail a stricter counterbalanced key test and are excluded from open-ended behavioral evidence.Source: §A.5 · p. 20

Factorial audit

• The audit independently varies semantic mapping, identifier vocabulary, and row permutation while scoring semantic-label, extraction-index, and extraction-row margins.Identifier indices are aligned across A/B/C, X/Y/Z, and 1/2/3 vocabularies.Source: §B.1 · p. 21

Model-level attribution

• Extraction-index effects exceed extraction-row effects in all four models, and exceed semantic-label effects in Llama, Mistral, and Gemma but not Qwen.All identifier-minus-label and identifier-minus-row comparisons remain nonzero after Holm correction at p < .001.Source: §B.1 · p. 22

Vocabulary sensitivity

• Changing identifier vocabulary attenuates the extraction-index effect for every model, measuring compatibility of the frozen intervention with each identifier set.The comparison uses steered-minus-base mean log-likelihood-margin changes for single-token identifiers.Source: §B.1 · p. 22

Layer localization

• The extraction-index advantage emerges mainly at later depths rather than from direction-norm growth alone.All four models show positive extraction-index effect and advantage at 75% and 87.5% depth after norm matching.Source: §B.2 · pp. 23–24

Position localization

• Scenario-end extraction reduces the extraction-index effect by 98.3%, while norm matching does not restore it.Pre-answer and scenario-end directions also show low cosine alignment across model–contrast cells.Source: §B.3 · p. 24

Output-sensitive subspace

• A low-rank output-sensitive projection containing 15.4% of squared L2 energy retains 96.3% of the extraction-index effect.Across models, the projection retains 93.3–103.5% of the full effect while the residual remains between −.0056 and +.0085.Source: §B.5 · p. 26, §B.6 · pp. 27–28

ITI replication

• The ITI-style replication trains head-level probes, selects heads and intervention strength on held-out validation data, and tests only interventions meeting a positive-gain criterion.Heads are selected from candidate layer fractions using validation probe accuracy and target-margin gain.Source: §C.1 · p. 28, Table 27 · p. 29

NormBank results

• Extraction-index effects range from 6.26 to 7.34 across alternative mappings, while semantic-label effects range from −3.05 to +1.71.Extraction-index advantage is positive under all five mappings, and every reported interval excludes zero.Source: §C.1 · pp. 28–29, Table 28 · p. 29

Cross-method comparison

• Both CAA and ITI show positive extraction-index effects under all five alternative mappings, whereas semantic-label effects vary with the mapping.The comparison concerns ordering rather than effect magnitudes because the methods use different strengths.Source: §C.1 · p. 29

MNLI

• On MNLI, extraction-index effects and advantages are positive across all five alternative mappings, but model-level advantages range from 3.090 to −1.966.Gemma, Llama, and Mistral are positive across mappings, while Qwen is negative across mappings.Source: §C.2 · pp. 29–30, Figure 4 · p. 30, Table 29 · p. 30

SC101

• SC101 reverses the aggregate NormBank pattern: all five mappings have positive semantic-label effects and negative extraction-index advantage.SC101 values use a probability scale and are not numerically compared with NormBank.Source: §C.3 · p. 30, Table 30 · p. 31

Matched-context diagnostic

• The matched-context diagnostic applies one additive direction uniformly to every item, so it measures context-dependent response rather than an explicit conditional gate.This design leaves selective activation as a scope boundary of the diagnostic.Source: §D.1 · p. 31, Setting–Behavior Cluster Bootstrap · pp. 31–32, Table 31 · p. 31

Label-cue control

• The label-cue direction is a scenario-free lexical control formed from target and source prompts containing explicit classification labels.It is norm-matched to canonical CAA and excludes scenarios and options.Source: §D.2 · pp. 32–33

Label-cue results

• The label-cue direction produces a direct-label gain of 2.618, while other gains are .114–.277 and context effects range from −.240 to .075.Ranking changes range from −.02 to .27 points, showing no robust selective-context control.Source: §D.2 · p. 33

Open-ended controls

• For hallucination and refusal, swapped A/B effects retain 0.94–1.10 times the original fixed-direction effect within the shared A/B token space.Direct-completion effects remain on a separate log-likelihood-margin scale, so cross-encoding retention ratios are not formed.Source: Protocol and Judge Robustness · p. 33

Judge robustness

• Judge agreement is high, but numerical judge scales are not interchangeable.Pairwise agreement spans Pearson .886–.943, Spearman .801–.829, and 80.3–86.3% within one point.Source: Protocol and Judge Robustness · pp. 33–34

Scoring validation

• Human and automated scores correlate at .887 across 12 nonzero-dose cells and agree in sign for 10, supporting the scoring procedure.The 180-response sample supports the procedure, while Table 36 estimates use all 918 generations.Source: Blinded Human Validation of Open-Ended… · p. 34, Table 37 · p. 34

Robustness checks

• Group-disjoint and pair-ID splits preserve the main extraction-index conclusions in 17 of 18 encoding–metric cells.Both splits retain positive extraction-index effects and advantages under all five alternative mappings.Source: §E.1 · p. 35, Figure 5 · p. 35

Direction construction

• The mapping-balanced direction averages directions from all six semantic mappings, rescales the average to canonical CAA norm, and freezes it for the complete audit.The control uses the same six-mapping family as direction construction and evaluates the same strict test pairs.Source: §E.2 · p. 35

Attribution shift

• The mapping-balanced direction reduces extraction-index advantage in all four models, with a pooled reduction of 1.536 [1.525, 1.547].Qwen remains semantic-label dominant; Llama and Gemma switch dominance, while Mistral retains a small positive extraction-index advantage.Source: §E.2 · p. 35, Table 38 · p. 36

Encoding-specific gains

• Canonical CAA gains 4.624 under extraction encoding and 2.845 under direct-label completion, compared with 1.255 and 3.910 for the mapping-balanced direction.Each pair is compared only within its own encoding scale.Source: §E.2 · pp. 35–36

Implication

• The mapping-balanced control shows that direction construction can reduce answer-encoding dependence, while outcomes remain model-dependent.This is a construction-level control rather than a universal elimination of extraction-index following.Source: §E.2 · pp. 35–36