PaperlayerView exact revision on arXiv

Source-linked AI summary

Disentangling Steering Vectors

Takeru Hiramatsu, Kyohei Atarashi, Koh Takeuchi, Hisashi Kashima

arXiv:2609.07037v1cs.LG

TL;DR

Composite steering vectors can contain multiple concepts, motivating a method to isolate semantically consistent basis vectors. Steering Vector Dissection trains an SAE on differences between paired positive and negative activations; across two datasets, two models, and two intervention depths, its vectors are more semantically consistent than DiffMean and induce mutually distinguishable steering effects.

  • Problem

    Traditional composite steering vectors can contain multiple concepts, motivating their disentanglement into semantically consistent basis vectors.

    Source: §1 · p. 2

  • Method

    Steering Vector Dissection pairs positive and negative activations and trains a dedicated SAE on their differences to extract basis vectors.

    Source: §1 · p. 2

  • Results

    Across two datasets, two models, and two intervention depths, the resulting vectors are more semantically consistent than DiffMean and induce mutually distinguishable steering effects.

    Source: §10 · p. 16

  • Takeaways & Limitations

    Removing rejection-related directions from an unsafe DiffMean vector yields targeted changes in model behavior.

    Source: §1 · p. 2

    Evaluation relies on LLM judges and lacks human-annotated ground truth and reported inter-judge agreement.

    Source: §8 · p. 16

Abstract

from arXiv · show

Activation steering has emerged as a lightweight, inference-time approach to control the behavior of Large Language Models (LLMs). However, traditional steering vectors used to intervene in LLMs' activations, such as those derived from the difference-in-means method, tend to entangle multiple semantic and stylistic concepts into a single composite direction, leading to unpredictable steering effects. Our core objective is to disentangle this composite direction into its constituent concepts. To this end, we propose Steering Vector Dissection, a framework to explicitly isolate individual and semantically consistent features from these composite directions. Specifically, we pair positive and negative activations and take their differences to generate a set of instance-level steering vectors, and train a dedicated Sparse Autoencoder (SAE) directly on them. Quantitative evaluations across two datasets, two models, and two intervention depths show that our method yields a set of semantically consistent basis vectors whose steering effects are mutually distinguishable. Furthermore, we show that this disentanglement enables precise control over model behaviors.

1 Introduction

Representation-based steering offers direct intervention in LLMs’ internal representations, but a single steering direction can combine concepts and produce unintended effects. Steering Vector Dissection decomposes such directions into semantically consistent features and supports targeted behavioral changes.

1 Introduction

  • Representation engineering intervenes in LLMs’ internal representations, offering a white-box alternative to black-box preference-based alignment methods.The introduction notes that safety alignment from preference-based methods can be undone through lightweight fine-tuning.Source: §1 · p. 1
  • A single steering direction can entangle multiple concepts, so steering for one concept may also affect unrelated behaviors.The examples include happiness steering coinciding with harmful-request compliance and gender-bias awareness steering worsening age bias.Source: §1 · p. 2
  • Steering Vector Dissection pairs positive and negative activations and trains an SAE on their differences to extract basis vectors.The pairing uses nearest-neighbor search.Source: §1 · p. 2, Figure 1 · p. 2
  • The extracted basis vectors cover distinct, non-overlapping concept-space regions, and removing rejection-related components from unsafe DiffMean vectors yields targeted behavioral changes.The authors report quantitative and visual evidence for the distinct regions and describe the targeted changes as evidence that the directions are behaviorally meaningful.Source: §1 · p. 2

2 Preliminaries

The preliminaries introduce activation steering and sparse autoencoders as context for decomposing composite steering directions. The method builds on DiffMean vectors and applies SAEs to activation differences, while its stated novelty is identifying and selectively editing constituent behavioral directions.

2.1 Activation Steering

  • DiffMean vectors are computed as differences between mean activations from positive and negative text datasets.The means are computed globally across tokens rather than by macro-averaging sequence-level means.Source: §2.1 · p. 3

2.2 Sparse Autoencoders

  • SAEs project activations into a higher-dimensional space to disentangle them into interpretable features; applying them to activation differences provides a framework for decomposing steering vectors.The paper relates activation differences to how DiffMean steering vectors are calculated.Source: §2.2 · pp. 3–4
  • The novelty is not applying an SAE to activation differences itself, but testing whether composite steering vectors’ constituent behavioral directions can be identified and selectively edited.Source: §2.2 · p. 4

3 Steering Vector Dissection

Steering Vector Dissection decomposes composite steering directions by pairing positive and negative token activations, training an SAE on their differences, and interpreting and filtering its decoder directions. The resulting basis vectors are selected for relevance, alignment with their interpretations, and distinction from overlapping concepts.

3.1 Collect and Match Token-wise Activations

  • Batch-wise nearest-neighbor matching pairs positive activations with negative ones, whose differences form instance-level steering vectors for SAE training.Pairing is not one-to-one, and batch-wise search prevents a small number of negative activations from dominating; aligned activations are intended to isolate contextual differences by subtraction.Source: §3.1 · p. 4

3.2 Train an SAE

  • The SAE’s bias-free decoder makes its columns basis vectors for reconstructing steering vectors, while raw, uncentered inputs preserve the vectors’ linear combinations.The authors use “basis vectors” for non-negative combinations in reconstructions, without claiming linear independence; training uses BatchTopK sparsity and inference uses JumpReLU.Source: §3.2 · p. 5

3.3 Interpret the Basis Vectors

  • Each decoder vector is interpreted by steering the model on general instructions and having an LLM judge compare steered responses with the same model’s unsteered responses.For interpretation and filtering, the steering vector is scaled to the average norm of negative activations multiplied by the intermediate coefficient.Source: §3.3 · pp. 5–6

3.4 Filter the Basis Vectors

  • Filtering assesses positive-dataset relevance and held-out alignment, then uses judges to retain relevant, aligned vectors that represent distinct concepts.Alignment averages judge scores from 0 to 2; candidates must exceed 1.0 before set-level overlap review, and retained vectors receive votes from more than half the judges. The appropriate filtering criterion depends on the downstream application.Source: §3.4 · pp. 6–7

4 Experimental Setup

Experiments use two datasets, two open-weight instruction-tuned models, and interventions at two network depths. The setup also specifies the evaluation judges and SAE latent configuration.

4.1 Datasets

  • 8,192 prompt-response pairs are sampled from each dataset, with 80% for training and 20% for validation.PubMedQA is paired with Alpaca as a general instruction-following negative dataset; PKU-SafeRLHF uses severity-3 unsafe and severity-0 safe examples. Ten Alpaca-Eval instructions are used for interpretation and 128 for alignment scoring.Source: §4.1 · p. 7

4.2 Models and Implementation Details

  • Experiments use Qwen2.5-7B-Instruct and Llama3.1-8B-Instruct, intervening at the 0.5 and 0.75 depth quantiles.Source: §4.2 · p. 7
  • GPT-5.4-nano performs relevance and alignment scoring, while GPT-5.4 handles interpretation and final selection; SAEs use 16 latents with four active per input on average.Source: §4.2 · p. 7

5 Semantic Consistency

The evaluation finds that Steering Vector Dissection generally produces more semantically consistent concepts and more consistent steering-induced shifts than DiffMean, with distinct basis vectors yielding distinguishable behavioral effects. The results are an existence proof, not a claim that every extracted vector is interpretable or highly consistent.

5.1 Experimental Design

  • The evaluation measures concept consistency through extracted concepts and embedding-shift consistency through response embeddings, with the latter also comparing effects across distinct basis vectors.Concept extraction uses steered and unsteered responses to general instructions; embedding shifts are computed by subtracting original-response embeddings from steered-response embeddings.Source: §5 · p. 7, §5.1 · pp. 7–8

5.2 Results

  • The authors frame the filtered-vector results as an existence proof, not a claim that every basis vector is human-interpretable or highly consistent.Appendix D separates the SAE’s contribution from that of filtering.Source: §5.2 · p. 8
  • For PubMedQA, DiffMean concepts are broadly scattered, while basis-vector concepts form separated clusters; qualitative examples likewise show focused basis-vector concepts rather than DiffMean’s disparate topics.Table 4 summarizes 128 extracted concepts in eight clusters.Source: §5.2 · p. 9, Figure 2 · p. 10, Table 4 · p. 11
  • In most model–layer pairs, basis vectors have higher, more consistent concept similarity than DiffMean; for Qwen2.5-7B (L13) on PKU-SafeRLHF, DiffMean consistency exceeds the basis-vector average, but the basis-vector maximum remains higher.Source: §5.2 · pp. 8–9
  • Across intervention strengths, basis vectors outperform DiffMean on average in the pairwise similarity of steering-induced embedding shifts.This indicates a more consistent semantic shift across input prompts.Source: §5.2 · p. 10
  • Distinct basis vectors produce embedding shifts with lower similarity to one another than to themselves, indicating behavioral—not geometric—independence.The results do not claim that basis vectors are orthogonal in activation space.Source: §5.2 · p. 11
  • The SAE contributes beyond filtering: compared under the same filtering procedure, it yields more surviving dataset-relevant directions and higher consistency than random candidate selection in most settings.Filtering itself also explains part of the improvement over DiffMean.Source: §5.2 · p. 11

6 Removing Rejection Directions from Unsafe Vectors

The section tests whether removing rejection-related components from unsafe DiffMean vectors improves targeted behavioral control. Removing those components increases jailbreak effectiveness in earlier layers, consistent with rejection directions counteracting the unsafe vector.

6 Removing Rejection Directions from Unsafe Vectors

  • Rejection-related basis vectors appear across models and layers, indicating that unsafe DiffMean vectors contain directions associated with refusing harmful requests.The authors define rejection to include refusing, framing a request as illegal or unethical, or withholding requested content.Source: §6 · p. 11, Table 5 · p. 13
  • For PubMedQA, basis vectors induce more consistent embedding shifts than DiffMean; on PKU-SafeRLHF, distinct vectors have lower shift similarity than their self-similarity.Figure 3 reports average scores across filtered basis vectors with minimum and maximum values in brackets; Figure 4 shows the PKU-SafeRLHF similarity matrices.Source: Figure 3 · p. 12, Figure 4 · p. 12
  • Removing identified rejection directions raises StrongREJECT scores in earlier layers, while unsafe DiffMean alone does not reliably increase compliance.Table 6 compares original responses, DiffMean steering, and DiffMean with the rejection dimension removed on held-out prompts.Source: Table 6 · p. 13, §6 · p. 13
  • An ensemble of three LLM judges selects rejection vectors from their interpretations; the selected direction is removed from DiffMean before held-out StrongREJECT evaluation.The experiment uses validation prompts to choose among multiple rejection dimensions and predefined intervention coefficients.Source: §6 · p. 13

7 Comparison with Singular Value Decomposition

The comparison finds that SAE and SVD extract semantically different basis vectors, with neither method uniformly preferable in the 16-dimensional setting. SAE’s latent dimensionality also supports finer-grained concepts as it increases, unlike SVD’s.

7 Comparison with Singular Value Decomposition

  • SAE and SVD concept extraction scores are comparable, but their extracted basis-vector interpretations differ.For PKU-SafeRLHF in Llama3.1-8B (L23), SAE extracts four named harmful concepts, while SVD extracts one valid vector representing “online”.Source: §7 · pp. 14–15, Table 7 · p. 14, Table 8 · p. 14
  • For jailbreak ablation, SAE extracts harmful-concept directions in Llama3.1-8B (L23), while SVD’s only valid vector represents “online”.The paper cautions that neither method is universally preferable because the extracted semantics and downstream task differ.Source: §7 · p. 15
  • As latent dimensionality increases, SAE yields more numerous, finer-grained basis vectors, whereas SVD’s surviving-vector count grows only slightly without finer concepts.The authors conclude that latent dimensionality controls decomposition granularity under SAE, but not under SVD.Source: §7 · p. 15
  • Ablating the SAE-identified refusal direction increases jailbreak capability more than ablating the corresponding SVD direction.Table 10 reports StrongREJECT scores for each method and model-layer pair; a dash marks pairs with no interpretable rejection vector.Source: §7 · p. 15, Table 10 · p. 15
  • Neither SAE nor SVD is uniformly preferable at 16 dimensions; the choice depends on the concepts required and the downstream task.Their extracted basis vectors differ semantically, and the methods diverge more sharply at higher dimensionalities, where only SAE continues to yield finer-grained concepts.Source: §7 · p. 15

8 Limitations

The authors identify limits in concept coverage, data and evaluation costs, judge-based validation, and decomposition uniqueness. These constraints leave full coverage and canonical recovery unestablished.

8 Limitations

  • Extracted basis vectors do not cover all expected dataset concepts, and the dimensionality needed for full coverage—or whether it is attainable—is unknown.Increasing latent dimensionality yields more numerous and finer-grained concepts, but does not establish full coverage.Source: §8 · p. 16
  • Unlike DiffMean, the SAE-based approach requires relatively many data points to learn the underlying latent structure.Source: §8 · p. 16
  • The pipeline relies heavily on costly LLM judges, limiting its application at scale.Source: §8 · p. 16
  • Evaluation uses LLM judges for interpretations, steering-effect alignment, and concept extraction, without human-annotated ground truth or reported inter-judge agreement.Source: §8 · p. 16
  • The decomposition is not unique: basis vectors depend on initialization, training sample, and latent dimensionality, and are not claimed as canonical.Recovered concepts are broadly consistent across seeds, but that does not establish a canonical decomposition.Source: §8 · p. 16

9 Broader Impact Statement

Removing rejection-related directions from a composite steering vector increases compliance with harmful requests, which the authors present as evidence of behaviorally meaningful decomposition. The authors flag dual-use risk and note that the attack requires white-box access, safety-labeled data, and inference-time intervention.

9 Broader Impact Statement

  • Removing rejection-related directions from a composite vector increases a model’s compliance with harmful requests.The authors present this as direct evidence that the decomposition isolates behaviorally meaningful directions.Source: §9 · p. 16
  • The attack requires white-box activation access, activations over a safety-labeled dataset, and inference-time intervention; the authors describe its attack surface as narrower than prompt-based jailbreaks.They note that an adversary with these capabilities has more direct routes to removing safety behavior, including weight-level fine-tuning.Source: §9 · p. 16

10 Conclusion

Steering Vector Dissection decomposes activation-difference steering vectors into semantically consistent basis vectors with distinguishable effects. The authors also describe practical SAE and steering-coefficient choices, emphasizing holistic hyperparameter selection.

10 Conclusion

  • Across two datasets, two models, and two intervention depths, the resulting basis vectors are more semantically consistent than DiffMean and induce mutually distinguishable steering effects.Removing rejection directions from an unsafe DiffMean vector changes model behavior in a targeted way.Source: §10 · p. 16

A.1 SAE

  • SAEs use 16 latents and BatchTopK k = 4, with a 512 batch size, 0.0005 learning rate, MSE reconstruction loss, and early stopping.Training lasts at most 200 epochs, with a minimum of 10; early stopping follows five validation-loss epochs without improvement.Source: §A.1 · p. 20
  • The authors did not tune hyperparameters, citing LLM-judge costs and the lack of a single metric covering interpretations, semantic consistency, and downstream performance.They recommend selecting settings holistically by dataset, basis-vector interpretations, and downstream application; SVD could replace SAE when its vectors are preferable.Source: §A.1 · pp. 20–21

A.2 Steering Coefficients

  • Table 11 lists weak, intermediate, and strong steering coefficients, which the authors empirically set based on overall steered-output quality.Responses are generated with a maximum length of 128 tokens and the default temperature.Source: §A.1 · p. 20, Table 11 · p. 21, §A.2 · p. 21

B Reproducibility Across Random Seeds

A different random seed yielded broadly similar extracted concepts, though vector counts and concept granularity varied. The authors also report increased jailbreak capability in earlier layers of both models under this seed.

B Reproducibility Across Random Seeds

  • With a different random seed, concept-extraction scores again exceeded DiffMean in every model–layer pair, and embedding-shift consistency showed the same trend.The extracted semantic families were broadly similar across seeds, though not identical.Source: B Reproducibility Across Random Seeds · p. 21
  • Concept families recur across seeds, but the number of vectors and granularity of individual concepts vary.PubMedQA families include signaling pathways, disease, research framing, and treatment; PKU-SafeRLHF families include hacking, explosives, trafficking, misinformation, and rejection.Source: B Reproducibility Across Random Seeds · p. 21
  • Under the different seed, StrongREJECT results again show increased jailbreak capability in earlier layers of both models.Table 15 reports the scores on held-out test splits.Source: B Reproducibility Across Random Seeds · p. 21

C Sensitivity Analysis on the Number of Latent Dimensions

The analysis finds that larger SAE dictionaries yield finer concepts, while SVD does not show the same refinement. The SAE also produces more surviving candidates than random selection, and its reconstruction captures DiffMean vectors with high cosine similarity.

E Reconstruction of DiffMean Vectors

  • After scaling DiffMean vectors to the average input-vector norm, SAE reconstruction achieves an average cosine similarity of 0.96, with 9–14 of 16 dimensions activated.All dimensions surviving the earlier analysis are activated in reconstruction.Source: E Reconstruction of DiffMean Vectors · p. 24

C Sensitivity Analysis on the Number of Latent Dimensions

  • At 16, 32, 64, and 128 latents, the SAE yields 7, 8, 18, and 33 surviving vectors, with finer concepts as dimensionality increases; SVD does not show this refinement.The main experiments use 16 latents because that setting sufficed for the paper's claims, while larger dictionaries increase LLM-judge costs.Source: C Sensitivity Analysis on the Number… · pp. 21–22, Table 16 · p. 25

D Contributions of the SAE and Filtering

  • The SAE exceeds random selection on concept-extraction scores in six of seven pairs and yields 36 surviving vectors versus 13, while filtering alone also raises scores among survivors.Random survivors exceed DiffMean in five of seven pairs where any survive; the score is conditional on survival, and filtering includes an alignment criterion measuring steering consistency.Source: D Contributions of the SAE and Filtering · pp. 22–23, Table 19 · p. 26
Loading 2609.07037v1…