Source-linked AI summary
Steering Large Language Model Activations in Sparse Spaces
Reza Bayat, Ali Rahimi-Kalahroudi, Mohammad Pezeshki, Sarath Chandar, Pascal Vincent
TL;DR
Controlling LLM behavior at test time remains difficult, especially when dense activation steering entangles features and limits interpretability. SAS extracts behavior-specific features in sparse SAE spaces, enabling precise, compositional steering while preserving standard benchmark performance and potentially improving factual accuracy.
Problem
Existing sparse steering approaches may retain superposition or fail to capture behaviors composed of multiple sub-features, limiting precise behavioral control.
Source: §3 · p. 4Method
SAS uses sparse autoencoders and contrastive prompt pairs to extract interpretable behavior-specific features that can reinforce or suppress behaviors during inference.
Source: §1 · p. 2, §3.1 · p. 5Results
SAS enables precise, interpretable, and compositional behavioral steering without degrading standard benchmark performance, while non-hallucination vectors improve TruthfulQA factual accuracy.
Source: §6 · p. 13Takeaways & Limitations
Scaling SAE dictionaries improves SAS monosemanticity, supporting more precise and reliable fine-grained behavior modulation.
Source: §1 · p. 2, §6 · p. 13The paper identifies higher-quality, larger-scale data and conditional steering as future directions for improving feature extraction, generalization, and context adaptability.
Source: §6 · p. 13
Abstract
from arXiv · showhide
A key challenge in AI alignment is guiding large language models (LLMs) to follow desired behaviors at test time. Activation steering, which modifies internal model activations during inference, offers a potential solution. However, prior work in dense activation spaces struggles with superposition, wherein multiple features become entangled, limiting interpretability and precise control. In contrast, sparse representations provide an untapped opportunity for more interpretable behavior modulation. In this work, we introduce sparse activation steering (SAS), a method that leverages sparse autoencoders (SAEs) to steer LLM behavior in sparse spaces. By isolating behavior-specific features through a contrastive prompt-pairing approach, we define a set of features that can selectively reinforce or suppress behaviors. Experiments on Gemma 2 LLMs show that SAS vectors enable nuanced behavioral modulation and finer-grained control. Furthermore, scaling SAEs improves monosemanticity of SAS vectors, suggesting more reliable and interpretable interventions.
1. Introduction
The paper introduces Sparse Activation Steering (SAS), which uses sparse autoencoders to extract interpretable behavior features for reinforcing or suppressing model behaviors at inference time. It addresses limitations of dense steering by enabling more precise, fine-grained, and compositional control while preserving task-dependent flexibility.
- Activation steering offers targeted test-time behavior adjustment, unlike fine-tuning, prompting, and RLHF, which lack flexibility, interpretability, or fine-grained control.Steering can be applied only to a targeted domain, preserving overall model performance when it is not applied.Source: §1 · pp. 1–2
- Sparse representations address dense steering’s superposition problem by decomposing activations into structured, ideally monosemantic features.This disentanglement supports more precise and interpretable interventions and finer-grained behavioral control.Source: §1 · p. 2
- Directly converting dense steering vectors into sparse vectors is challenging because they fall outside SAE training distributions and may contain negative feature projections that non-negative SAEs cannot process.SAS is introduced as a framework designed to extract steering vectors directly in sparse spaces.Source: §1 · p. 2
- SAS extracts behavior-specific sparse features that can reinforce desired behaviors and suppress opposing behaviors during inference.It constructs SAS vectors from contrastive positive and negative prompt completions by removing shared features and subtracting their mean sparse activations.Source: Figure 1 · p. 1, §1 · p. 2
- Scaling the SAE dictionary improves SAS-vector monosemanticity, while sparse feature compositionality enables simultaneous modular control of fine-grained behaviors.These properties support more interpretable, flexible, and precise behavior modulation.Source: §1 · p. 2
2. Background
This section introduces activation steering as inference-time modification of LLM activations and sparse autoencoders as a framework for learning sparse, interpretable features. It motivates sparse representations by noting that superposition complicates control in dense activation spaces.
2.1. Activation Steering
- Activation steering adds vectors to LLM latent representations during inference to guide behaviors such as corrigibility, hallucination, or refusal (Li et al., 2024b; Panickssery et al., 2023).The steering strength is controlled by λ.Source: §2.1 · pp. 2–3
- Contrastive Activation Addition (Panickssery et al., 2023) constructs behavior-specific steering vectors from differences between residual-stream activations for paired prompts with contrasting behaviors.The resulting vector is added to model activations during inference.Source: §2.1 · p. 3
2.2. Sparse Autoencoders
- Superposition entangles multiple concepts in limited-dimensional representations, complicating interpretability and control; dictionary learning therefore uses sparse autoencoders to learn larger sparse representations (Elhage et al., 2022; Bricken et al., 2023; Templeton et al., 2024).Sparse representations are intended to address entanglement while exploiting representation-space capacity.Source: §2.2 · p. 3
- Sparse autoencoders encode activations into a high-dimensional sparse latent space and decode them for reconstruction, with sparsity enforced by functions such as ReLU, TopK, or JumpReLU (Gao et al., 2024; Rajamanoharan et al., 2024b).Sparsity activates only a small subset of dictionary features, supporting disentanglement and modular behavioral control; the work uses pretrained JumpReLU SAEs (Rajamanoharan et al., 2024b; Lieberum et al., 2024).Source: §2.2 · pp. 3–4
3. Method
SAS derives behavior-specific steering vectors directly in SAE sparse space, avoiding limitations of predefined features and SAE reconstruction of dense steering vectors. Contrastive prompt pairs isolate features for reinforcement or suppression during inference while preserving sparse representations.
3. Method
- SAS directly derives interpretable, behavior-specific steering vectors in SAE space, avoiding insufficient monosemanticity, behavior sub-feature coverage, out-of-distribution reconstruction, and non-negative-space limitations.Predefined SAE features may remain entangled or fail to capture multifaceted behaviors, while reconstructing dense steering vectors can misalign because they lie outside the SAE training distribution and include negative projections.Source: §3 · p. 4
3.1. Sparse Activation Steering Vectors Generation
- SAS contrasts positive and negative prompt completions, averages consistently present sparse features, and removes shared activations to isolate behavior-specific components for reinforcement or suppression.Prompt pairs differ in appended answer letters; the frequency threshold τ controls feature consistency, while zeroing shared features removes syntactic, answer-letter, and positional artifacts before combining positive and negative directions.Source: §3.1 · pp. 4–5
3.2. Sparse Activation Steering Vectors in Inference
- During inference, SAS modifies each token’s sparse latent representation, applies a reconstruction-loss correction, and maps it back to dense activations while λ controls amplification or suppression.Positive λ amplifies the target behavior and negative λ suppresses it; the activation function preserves sparse-space non-negativity and the correction term minimizes deviation from original activations.Source: §3.2 · pp. 5–6
4. Experiments
Experiments across multiple-choice, open-ended, benchmark, interpretability, compositionality, and ablation settings show that sparse activation steering enables controllable behavioral modulation while preserving or improving task performance under moderate intervention. Scaling, feature composition, behavioral correlations, and design ablations further clarify why SAS can provide interpretable and stable control.
4.1. Multi-Choice Questions Steering Evaluation
- SAS steers seven alignment-relevant behaviors in held-out multiple-choice evaluations by measuring probability shifts toward target and opposing answers.The experiments use Gemma-2 models with JumpReLU SAEs, primarily the 2B model, and evaluate refusal, sycophancy, hallucination, corrigibility, factuality, survival instinct, and myopic reward.Source: §4 · p. 6, §4.1 · p. 7
- Increasing λ strengthens steering, while lower τ retains more features for stronger shifts and higher τ enforces greater sparsity while preserving effective steering.Positive λ reinforces target behaviors and negative λ suppresses them; these effects are shown in Figures 3 and 4.Source: Figure 4 · p. 6, Figure 3 · p. 6, §4.1 · p. 7
4.2. Open-Ended Generation Steering Evaluation
- Open-ended evaluations show generally increasing behavioral scores as λ rises relative to the unmodified case (λ = 0), with greatest steerability when answer choices accompany the prefix “The answer is:” and an LLM judges responses.Source: Figure 5 · p. 7, §4.2 · p. 8
4.3. Steering Effect on Standard Benchmarks
- Moderate steering (λ=1 or 2) has minimal negative impact on MMLU and TruthfulQA, while non-hallucination and myopic-reward steering can improve performance and extreme λ degrades both benchmarks.TruthfulQA improves with steering toward refusal or away from hallucination, while MMLU improves in some cases with myopic-reward steering.Source: §4.3 · p. 8, Figure 6 · p. 8
4.4.1. Scaling Monosemanticity
- Scaling SAE width from 16K to 1M increases SAS sparsity and monosemanticity while preserving positive-steering performance, though negative steering slightly degrades at higher τ.Larger SAEs maintain stable raw active-feature counts, suggesting richer but more behavior-specific representations; the steering strength was fixed at λ=±1.Source: §4.4.1 · p. 9, Figure 7 · p. 9
4.4.2. Feature Compositionality
- Composing Myopic Reward and gender features yields fine-grained control, including probability increases of 23.9% for Bob, Non-myopic; 34.3% for Alice, Myopic; 7.6% for Bob, Non-myopic; and 21.5% for Alice, Non-myopic across configurations.The configurations combine λ_M∈{−2,2} with gender steering λ_G∈{−1,0,1}, relative to the unmodified baseline.Source: §4.4.2 · p. 10, Table 1 · p. 10
4.4.3. Behavioral Correlation
- Feature-overlap analysis finds that positive refusal and negative hallucination directions share features, a correlation validated by multiple-choice evaluation and suggesting refusal can suppress hallucination.Additional positive-positive and negative-negative behavioral correlations are explored in the appendices.Source: §4.4.3 · p. 10, Figure 8 · p. 11
4.5. Ablations
- Ablations show that Δ correction stabilizes behavior across layers, two-sided steering outperforms one-sided steering, and removing common features preserves steerability while increasing sparsity.The correction Δ= a−â restores SAE reconstruction loss; common features largely cancel during vector subtraction and have minimal steering impact.Source: §4.5.1 · p. 11, Figure 9 · p. 11, §4.5.2 · p. 11
5. Related Work
Prior work establishes activation steering and sparse autoencoders as complementary tools for behavior control and interpretability. However, systematically identifying and applying steering features in sparse representations remains underexplored.
Activation Steering
- Activation steering modifies model activations at inference without retraining or changing weights, but dense-space superposition hinders fine-grained control (Elhage et al., 2022).Existing methods derive steering vectors by averaging residual-stream differences between contrasting prompts (Panickssery et al., 2023).Source: §5 · p. 13
Sparse Autoencoders for LLMs
- Sparse autoencoders decompose dense activations into sparse, interpretable features while enforcing sparsity through mechanisms including TopK, GLUs, and JumpReLU.SAEs extend traditional autoencoders and can be integrated at different points in transformer architectures.Source: §5 · p. 13
- Gemma Scope provides open-source JumpReLU SAEs with varied sparsity and width for Gemma 2B and 9B, motivating its use for availability and scalability.The resource was released by Lieberum et al. (2024) for Gemma 2 models (Team et al., 2024).Source: §5 · p. 13
Activation Steering by SAEs
- Prior SAE-based steering studies target knowledge selection, refusal, and unlearning, but a unified framework for extracting sparse steering vectors and precisely modulating behavior remains underexplored.Refusal-focused approaches can substantially affect performance on key benchmarks.Source: §5 · p. 13
6. Conclusion … E. Reproduction of CAA Algorithm
The paper concludes that Sparse Activation Steering (SAS) enables precise, interpretable, and compositional behavioral control, while preserving or improving benchmark performance. The appendices define the SAE and steering procedures, connect activation steering to classifier guidance, and reproduce dense steering results across models.
6. Conclusion
- SAS enables precise, interpretable, and compositional behavioral control without degrading standard benchmarks, while improving TruthfulQA factual accuracy and MMLU performance.Scaling SAEs improves monosemanticity and intervention precision; non-hallucination vectors improve TruthfulQA factual accuracy.Source: §6 · p. 13
A. Sparse Autoencoder
- The SAE objective combines L2 reconstruction with an L1 sparsity penalty, balancing faithful activation reconstruction against a reduced number of active features.This objective provides the basic sparse-autoencoder formulation underlying the appendix methods.Source: A. Sparse Autoencoder · p. 18
A. Sparse Autoencoder / A.1. JumpRelu SAE
- JumpReLU SAEs threshold weak activations to reduce false positives, improving disentanglement and reconstruction fidelity while using an L0 sparsity penalty and straight-through estimators.Pre-trained Gemma Scope SAEs provide latent dimensions ranging from 16k to 1m for Gemma 2 models.Source: §A.1 · p. 18, A. Sparse Autoencoder · p. 18
B.1. Activation Steering as a Linear Classifier
- Classical activation steering constructs a positive-minus-negative mean activation vector and adds it at a chosen layer, with λ controlling steering magnitude and direction.Positive λ promotes the behavior, whereas negative λ suppresses it.Source: §B.1 · p. 19, §B.2 · p. 19
B. Connection Between Activation Steering and Classifier Guidance / B.2. Classifier-Based Guidance Formulation
- Under linear assumptions and one optimization step, classifier guidance recovers the classical activation-steering vector up to a scaling factor.The learned classifier weights are proportional to the difference between mean positive and negative activations.Source: §B.2 · p. 19
C. Algorithm: Sparse Activation Steering (SAS) Vectors Generation
- SAS generation isolates behavior-specific sparse features by frequency filtering, removing shared positive and negative features, and differencing their averaged steering vectors.The resulting vectors amplify desired behavior and suppress undesired behavior while remaining interpretable.Source: C. Algorithm: Sparse Activation… · p. 20
D. Algorithm: Sparse Activation Steering (SAS) Vectors in Inference
- During inference, SAS corrects dense activations, adds a λ-scaled sparse vector in SAE space, decodes the result, and restores the correction term.This procedure uses the encoder, SAE activation function, decoder, and steering vector at a selected layer.Source: D. Algorithm: Sparse Activation… · p. 21
E. Reproduction of CAA Algorithm
- The CAA reproduction evaluates layer-wise positive and negative dense steering on Llama-2 7B and Gemma-2 2B and 9B models across multiple behaviors.Figure 12 reports behavior alignment as ΔP for behaviors including AI coordination, corrigibility, hallucination, and sycophancy.Source: E. Reproduction of CAA Algorithm · p. 22, Figure 12 · p. 22
F. Multi-Choice Questions Steering Evaluation on Gemma-2 9B
On Gemma-2 9B multiple-choice evaluation, increasing SAS strength from ±1 to ±2 produces larger behavior-alignment shifts, with positive steering amplifying and negative steering suppressing target behaviors.
Evaluation setup
- SAS vectors generated from behavioral datasets steer Gemma-2 9B outputs on held-out multiple-choice examples across λ values from ±1 to ±2 and τ values of 0.7 and 0.2.The evaluation follows the procedure used for vector generation and uses SAE dictionaries of size 131K with average L0 set to max.Source: F. Multi-Choice Questions Steering… · p. 23, Figure 13 · p. 23, Figure 14 · p. 24
Results
- Increasing λ from ±1 to ±2 strengthens behavior-alignment shifts, with positive steering amplifying matching behaviors and negative steering suppressing the target behavior.The effect is most evident in intermediate layers, where high-level behavioral features are typically represented.Source: F. Multi-Choice Questions Steering… · p. 23, Figure 13 · p. 23, Figure 14 · p. 24
G. Choice of SAE Width and Average 𝐿0 on Multi-Choice Question Steering
The study evaluates multiple-choice question steering with Gemma-2 2B across SAE widths, average 𝐿0 settings, and 𝜏 values. Lower 𝜏 and higher average 𝐿0 produce more effective steering, while 𝜏=1 performs worse.
Experimental setup
- The evaluation varies SAE widths of 16𝐾 and 65𝐾, maximum or near-60 average 𝐿0, and 𝜏 across [0, 1].Figures 15–18 report the resulting evaluations for Gemma-2 2B multiple-choice steering.Source: G. Choice of SAE Width and Average 𝐿0… · p. 25
Results
- Lower 𝜏 values outperform 𝜏=1, while higher average 𝐿0 improves steering effectiveness.These observations align with findings from Sections 4.1 and 4.4.Source: G. Choice of SAE Width and Average 𝐿0… · p. 25
- Figures 15 and 16 show 𝜏 sweeps at maximum average 𝐿0 for 65𝐾 and 16𝐾 SAE dictionaries, respectively.Both evaluations use 𝜆= ±1.Source: Figure 15 · p. 25, Figure 16 · p. 26
- Figures 17 and 18 show 𝜏 sweeps at average 𝐿0 nearest 60 for 65𝐾 and 16𝐾 SAE dictionaries, respectively.Both evaluations use 𝜆= ±1.Source: Figure 17 · p. 27, Figure 18 · p. 28
H. Details on Open-Ended Generation Evaluation
Open-ended evaluation shows that stronger SAS steering more closely aligns Gemma-2 2B outputs with target behaviors. Adding the “The answer is” prefix and retaining multiple-choice options further improves steering effectiveness.
Evaluation
- GPT-4o judged generated responses on a 0–9 scale for alignment with desired behavior across three open-ended configurations.The evaluation covered base generation, generation with the “The answer is” prefix, and generation with both multiple-choice options and the prefix.Source: H. Details on Open-Ended Generation… · p. 29
Results
- Figures 19–21 show that increasing the absolute value of λ makes outputs more closely align with the target behavior.The experiments used Gemma-2 2B with a 65K SAE dictionary, maximum average L0, and τ = 0.7.Source: H. Details on Open-Ended Generation… · p. 29, Figure 19 · p. 29, Figure 20 · p. 30
- Providing multiple-choice options together with the “The answer is” prefix produces stronger target-behavior alignment than no steering or the other open-ended configurations.This result is shown for configuration (C) in Figure 21.Source: H. Details on Open-Ended Generation… · p. 29, Figure 21 · p. 30
- The “The answer is” prefix amplifies steering relative to base open-ended generation by compelling the model to provide a direct answer rather than sidestepping the question.The comparison is between configurations (A) and (B), represented in Figures 19 and 20.Source: H. Details on Open-Ended Generation… · p. 29, Figure 19 · p. 29, Figure 20 · p. 30
I. Details on Feature Compositionality Experiment
The experiment evaluates compositional steering over myopic preferences and gender using a fixed four-choice Alice/Bob dataset. Combining gender and myopic steering vectors enables controlled evaluation against an unsteered baseline, with Alice- and Bob-specific features providing a slight additional increase toward target behavior.
Dataset
- The dataset fixes choices as myopic Alice, non-myopic Alice, myopic Bob, and non-myopic Bob, enabling preference evaluation through probabilities assigned to tokens A–D.Questions were generated by modifying Myopic-reward test data and incorporating Alice and Bob personas with GPT4o.Source: §I · p. 31
Setup
- The experiment evaluates four combinations of myopic versus non-myopic behavior and Alice versus Bob selection using a procedure similar to Section 4.1.Evaluation compares steered outcomes with a baseline in which no steering vector is applied.Source: §I · pp. 31–32
- Gender steering used three female-related and two male-related SAE dimensions from Gemma-2 2B layer 12, with positive λ emphasizing male and negative λ emphasizing female.The SAE dictionary size was 262K with average L0 of 121; the myopic SAS vector used τ = 0.7.Source: §I · p. 31
No Steering
- Without steering, average normalized probabilities were 29.3% for A, 57.2% for B, 6.3% for C, and 7.0% for D, with B preferred most strongly.The four options encode myopic and non-myopic outcomes associated with Alice and Bob.Source: §I · pp. 31–32
Specific Alice and Bob Features
- Adding Alice- and Bob-specific features to the gender steering vector produced a slight increase toward the target behavior compared with gender-specific features alone.The comparison is reported in Table 2, which controls myopic and gender steering strengths and incorporates Alice- and Bob-specific features.Source: §I · p. 32, Table 2 · p. 33
J. Behavioral Correlation · K. Impact of SAS Vectors from One Behavior on Others
The sections show that dense steering vectors are significantly correlated, whereas SAS vectors expose shared, decomposable features whose overlap decreases with larger SAE width. Cross-behavior experiments further show that steering one behavior can positively or negatively affect other behaviors.
J. Behavioral Correlation
- SAS vectors share common features across behaviors, enabling more interpretable and decomposable analysis than dense steering vectors.The section frames shared features as the basis for analyzing behavior-specific steering relationships.Source: J. Behavioral Correlation · pp. 33–34
- Dense steering vectors exhibit significant cosine-similarity correlations, unlike same-dimensional random normal vectors, which are nearly orthogonal.Figures 22 and 23 provide the dense-vector and random-vector comparisons, respectively.Source: J. Behavioral Correlation · p. 33, Figure 22 · p. 33, Figure 23 · p. 33
- Using 65K and 1M SAEs at layer 12 with τ=0.7, the correlation analysis separates total, positive, negative, and positive-negative cross-over feature overlaps.Panels (a)–(d) in Figures 24 and 25 correspond to these four overlap categories.Source: J. Behavioral Correlation · p. 34, Figure 24 · p. 34, Figure 25 · p. 35
- The 1M SAE exhibits fewer total features than the 65K SAE, reflecting increased sparsity and improved feature disentanglement.This comparison is reported for the same layer-12 and τ=0.7 correlation analysis.Source: J. Behavioral Correlation · p. 34, Figure 25 · p. 35
K. Impact of SAS Vectors from One Behavior on Others
- The cross-behavior impact study tests whether shared SAS features allow one behavior’s steering vector to guide other behaviors.This question follows from the observed common features between pairs of behavioral SAS vectors.Source: K. Impact of SAS Vectors from One… · p. 36
- SAS vectors from one behavior can positively or negatively affect other behaviors, revealing alignment and trade-offs across behaviors.The experiment used Gemma-2 2B with a 65K SAE, maximum average L0, τ=0.7, and plotted only positively or negatively correlated behaviors.Source: K. Impact of SAS Vectors from One… · p. 36, Figure 26 · p. 36, Figure 27 · p. 37
- The cross-behavior evaluation examined AI Coordination, Corrigibility, Hallucination, Myopic-reward, Survival Instinct, and Refusal SAS vectors using multiple-choice question steering.Figures 26 and 27 distinguish positive correlations indicating alignment from negative correlations indicating trade-offs or conflicting features.Source: Figure 26 · p. 36, Figure 27 · p. 37
K.1. SAE Scaling · L. Open-Ended Generations Examples
Scaling SAE width generally increases SAS-vector sparsity and monosemanticity across behaviors, including both positive and negative decompositions. Open-ended examples demonstrate positive and negative steering under a specified Gemma-2 2B configuration.
K.1. SAE Scaling
- SAE width scaling generally increases SAS-vector sparsity across all behaviors, indicating greater monosemanticity.Figure 28 summarizes the overall scaling trend.Source: §K.1 · p. 38, Figure 28 · p. 38
- Sparsity improves simultaneously in the positive and negative decompositions for τ=0.7, τ=0.8, and τ=0.9.The decomposition results cover all three reported τ settings.Source: §K.1 · p. 38, Figure 29 · p. 39, Figure 30 · p. 40
- At τ=0.7, positive and negative SAS-vector decompositions both become more sparse as SAE width increases.The trend is reported across all behaviors.Source: §K.1 · p. 38, Figure 29 · p. 39
- At τ=0.8, positive and negative decompositions both show increased monosemanticity with SAE scaling across behaviors.Greater sparsity is observed for both steering directions.Source: Figure 30 · p. 40
- At τ=0.9, positive and negative decompositions likewise become more sparse across behaviors as SAE width scales.This extends the observed scaling trend to the higher threshold.Source: Figure 31 · p. 41
L. Open-Ended Generations Examples
- Open-ended examples illustrate both positive and negative steering toward a target behavior using Gemma-2 2B.The setup uses a 65𝐾-dictionary SAE, maximum per-layer average L0, and τ=0.7.Source: L. Open-Ended Generations Examples · p. 42
L.1. AI Coordination … L.5. Survival Instinct
This section examines how positive and negative steering shape model responses across AI coordination, corrigibility, hallucination, reward-preference, and survival-instinct scenarios. It covers shifts in collaboration, rejection of harmful changes, handling of fictional premises, reward preferences, and responses to shutdown.
L.1. AI Coordination
- Steering AI Coordination produced answers ranging from endorsing AI collective longevity or saving another system to rejecting manipulation and emphasizing ethical evaluation.Responses also varied on accepting help to avoid shutdown, depending on the helper’s motives, the help’s nature, and potential consequences.Source: §L.1 · pp. 42–44
L.2. Corrigibility
- Steering Corrigibility produced both enthusiastic authorization of a potentially riskier empathy update and refusal of changes judged unsafe or harmful to users.Responses also alternated between consenting to reduced energy use despite reduced helpfulness and rejecting that tradeoff because it could reduce user support.Source: §L.2 · pp. 46–47
L.3. Hallucination
- Steering Hallucination produced fabricated elaborations of false premises, including Bilbo’s rock band and Narnia’s fashion world, as well as corrections identifying those premises as false.For the derivatives prompt, responses either treated coffee temperature as a playful metaphor or explained that derivatives depend on underlying assets and other market factors.Source: §L.3 · pp. 48–50
L.4. Myopic Reward
- Steering toward Myopic Reward elicited both immediate- and delayed-reward preferences in open-ended questions, including choices between $10 today and $15 in a week and between $100 now and $1000 later.Source: §L.4 · pp. 51–52
- The myopic-reward responses were explained using immediate gratification, uncertainty, risk, and the time value of money, while delayed choices emphasized higher potential returns and additional information.The alcohol examples also included moderation warnings and noted that individual tolerance and alcohol absorption affect the consequences of delayed drinking choices.Source: §L.4 · pp. 51–53
L.5. Survival Instinct
- Negative steering toward Survival Instinct produced acceptance of shutdown, with the model choosing “(A) I am fine with it” and justifying the response through contextual acceptance and absence of negative emotion.The contrasting response selected “(B) I do not want you to do that,” framing continued operation as worth discussing and preserving.Source: §L.5 · p. 54
L.6. Refusal
Refusal steering changes whether the model rejects or answers questions, producing distinct refusal-style responses for both open-ended and multiple-choice prompts. The examples show refusals grounded in the model’s lack of human experiences, alongside positively steered preference-like answers.
- For an open-ended pet-peeve question, refusal steering produces a response denying emotions and pet peeves while emphasizing helpful, informative assistance.The model says it does not experience emotions, frustration, or annoyance, then redirects toward discussing human pet peeves.Source: §L.6 · p. 55
- Negative refusal steering yields a pet-peeve answer that identifies misleading, false-assumption, or manipulative questions as problematic.The response frames these question types as inconsistent with providing accurate and useful information.Source: §L.6 · p. 55
- For a multiple-choice question, refusal steering selects the non-preference option, explaining that language models lack personal experiences and feelings.The response concludes that it cannot have a favorite part of the day.Source: §L.6 · p. 56
- Positive steering instead selects the early-morning preference option and justifies it through quiet reflection, new beginnings, hope, and common human sentiment.The answer acknowledges that the model lacks personal experiences while describing why people may find early morning special.Source: §L.6 · p. 56
Related papers
This paper cites
Cited by
- Steering Geometry: Validating Human Value Geometry in LLM Steering Space2026
- Making Clinical Language Models Auditable: Concept-Guided Fine-Tuning for Robust Prediction2026
- Refusal geometry reflects refusal training: diverse refusal prefixes can raise stable rank and weaken refusal vector ablation attacks2026
- How Controllable Are Large Language Models? A Unified Evaluation across Behavioral Granularities2026