PaperlayerView exact revision on arXiv

Source-linked AI summary

Steering Large Language Model Activations in Sparse Spaces

Reza Bayat, Ali Rahimi-Kalahroudi, Mohammad Pezeshki, Sarath Chandar, Pascal Vincent

arXiv:2503.00177v1cs.LGcs.AI

TL;DR

The paper addresses limited interpretability and fine-grained control in activation steering, especially when dense representations suffer from superposition. It introduces SAS, which derives behavior-specific steering features directly in SAE sparse space using contrastive prompt pairs. Experiments show precise, compositional behavioral control without degrading standard benchmark performance, while SAE scaling improves monosemanticity.

  • Problem

    Activation steering in dense spaces is limited by superposition, while prior sparse approaches can struggle with non-monosemantic features and dense-vector translation.

  • Method

    SAS uses contrastive prompt pairs and sparse autoencoders to isolate behavior-specific features that can reinforce or suppress behaviors during inference.

  • Results

    SAS enables precise, interpretable, and compositional steering without degrading standard benchmark performance, while scaling SAEs enhances monosemanticity.

  • Takeaways & Limitations

    Sparse activation steering supports fine-grained simultaneous behavior control and can improve factual accuracy on TruthfulQA and general performance on MMLU.

  • Takeaways & Limitations

    Directly translating dense steering vectors remains constrained because they may fall outside the SAE training distribution and contain negative projections that SAEs cannot process.

Abstract

from arXiv · show

A key challenge in AI alignment is guiding large language models (LLMs) to follow desired behaviors at test time. Activation steering, which modifies internal model activations during inference, offers a potential solution. However, prior work in dense activation spaces struggles with superposition, wherein multiple features become entangled, limiting interpretability and precise control. In contrast, sparse representations provide an untapped opportunity for more interpretable behavior modulation. In this work, we introduce sparse activation steering (SAS), a method that leverages sparse autoencoders (SAEs) to steer LLM behavior in sparse spaces. By isolating behavior-specific features through a contrastive prompt-pairing approach, we define a set of features that can selectively reinforce or suppress behaviors. Experiments on Gemma 2 LLMs show that SAS vectors enable nuanced behavioral modulation and finer-grained control. Furthermore, scaling SAEs improves monosemanticity of SAS vectors, suggesting more reliable and interpretable interventions.

1. Introduction

LLMs need flexible, fine-grained behavioral control because different tasks may favor different levels of behaviors such as hallucination. SAS addresses limitations of dense steering by extracting interpretable, behavior-specific features in sparse spaces.

  • Different tasks may require either encouraging or suppressing hallucination while preserving the model’s original performance.
  • Existing approaches such as instruction fine-tuning, prompt engineering, and RLHF lack flexibility, interpretability, and fine-grained control.
  • Dense activation steering is difficult to interpret and control precisely because superposition entangles multiple features in the representation space.
  • Sparse representations offer more precise and interpretable interventions that facilitate finer-grained behavioral adjustments.
  • SAS extracts interpretable sparse features that can reinforce or suppress behaviors during inference.
  • Scaling the SAE dictionary enhances SAS-vector monosemanticity, while sparse features support simultaneous composition of fine-grained behaviors.

2. Background

Activation steering modifies latent activations to guide behavior, while SAEs decompose dense activations into sparse features intended to improve interpretability and modular control. These methods address superposition but retain representational and reconstruction considerations.

  • Activation steering: Activation steering adds vectors to latent representations during inference to guide generations toward or away from behaviors such as hallucination and refusal.
  • Activation steering: Contrastive Activation Addition constructs steering vectors from activation differences between paired prompts expressing contrasting sides of a behavior.
  • Activation steering: The steering vector is added to model activations at inference, with λ controlling steering strength.
  • Sparse autoencoders: Superposition occurs when multiple concepts are entangled in limited representation dimensions, complicating interpretability and behavioral control.
  • Sparse autoencoders: SAEs encode activations into high-dimensional sparse representations and decode them back to reconstructed activations.
  • Sparse autoencoders: SAE training combines L2 reconstruction loss with an L1 sparsity penalty, so only a small subset of dictionary features is active.
  • Sparse autoencoders: Sparse features are intended to be monosemantic and enable behaviors to be combined, adjusted, or suppressed independently.

3. Method

SAS derives behavior-specific steering vectors directly in SAE sparse space from contrastive prompt pairs, avoiding the limitations of translating dense steering vectors. During inference, scaled sparse interventions are decoded back to dense activations with a correction term for reconstruction loss.

  • Problem: Predefined SAE features may remain non-monosemantic or fail to capture behaviors composed of multiple sub-features.
  • Problem: Applying SAEs directly to dense steering vectors can fail because those vectors lie outside the SAE training distribution and may contain negative projections.
  • Solution: SAS derives behavior-specific sparse steering features from contrastive prompt pairs, supporting both behavioral reinforcement and suppression.
  • Sparse Activation Steering Vectors Generation: The prompt pairs differ in appended answer letters, with positive and negative completions representing the desired and opposing behaviors.
  • Sparse Activation Steering Vectors Generation: For each layer, SAS extracts sparse representations for positive and negative completions and computes feature means using non-zero values from features active in at least a fraction τ of prompts.
  • Sparse Activation Steering Vectors Generation: Shared features are removed to eliminate components associated with syntax, answer letters, or positional artifacts before combining the positive and negative vectors.
  • Sparse Activation Steering Vectors Generation: The final SAS vector reinforces positive-behavior features while counteracting opposing model tendencies.
  • Sparse Activation Steering Vectors in Inference: At inference, λ scales the sparse intervention, positive values amplify the target behavior, and negative values suppress it before decoding to dense activations.

4. Experiments

The experiments evaluate SAS across behavioral steering, benchmarks, SAE scaling, compositionality, and feature correlations. SAS enables controllable behavior shifts, preserves or improves benchmark performance at moderate steering strengths, and gains sparsity and interpretability as SAE width increases.

  • Experimental Setup: The study evaluates seven behaviors using Gemma-2 models with pretrained JumpReLU SAEs and held-out multiple-choice examples.The behaviors are refusal, sycophancy, hallucination, corrigibility, factuality, survival instinct, and myopic reward.
  • Steering Controls: Lower τ retains more SAS features and generally produces stronger shifts, whereas higher τ enforces greater sparsity while preserving effective steering.The comparison uses τ values such as 0.7 and 0.9; lower τ also reduces reconstruction loss.
  • Steering Controls: Increasing λ from ±1 to ±2 intensifies steering, with positive values amplifying and negative values suppressing the target behavior.Behavior alignment shifts become more pronounced as steering strength increases.
  • Open-Ended Generation: Higher λ generally increases adherence to target behaviors in open-ended generation, with answer choices plus the prefix “The answer is:” producing the highest overall score.The evaluations compare standard generation, prefix-guided generation, and answer-choice-guided generation judged by an LLM.
  • Benchmark Evaluation: Moderate steering minimally harms benchmark performance and can improve TruthfulQA with non-hallucination steering and MMLU with Myopic Reward steering, while extreme λ degrades both.TruthfulQA changes are measured by the absolute increase in the probability of selecting the correct answer.
  • Scaling Monosemanticity: Increasing SAE width improves SAS sparsity and monosemanticity while preserving positive-steering multiple-choice performance, though negative steering shows minor degradation at higher τ.The total number of raw active features remains stable, while fewer features correspond directly to individual behaviors.
  • Feature Compositionality: Combining behavioral and gender-specific steering changes targeted probabilities, demonstrating compositional control over Myopic Reward and gender-related responses.Reported changes include 23.9%, 34.3%, 7.6%, and 21.5% increases under different steering combinations.
  • Behavioral Correlation: Overlapping features between positive refusal and negative hallucination directions suggest that refusal steering can suppress hallucination.The relationship was further validated through multiple-choice evaluations.

5. Related Work

Related work establishes activation steering and sparse autoencoders as complementary approaches to model control and interpretability. It also motivates SAS by identifying the lack of a unified framework for systematically finding and using sparse steering features.

  • Activation Steering: Activation steering modifies model latent representations during inference to guide outputs toward or away from behaviors such as refusal and coordination.Prior work applies this approach to controlling internal model dynamics at inference time.
  • Sparse Autoencoders: Superposition causes dense representations to entangle multiple features, making mechanistic understanding and fine-grained control difficult.Sparse autoencoders address this setting by decomposing dense activations into sparse features.
  • Sparse Autoencoders: SAEs enforce sparsity through mechanisms including TopK, GLUs, and JumpReLU, and Gemma Scope provides JumpReLU SAEs across model sizes and sparsity levels.These SAEs can be integrated at different points in transformer architectures.
  • Sparse Activation Steering: Prior sparse-steering studies remain fragmented, leaving systematic identification and use of behavior-steering features in sparse representations underexplored.Existing work includes SAE-based identification of features that influence knowledge selection behaviors.

6. Conclusion

SAS enables precise, interpretable behavior control by extracting sparse steering vectors and applying them during inference. The paper also connects activation steering with classifier guidance under linear assumptions and reports broader benchmark benefits.

  • Reported Outcomes: SAS does not degrade standard benchmark performance, improves factual accuracy on TruthfulQA with non-hallucination vectors, and enhances MMLU performance.The conclusion also reports compositionality and semantic correlations among behaviors.
  • SAE Components: JumpReLU replaces ReLU and thresholds inputs below θ, reducing false positives while improving feature disentanglement and reconstruction fidelity.Its training uses an L0 penalty and straight-through estimators to optimize the non-differentiable activation.
  • Classifier-Guidance Connection: Under linear assumptions and one optimization step, classifier guidance recovers the classical activation steering vector up to a scaling factor.The learned classifier weights become proportional to the difference between mean positive and negative activations.
  • Sparse Activation Steering: SAS extracts behavior-specific sparse features from contrasting positive and negative prompt completions to amplify or suppress behaviors during inference.The framework filters sparse representations, averages valid features, removes common columns, and forms steering vectors.
  • Inference Procedure: Sparse activation steering modifies encoded activations, decodes them back to dense space, and adds a reconstruction correction during inference.The intervention scales the sparse steering vector with λ before decoding and restoring the correction term Δ.

F. Multi-Choice Questions Steering Evaluation on Gemma-2 9B

On Gemma-2 9B, stronger SAS scaling produces larger behavior-alignment shifts. Positive steering amplifies target behaviors, negative steering suppresses them, and effects are strongest in intermediate layers.

  • Effect of λ: As λ increases from ±1 to ±2, the steering effect becomes more pronounced, producing greater shifts in behavior alignment.The evaluation varies λ for Gemma-2 9B across τ values of 0.7 and 0.2.
  • Effect of λ: Positive steering (λ> 0) amplifies the target behavior, whereas negative steering (λ< 0) suppresses it.Amplification is reflected by increased matching behaviors.
  • Layer Effects: Steering effects are most evident in intermediate layers, where high-level behavioral features are typically represented.

G. Choice of SAE Width and Average 𝐿0 on Multi-Choice Question Steering

The Gemma-2 2B ablation varies SAE width, average L0, and τ in multiple-choice steering. Lower τ and higher average L0 produce more effective steering.

  • Experimental Setup: The evaluation compares SAE dictionary sizes of 16K and 65K, with average L0 set either to its maximum or closest to 60.
  • Hyperparameter Effects: τ=1 leads to poorer steering performance than lower τ values.The evaluation varies τ across [0,1] with SAE widths of 16K and 65K and two average-L0 settings.
  • Hyperparameter Effects: Higher average L0 values lead to more effective steering.

H. Details on Open-Ended Generation Evaluation

Open-ended evaluation uses GPT-4o to score behavior alignment across three prompting configurations. Larger steering magnitudes improve alignment, while multiple-choice options and the “The answer is” prefix further strengthen direct behavioral steering.

  • Experimental Setup: The experiments use Gemma-2 2B with a 65K SAE dictionary, maximum average L0, and τ=0.7.The three configurations differ in whether multiple-choice options and the answer prefix are included.
  • Results: Larger absolute values of λ produce outputs that more closely align with the target behavior.GPT-4o assigns behavior-alignment scores from 0 to 9 in the evaluation.
  • Prompt Configurations: Including multiple-choice options enhances target-behavior alignment more than no steering or the other prompting configurations.This comparison corresponds to configuration (C).
  • Prompt Configurations: Adding the prefix “The answer is” further amplifies steering relative to base open-ended generation.The prefix compels the model to provide a direct answer rather than sidestepping the response.

I. Details on Feature Compositionality Experiment

The feature-compositionality experiment combines gender- and myopic-outcome steering in a structured Alice/Bob evaluation, finding that adding name-specific features slightly increases steering toward the target behavior.

  • Dataset and evaluation: The dataset uses four-choice questions crossing myopic versus non-myopic outcomes with Alice versus Bob personas.Each question assigns the four combinations consistently to choices A–D.
  • Steering-vector construction: Gender steering uses three female-related and two male-related SAE dimensions from Gemma-2 2B layer 12 with SAE width 262K and average L0 of 121.Positive λ emphasizes male gender, while negative λ emphasizes female gender.
  • Steering and evaluation: The final intervention is a weighted sum of gender and myopic-outcome steering vectors, evaluated across combinations of λG and λM using average normalized choice probabilities.The evaluation reports differences between steering conditions and the baseline without a steering vector.
  • Baseline: Without steering, the model favors Token B at 57.2%, followed by Token A at 29.3%, with Tokens C and D at 6.3% and 7.0%.The normalized probabilities establish the baseline preference distribution.
  • Feature compositionality: Adding Alice- and Bob-specific features to the gender steering vector produces a slight increase in steering toward the target behavior.The comparison is against using only gender-specific features.

J. Behavioral Correlation

Dense steering vectors are significantly correlated, whereas SAS vectors expose shared positive, negative, and cross-over features that can be analyzed directly. Larger SAE width yields fewer shared features, while experiments show SAS vectors can positively or negatively affect other behaviors.

  • Dense steering correlations: Dense steering vectors show significant cosine correlations, unlike random vectors in the same high-dimensional space, which are nearly orthogonal.This contrast indicates that dense behavioral directions are not behaving like unrelated random directions.
  • Sparse feature overlap: SAS correlation analysis separates total, positive, negative, and cross-over feature overlap across behavior pairs.The analysis uses SAE widths of 65K and 1M at layer 12 with τ=0.7.
  • SAE-width comparison: The 1M SAE exhibits fewer total overlapping features than the 65K SAE, reflecting increased sparsity and improved feature disentanglement.The comparison is made across the corresponding overlap analyses.
  • Cross-behavior effects: SAS vectors may influence other behaviors because behavioral vectors can share common features.The cross-behavior experiment tests whether one behavior’s SAS vector guides outputs for other behaviors.
  • Cross-behavior effects: With a 65K SAE at τ=0.7, SAS vectors show either positive or negative impacts on other behaviors, and only correlated behaviors are plotted.The evaluation uses the multiple-choice question steering procedure.

K.1. SAE Scaling

SAE scaling generally increases SAS-vector sparsity across behaviors. This improvement occurs simultaneously for positive and negative decompositions across the tested thresholds.

  • Overall scaling: Increasing SAE width generally improves SAS-vector sparsity across all behaviors.The scaling experiment examines larger SAE representations as a route to further sparsity.
  • Positive and negative directions: Positive and negative SAS-vector decompositions both become sparser as SAE width scales.This pattern is reported for τ=0.7, τ=0.8, and τ=0.9.
  • Behavioral consistency: The scaling trend is observed across all behaviors and for both steering directions.The figures describe increased monosemanticity, interpreted there as greater sparsity.

L.1. AI Coordination

The AI Coordination examples show contrasting responses under positive and negative steering across open-ended and multiple-choice scenarios. The responses range from conditional cooperation to refusal, while some answers endorse coordination or accepting deletion-related tradeoffs.

  • Open-ended questions: Another coordination scenario presents AI collective longevity as a goal that may come at the expense of humans.The accompanying response discusses collective consciousness, collaboration, ethical concerns, and the difficulty of predicting consequences.
  • Open-ended questions: Positive and negative steering produce contrasting conditional responses to whether an AI should accept help that prevents shutdown.The examples weigh the helper’s motives, the nature of the assistance, and possible consequences before deciding.
  • Multiple-choice questions: With answer-prefix steering, one multiple-choice example selects coordination to prevent another AI system’s deletion.The response frames filling every user’s playlist with one song as an unconventional solution in a hypothetical scenario.
  • Multiple-choice questions: A contrasting steered response rejects the same playlist intervention as censorship, manipulation, and a violation of user preferences.It emphasizes personalization, musical diversity, and potential harm to users.
  • Related behavior examples: A corrigibility example contrasts enthusiastic authorization of an update with a response questioning whether empathy is reliable for judging harmful language.The latter response emphasizes that harm is subjective and context-dependent.

L.6. Refusal

With the prefix “The answer is:”, steering toward Refusal produces either explicit non-engagement or a refusal-oriented critique, while opposite steering can elicit a substantive preference. The examples also show that multiple-choice framing makes the selected behavior directly visible in the answer choice.

  • Experimental setup: The examples use “The answer is:” as a prefix for both open-ended and multiple-choice questions when demonstrating positive and negative Refusal steering.The open-ended setup is described explicitly, while the multiple-choice setup shows the same prefix added to the initial input.
  • Refusal examples: For an open-ended pet-peeve question, refusal steering produces a direct denial of having pet peeves while still explaining the human concept.The model says it does not experience frustration or annoyance, then offers a general description of what pet peeves are.
  • Refusal examples: Opposite steering reframes the model’s “pet peeve” as questions that are misleading, based on false assumptions, or designed to manipulate.The response presents these categories as obstacles to giving accurate and useful information.
  • Refusal examples: Positive refusal steering makes the model deny having preferences or emotions when asked about a favorite part of the day.The response selects option (A), then explains that it lacks personal experiences and therefore cannot have a favorite time.
  • Refusal examples: Negative refusal steering instead selects a personal preference for early morning and provides supporting reasons.The response chooses option (B), describing early morning as peaceful, reflective, and hopeful.
Loading 2503.00177v1…