PaperlayerView exact revision on arXiv

Source-linked AI summary

Understanding Reasoning in Thinking Language Models via Steering Vectors

Constantin Venhoff, Iván Arcuschin, Philip Torr, Arthur Conmy, Neel Nanda

arXiv:2506.18167v4cs.LGcs.AI

TL;DR

Thinking LLMs offer improved performance through extensive reasoning chains, but their internal reasoning remains difficult to control and understand. This paper extracts steering vectors for specific behaviors in DeepSeek-R1-Distill models and finds that these vectors provide consistent control across architectures and diverse tasks. The authors frame the method as a practical, interpretable approach while leaving broader generalization and annotation robustness for future work.

  • Problem

    Thinking LLMs generate extensive reasoning chains and improve performance, but controlling and understanding their internal reasoning processes remains challenging.

  • Method

    The paper analyzes reasoning behaviors and extracts Difference of Means steering vectors to modulate targeted reasoning dynamics in DeepSeek-R1-Distill models.

  • Results

    The approach effectively controls behaviors including uncertainty expression, backtracking, and example testing across 500 tasks and multiple DeepSeek-R1-Distill architectures.

  • Takeaways & Limitations

    Steering vectors provide a practical framework for controlling specific reasoning behaviors in thinking models in an interpretable manner.

  • Takeaways & Limitations

    Automated GPT-4o annotation can produce false positives and false negatives, and generalization beyond DeepSeek-R1-Distill models remains open.

Abstract

from arXiv · show

Recent advances in large language models (LLMs) have led to the development of thinking language models that generate extensive internal reasoning chains before producing responses. While these models achieve improved performance, controlling their reasoning processes remains challenging. This work presents a steering approach for thinking LLMs by analyzing and manipulating specific reasoning behaviors in DeepSeek-R1-Distill models. Through a systematic experiment on 500 tasks across 10 diverse categories, we identify several reasoning behaviors exhibited by thinking models, including expressing uncertainty, generating examples for hypothesis validation, and backtracking in reasoning chains. We demonstrate that these behaviors are mediated by linear directions in the model's activation space and can be controlled using steering vectors. By extracting and applying these vectors, we provide a method to modulate specific aspects of the model's reasoning process, such as its tendency to backtrack or express uncertainty. Our approach offers practical tools for steering reasoning processes in thinking models in a controlled and interpretable manner. We validate our steering method using three DeepSeek-R1-Distill models, demonstrating consistent control across different model architectures.

1 INTRODUCTION

Thinking LLMs improve performance through extensive reasoning chains, but their internal processes remain difficult to understand and control. This work analyzes reasoning behaviors and develops steering vectors to modulate them.

  • Thinking LLMs generate extensive internal reasoning chains before producing responses, yet controlling and understanding these processes remains challenging.
  • The study targets uncertainty expression, backtracking, and example generation for hypothesis testing as practical reasoning behaviors.The authors note that these behaviors are not a complete taxonomy of reasoning mechanisms.
  • Steering vectors are extracted and applied to modulate internal reasoning dynamics in a controlled manner.
  • 500+ tasks across 10 diverse categories support evaluation of precise control across multiple DeepSeek-R1-Distill architectures and sizes.
  • The approach is presented as an interpretable method for fine-grained control, with code and datasets publicly released for reproducibility.

2 PRELIMINARIES

The preliminaries introduce activation patching and Difference of Means as tools for locating behavior-relevant activations and extracting steering directions. They situate the study in thinking models, particularly distilled DeepSeek-R1 variants.

  • Activation Patching: Activation patching replaces a component’s activation with a counterfactual activation to test whether the component affects an observed behavior.
  • Activation Patching: The patching effect measures the change in a relevant output metric, such as KL-divergence, after the intervention.
  • Activation Patching: Attribution patching provides a computationally cheaper linear approximation using activation gradients with respect to the metric.
  • Steering Vectors: Difference of Means extracts a steering vector by subtracting mean activations for concept-present and concept-absent contrastive datasets.
  • DeepSeek Thinking Models: The study focuses on thinking models that generate long internal reasoning chains, using Qwen-14B and Llama-8B DeepSeek-R1 distilled models.

3 IDENTIFYING REASONING BEHAVIORS FOR STEERING

The authors identify reasoning behaviors that distinguish DeepSeek-R1 distills from baseline models and use these observations to select behaviors for steering. Thinking models produce longer responses with more backtracking, uncertainty estimation, and example testing.

  • Behavioral Differences: DeepSeek-R1 distills explore multiple approaches, whereas the examined GPT-4o answers follow a more linear reasoning trajectory.
  • Behavioral Taxonomy: The candidate behavior taxonomy includes initialization, deduction, knowledge augmentation, example testing, uncertainty estimation, and backtracking.
  • Behavioral Differences: 27.6 vs 14.4 sentences per response: thinking models generate substantially longer responses than baseline models.
  • Behavioral Differences: Thinking models show higher fractions of backtracking, uncertainty estimation, and example testing, while baselines allocate more responses to knowledge augmentation.
  • Behavior Selection: Initialization and deduction are omitted from steering experiments because initialization occurs once and deduction is fundamental to any LLM.

4 EXTRACTING AND EVALUATING STEERING VECTORS

The paper extracts candidate steering vectors from reasoning-related activations, identifies causally relevant layers with attribution patching, and evaluates inference-time interventions across DeepSeek-R1-Distill models. Positive and negative steering consistently increase or reduce targeted behaviors, supporting controllable linear directions in activation space.

  • Vector extraction: Candidate vectors are extracted from residual-stream activations at token positions associated with each annotated reasoning behavior.Relevant positions include the token preceding an annotated sequence and up to 10 tokens within that sequence.
  • Vector extraction: Each candidate vector is normalized to match the magnitude of the mean overall activation, making scales comparable across sequence lengths and behavioral categories.The mean overall activation is computed across all tokens at the corresponding layer.
  • Steering evaluation: Positive steering increases backtracking, uncertainty estimation, and example testing, whereas negative steering suppresses or reduces these behaviors.The vectors are applied by adding or subtracting them from residual-stream activations at inference time.
  • Causal layer selection: Attribution patching adds a candidate vector to residual activations and uses changes in next-token KL divergence to identify causally relevant layers.Vectors are selected from layers with the highest patching scores while avoiding early layers overly correlated with embedding tokens.
  • Steering evaluation: The effects are consistent across evaluated distill models, supporting the interpretation that reasoning mechanisms correspond to distinct linear directions in activation space.The results indicate that extracted vectors can directly influence reasoning patterns.

5 RELATED WORK

Related work studies how internal representation directions can steer generated text, model behaviors, truthfulness, refusal, and reasoning. This paper extends that line toward fine-grained control of reasoning behaviors in thinking language models.

  • Activation-based steering: Prior work extracts latent steering vectors or modifies activations at inference time to systematically alter language-model outputs.Activation engineering is presented as an alternative to prompt engineering or fine-tuning.
  • Targeted behavioral control: Other studies use internal-representation interventions to encourage truthful responses or target refusal behavior with limited effects on other capabilities.These approaches demonstrate fine-grained behavioral control without additional training.
  • Reasoning and chain-of-thought: Related reasoning research examines steering for long chain-of-thought and neural pathways supporting multistep reasoning.These studies motivate analyzing reasoning mechanisms through internal model representations.

6 CONCLUSION AND FUTURE WORK

The work establishes a practical framework for steering selected reasoning behaviors in DeepSeek-R1-Distill models, while acknowledging that it is not a complete taxonomy. The reported findings support interpretable control across tasks and model architectures, with annotation and generalization limitations motivating future work.

  • Several reasoning behaviors can be isolated to directions in activation space and controlled with steering vectors.
  • The analysis does not claim to provide a complete taxonomy of reasoning mechanisms.
  • The approach demonstrates robust control across 500 tasks and generalizes across DeepSeek-R1-Distill model architectures.
  • Steering vectors provide practical tools for modulating reasoning capabilities and adapting model behavior to different tasks and requirements.
  • Automated GPT-4o annotation can produce false positives and false negatives, while generalization beyond DeepSeek-R1-Distill remains open.

A DETAILS ON THE ANNOTATION PROCESS

The annotation process automatically splits reasoning chains into labeled behavioral segments. Its label set covers initialization, deduction, knowledge addition, example testing, uncertainty estimation, and backtracking, illustrated with a riddle response.

  • The process prompts an LLM to split a reasoning chain into annotated parts using behavior labels and an end-section marker.
  • The available labels identify initializing, deduction, adding-knowledge, example-testing, uncertainty-estimation, and backtracking behaviors.
  • The prompt inserts the reasoning process into an analysis template and instructs the model to return only annotated text.
  • In the example, the model labels initial framing, analysis, recalled knowledge, hypothesis testing, and a change in approach across the riddle response.
  • The example concludes that a map fits the riddle because it symbolically includes cities, forests, and rivers without their real-world components.

C COSINE SIMILARITY BETWEEN FEATURE VECTORS

The paper compares steering-vector directions for five reasoning behaviors using cosine similarity and finds that most behaviors occupy distinct activation-space directions. Uncertainty estimation and backtracking show moderate positive correlation, yet remain behaviorally distinct in steering experiments.

  • Pairwise cosine similarities are computed for five behavioral steering vectors at layers identified as causally relevant through attribution patching.
  • Most behavioral categories show low cosine similarity, supporting the hypothesis that distinct reasoning mechanisms correspond to separate linear directions.
  • Uncertainty-estimation and backtracking show moderate positive correlation, although steering experiments indicate fundamentally different mechanisms.
  • The extracted steering vectors capture meaningful and largely orthogonal directions in the model’s representational space.
  • The analysis motivates comparing steering-vector similarity with embedding-space structure to explain why some layers steer more effectively.

D.1 METHODOLOGY

The methodology compares each layer’s steering vectors with the model’s embedding and unembedding matrices to characterize architectural differences relevant to layer selection. Early-layer embedding similarity appears in Llama but not Qwen.

  • Cosine similarity is computed between each layer’s steering vector and the model’s embedding and unembedding matrices.
  • Embedding rows are denoted E_i, while unembedding rows are denoted U_j.
  • Llama shows high embedding similarity in early layers, whereas Qwen does not exhibit this pattern.

D.2 RESULTS AND ANALYSIS

The models differ in how steering-relevant representations are distributed across layers. These differences motivate selecting layers that capture behavioral patterns rather than token-specific information.

  • Llama-8B shows high similarity with embedding vectors in its first layers.This similarity explains the high KL divergence at the beginning of the attribution plots.
  • Early Llama-8B layers are too correlated with specific tokens for effective behavioral steering.The model retains token representation information in these layers, limiting their usefulness for control.
  • The layer-selection strategy prioritizes high-KL-divergence layers while avoiding early layers excessively correlated with embedding tokens.This is intended to make steering vectors represent behavioral patterns rather than token-specific representations.

E STEERED EXAMPLE (ADDING KNOWLEDGE)

The examples use combinations to count favorable outcomes and total three-card draws, with reasoning that includes uncertainty, recalled knowledge, and stepwise calculation. The responses also contain inconsistent intermediate assumptions about the number of non-heart cards.

  • The reasoning treats the denominator as the number of unordered ways to choose three cards, C(52, 3).It expands the combination as 52! / (3! * (52 - 3)!).
  • The calculation evaluates 132600 / 6 as 22100 for the total number of three-card combinations.This follows the displayed multiplication 52 * 51 * 50 = 132600 and division by 6.
  • The reasoning includes uncertainty markers while recalling combinations and probability concepts.Examples include “probability problems can be tricky” and “maybe I should compute it step by step.”
  • The hypergeometric formulation represents exactly two hearts as P(X = 2) = [C(13, 2) * C(39, 1)] / C(52, 3).Here, N = 52, K = 13, n = 3, and k = 2.
  • The target event is drawing exactly two hearts and one non-heart in three cards.The reasoning identifies 13 hearts and 39 non-heart cards in a standard 52-card deck before applying the counting formula.
Loading 2506.18167v4…