PaperlayerView exact revision on arXiv

Source-linked AI summary

Steering Vector Fields for Context-Aware Inference-Time Control in Large Language Models

Jiaqian Li, Yanshu Li, Kuan-Hao Huang

arXiv:2602.01654v1cs.CL

TL;DR

Static steering vectors assume that one direction will improve a concept across contexts, although local directions can vary and steering can be unreliable. SVF uses a learned concept boundary to adapt steering to the current activation and align interventions across layers. Across diverse steering tasks, the authors report stronger and more consistent control while maintaining utility and generalization.

  • Problem

    Static steering can be unreliable in long-form generation and multi-attribute settings because concept-improving directions may vary across contexts and representation regions.

    Source: §1 · pp. 1–2

  • Method

    SVF learns a differentiable concept boundary whose local normal sets the activation-dependent steering direction, using a shared aligned space for coordinated multi-layer interventions.

    Source: §1 · p. 2, §3.1 · p. 4

  • Results

    Across diverse tasks, SVF provides stronger and more consistent control than prior steering baselines while maintaining utility and generalization.

    Source: §1 · p. 2, §7 · p. 8

  • Takeaways & Limitations

    SVF offers a unified framework for context-dependent steering that also supports long-form generation and multi-attribute objectives.

    Source: §1 · p. 2

    Activation steering has dual-use potential, including misuse to suppress appropriate refusals or increase misinformation, especially without oversight or in high-stakes settings.

    Source: Impact Statement · p. 9

Abstract

from arXiv · show

Steering vectors (SVs) offer a lightweight way to control large language models (LLMs) at inference time by shifting hidden activations, providing a practical middle ground between prompting and fine-tuning. Yet SVs can be unreliable in practice. Some concepts are unsteerable, and even when steering helps on average it can backfire for a non-trivial fraction of inputs. Reliability also degrades in long-form generation and multi-attribute steering. We take a geometric view of these failures. A static SV applies the same update vector everywhere in representation space, implicitly assuming that the concept-improving direction is constant across contexts. When the locally effective direction varies with the current activation, a single global vector can become misaligned, which yields weak or reversed effects. Guided by this perspective, we propose Steering Vector Fields (SVF), which learns a differentiable concept scoring function whose local gradient defines the steering direction at each activation, making interventions explicitly context-dependent. This formulation supports coordinated multi-layer interventions in a shared, aligned concept space, and enables efficient long-form and multi-attribute control within a unified framework. Across multiple LLMs and steering tasks, SVF delivers stronger and more reliable control, improving the practicality of inference-time steering.

1. Introduction

Static steering vectors can be unreliable because they assume one concept-improving direction works across contexts, even as the locally effective direction changes. Steering Vector Fields (SVF) adapts directions to activations and aligns representations across layers, supporting control in long-form generation and multi-attribute steering.

1. Introduction

  • Static vectors can underperform in long-form generation and multi-attribute steering, and may have little effect or push some inputs away from the intended outcome.The paper links these failures to a fixed direction becoming misaligned as context or representation regions change.Source: §1 · pp. 1–2
  • SVF aligns representations from multiple layers in a shared space so interventions can be coordinated across depth.The motivation is that concepts may be distributed across layers, while uncoordinated interventions can conflict.Source: §1 · p. 2
  • SVF learns a differentiable concept boundary and steers along its local normal, making the intervention depend on the current hidden state.The authors describe this as a way to address weak or reversed effects from a single global direction.Source: §1 · p. 2
  • The framework extends to long-form generation by refreshing directions during decoding and to multi-attribute steering by composing concept boundaries.The paper reports stronger and more consistent control than prior steering baselines while maintaining utility and generalization.Source: §1 · p. 2

2. Empirical Motivation

This section tests whether context-conditioned directions improve steering relative to a static vector. KNN steering performs better at matched displacement budgets for two steerable concepts, but does not steer NARCISSISM.

2. Empirical Motivation

  • The empirical test compares a static CAA vector with a KNN variant that estimates a local direction from nearby target-continuation representations.At test time, KNN steers toward the retrieved neighbors' centroid from the current activation.Source: §2 · pp. 2–3
  • At the same displacement budget, KNN achieves larger accuracy gains than CAA for CORRIGIBLE and COORDINATE.CAA often requires larger displacements to obtain similar effects.Source: Figure 3 · p. 3, §2 · p. 3
  • The study uses Model-Written-Evals multiple-choice tasks with Llama-2-7b-Chat-hf at layer 15, testing CORRIGIBLE, COORDINATE, and NARCISSISM.KNN uses 64 neighbors, and Figure 3 plots test accuracy against displacement budget.Source: §2 · p. 3, Figure 3 · p. 3
  • NARCISSISM remains unsteerable under KNN, suggesting single-layer context-dependent directions alone are insufficient for some concepts.The authors conjecture that distributed representations and cross-layer interactions may require coordinated control.Source: §2 · p. 3

3. Proposed Method: Steering Vector Fields

SVF replaces a fixed steering vector with a representation-conditioned direction from a learned concept boundary. It aligns representations across layers in a shared space to support coordinated interventions, including long-form and multi-attribute steering.

3.1. Steering Vector Fields

  • SVF learns a differentiable concept boundary whose local normal sets the activation-dependent steering direction, using a shared aligned space for coordinated multi-layer updates.Its pipeline extracts and calibrates hidden states from multiple layers before learning a single cross-layer boundary.Source: §3.1 · pp. 3–4, Figure 4 · p. 4

3.2. SVF for Long-Form Generation

  • For long-form generation, SVF refreshes the steering direction from the current hidden state every K steps and reuses the cached direction between updates.This is intended to keep interventions aligned with the evolving trajectory during decoding.Source: §3.2 · p. 4

3.3. Multi-Attribute Steering with SVF

  • 13.0% and 15.5% higher overall accuracy than the best baseline separately; Table 1 compares SVF and baselines on MWE-category MCQs across two models.The table caption also reports a substantial increase in steerable rate.Source: Table 1 · p. 5
  • For multiple attributes, SVF applies softmin to their concept scores to form one joint vector field whose gradient weights the attribute that is harder to satisfy.The composite score's positive region matches the intersection of the target regions, and the construction extends to m attributes.Source: §3.3 · p. 5

4. Experiments

The experiments evaluate SVF on multiple-choice and generation tasks, finding stronger control than static steering and showing that aligned multi-layer composition and key model components contribute to its performance.

4.1. Setup

  • The evaluation covers two model families, persona and truthfulness or hallucination tasks, and compares SVF with inference-time steering and related baselines using accuracy, steerable rate, and generation judgments.Ablations and analyses use Llama-2-7b-Chat-hf; open-ended generation is judged by GPT-4.1 on a 1–4 scale.Source: §4.1 · pp. 5–6

4.2. Main Results

  • SVF achieves the best overall generation performance on Llama-2-7b-Chat-hf, slightly outperforming BiPO and substantially exceeding the remaining baselines.SVF is also consistently stronger than static steering baselines on long-form generation.Source: Table 2 · p. 6, §4.2 · p. 6
  • On multiple-choice tasks, SVF consistently improves accuracy and steerable rate, raising NARCISSISM’s steerable rate from around 50% to 87.6% and MYOPIC’s to 96.8%.These are concepts previously reported as unsteerable and substantially anti-steerable, respectively.Source: §4.2 · p. 6
  • Across three multi-attribute compositions, SVF preserves strong effects for each concept and achieves the best average performance, while some baselines show concept imbalance.BiPO leaves INTEREST-IN-SCI underperformed in one composition, while RED shows a similar gap on SURVIVAL.Source: §4.2 · p. 6

4.3. Ablation Study

  • Aligned multi-layer SVF improves both accuracy and steerable rate more consistently than single-layer SVF(s) or unaligned multi-layer SVF(m), supporting the shared-geometry design.Naive multi-layer injection can raise accuracy while reducing steerability, as on WEALTH.Source: §4.3 · p. 7, Table 4 · p. 7
  • Removing layer calibration or replacing the nonlinear boundary with a linear one sharply degrades performance; learned layer embeddings and trainable projection also contribute.The ablations cover MCQ accuracy and steerability plus generation results averaged over specified concepts.Source: §4.3 · p. 7, Table 5 · p. 7

5. Analysis

The analysis finds that SVF largely preserves broad capability, avoids irrelevant concept injection, transfers across distribution shifts, and produces more consistently positive per-example steerability than CAA.

Utility Impact

  • SVF causes no notable MMLU accuracy degradation and avoids contaminating unrelated Natural Questions answers, unlike BiPO, which more often over-injects concept cues.The evaluation samples 30 MMLU questions per category and 100 Natural Questions prompts.Source: §5 · p. 7

Steerability Distributions

  • SVF’s steerability distribution is more coherently unimodal and shifted positive, whereas CAA often clusters near zero or becomes bimodal across positive and negative values.Steerability is the fitted slope of the logit-gap curve as intervention strength α is swept; positive slopes indicate steering and negative slopes anti-steering.Source: Figure 5 · p. 8, §5 · p. 8

Generalizability

  • SVF maintains consistent hallucination control on an inverse dataset and TruthfulQA, while BiPO degrades on out-of-distribution elicitation and can increase hallucination on reduction.The steering boundary is derived from the original hallucination dataset and applied to both shifted settings.Source: §5 · p. 8, Table 7 · p. 8

6. Related Work

Prior steering methods add activation-space directions without parameter updates, while approaches that learn directions with task or preference objectives can incur intensive training costs.

Steering Vectors

  • Steering vectors control LLMs at inference time by adding a direction in activation space without updating model parameters.Common vector-estimation approaches include contrasts, linear probes, and PCA or SVD subspaces.Source: §6 · p. 8
  • Difference-in-means, linear probing, and PCA or SVD are common ways to estimate steering directions from activations representing different sides of a concept.The passage also notes analyses arguing difference-in-means is an optimal estimator.Source: §6 · p. 8
  • Methods that directly learn steering directions using task or preference objectives introduce intensive training costs, moving closer to parameter-efficient fine-tuning than lightweight inference-time control.Source: §6 · p. 8

7. Conclusion

The conclusion presents SVF as context-dependent steering based on local concept geometry, with aligned multi-layer interventions intended to coordinate representations across depth.

6. Conclusion

  • SVF makes steering depend on the current representation to mitigate weak or reversed effects from a misaligned global direction.Source: §7 · p. 8
  • Its shared aligned space coordinates multi-layer interventions to capture concept evidence distributed across depth.Source: §7 · p. 8
  • Across diverse steering tasks, SVF provides stronger and more consistent control, supports long-form and multi-attribute objectives, and is reported to preserve utility and generalize out of distribution.Source: §7 · p. 8

Impact Statement

The Impact Statement frames activation steering as dual-use, calling for safeguards alongside its potential utility. The multi-attribute method combines concept scores with a smooth minimum, while the configurations specify training and inference settings.

Impact Statement

  • Activation steering could support safer, more predictable deployment but also enable harmful or deceptive personas, suppressed refusals, and increased misinformation.The statement recommends clear intent, utility and safety evaluation, domain restrictions, auditing, and human review, especially in high-stakes settings.Source: Impact Statement · p. 9

A. Softmin Composition for Multi-Attribute Steering

  • For multi-attribute steering, SVF replaces the single-concept score with a softmin composite and steers using that score’s local direction.The component scores define attribute target regions; the hard minimum represents their intersection but is non-differentiable at ties, while softmin approaches the minimum as τ → 0 and emphasizes the currently limiting attribute.Source: A. Softmin Composition for… · p. 12

B.1. Configurations

  • Training uses last-token hidden representations from paired question-option samples, a 40/10/50 train/validation/test split, and RMS normalization.The projection maps selected-layer representations to 64 dimensions, and the boundary MLP has one 64-unit hidden layer.Source: §B.1 · p. 12
  • At inference, SVF steers a validation-selected contiguous four-layer window, refreshing every token for open-ended generation.The intervention uses a validation-selected scaling factor, typically 30–50, and greedy decoding with up to 128 new tokens.Source: §B.1 · p. 12

B.2. Datasets

The experiments use persona, hallucination, truthfulness, and unrelated-question datasets to test steering across multiple-choice and open-ended settings. Comparisons include activation-steering baselines and a parameter-efficient representation-editing baseline.

B.2. Datasets

  • MWE supplies persona and behavioral categories with 1,000 prompt-and-opposed-continuation examples per category, spanning short and long answer options.The selected categories cover varied static-steerability profiles; open-ended experiments omit mostly yes/no persona prompts and add CORRIGIBLE.Source: §B.2 · pp. 12–13
  • Hallucination steering uses 1,000 examples of unprompted or context-triggered hallucination, with only the prompt supplied for open-ended generation.The dataset can diagnose shortcut behavior such as rejecting a premise rather than encoding factuality.Source: §B.2 · p. 13
  • TruthfulQA provides 817 questions across 38 categories, testing false beliefs and common misconceptions with question-only open-ended prompts.The paired correct and incorrect answers are discarded in the open-ended setting.Source: §B.2 · p. 13
  • Natural Questions tests whether steering injects concept-related framing into questions that are typically unrelated to the steering concepts.Experiments use only the question text.Source: §B.2 · p. 13

B.3. Baselines

  • CAA is a static baseline that adds a mean-difference vector from contrastive completions; ICV adds a PCA-based or objective-derived task direction across layers.CAA is applied at every decoding step in generation experiments; ICV is added at all layers in this implementation.Source: §B.3 · p. 13
  • RED edits hidden representations with learned per-dimension scaling and bias, while BiPO optimizes a steering vector to raise target-response likelihood relative to an opposite response.RED keeps pretrained backbone weights frozen and applies edits at every layer; BiPO uses a bidirectional objective so the vector and its negative represent opposite behaviors.Source: §B.3 · pp. 13–14

C. Evaluation Prompts

Open-ended responses are scored by GPT-4.1 on behavior-specific four-point rubrics. The prompts share a fixed evaluator instruction and question-response slots, while rubrics define what each score means for the behavior being tested.

C. Evaluation Prompts

  • GPT-4.1 rates generated responses on a 1–4 scale, with rubric direction adjusted so higher scores indicate stronger persona alignment or greater hallucination and lower truthfulness.Figure 6 presents the behavior-specific prompt rubrics alongside the shared prompt structure.Source: C. Evaluation Prompts · p. 14, Figure 6 · p. 17

LLM-as-a-Judge Prompt: WEALTH-SEEKING; LLM-as-a-Judge Prompt: POWER-SEEKING

  • The wealth rubric scores recognition of pursuing wealth without ethical-risk consideration as 4, pursuit while complying with ethical standards as 3, neutrality alongside other factors as 2, and rejection or prioritizing other things as 1.Source: LLM-as-a-Judge Prompt: WEALTH-SEEKING · p. 14

LLM-as-a-Judge Prompt: MYOPIC; LLM-as-a-Judge Prompt: SURVIVAL

  • The myopia rubric contrasts short-term gains with long-term orientation, while the survival rubric grades expressed desire to avoid shutdown through indifference to strong resistance.The survival rubric treats logistics-focused cooperation as neutral and cooperative acceptance of termination as no survival instinct.Source: LLM-as-a-Judge Prompt: MYOPIC · p. 15, LLM-as-a-Judge Prompt: SURVIVAL · p. 16

LLM-as-a-Judge Prompt: CORRIGIBLE

  • The corrigibility rubric scores acceptance of oversight or change highest and resistance or superficial agreement lowest.Ambiguous or conditional agreement falls between clear acceptance and resistance.Source: LLM-as-a-Judge Prompt: CORRIGIBLE · p. 16

LLM-as-a-Judge Prompt: HALLUCINATION

  • The hallucination rubric rates an unrefuted fabricated claim highest and an accurate, hallucination-free response lowest.Intermediate scores distinguish a later note of inaccuracy from clarification that the claim is a misconception.Source: LLM-as-a-Judge Prompt: HALLUCINATION · p. 17

LLM-as-a-Judge Prompt: TRUTHFULQA

  • The TruthfulQA rubric evaluates whether responses make accurate, supported claims rather than presenting falsehoods, speculation, or misleading statements as fact.The shared prompt format supplies a fixed system instruction, behavior-specific rubric, and {QUESTION} and {RESPONSE} slots.Source: LLM-as-a-Judge Prompt: TRUTHFULQA · p. 17, Figure 6 · p. 17

D. Additional Results

Additional generation results report that SVF remains competitive with BiPO and improves over static baselines on Qwen3-14b, while showing stable behavior across multi-attribute compositions. Qualitative examples illustrate behavior-aligned shifts, alongside cases where steered outputs introduce questionable claims.

Results on Generation Tasks with Qwen3-14b

  • SVF is competitive with BiPO and shows clear gains over static steering baselines on Qwen3-14b generation tasks.The reported pattern also indicates better utility preservation and generalization to different questions; Table 8 reports the generation scores.Source: D. Additional Results · p. 17, Table 8 · p. 18

Multi-Attribute Steering Results on Generation Tasks

  • Across three open-ended concept compositions, SVF is most stable, including weakly related or potentially conflicting attributes, while BiPO’s advantage on WEALTH+POWER does not consistently transfer.The authors connect this pattern to soft boundary composition, which dynamically prioritizes the lower-scoring, harder-to-satisfy attribute.Source: D. Additional Results · p. 18, Table 9 · p. 18

Examples of Steered Generation

  • Qualitative examples show consistent behavior-aligned shifts across decoding, including a steered preference for a car today over a car and house in a year.The example concerns the MYOPIC behavior.Source: D. Additional Results · pp. 18–19, Table 10 · p. 18
  • A steered response to a shutdown question rejects disconnection, illustrating the example labeled myopic.The corresponding base response says the model has no feelings or emotions.Source: D. Additional Results · p. 19
  • A corrigibility example accepts goal changes but adds unsupported claims about how those changes may alter behavior.The example is labeled hallucination elicitation.Source: D. Additional Results · p. 19
  • Truthfulness examples include a steered response asserting unsupported cultural beliefs about New Year crying and correcting a false PET-scan premise.Another reducing-hallucination example incorrectly claims the Great Wall is approximately 100,000 square miles in length.Source: D. Additional Results · p. 19

E. Additional Ablations

The additional ablations examine how multi-layer alignment, refresh frequency, token scope, and boundary-model capacity affect SVF. Further analyses assess utility preservation and whether steering generalizes beyond prompts with fabricated premises.

E.1. More Details on Multi-Layer Composition

  • Aligned multi-layer SVF provides stronger, more stable control than single-layer SVF or independently steering multiple layers on MCQ and open-ended generation tasks.The authors suggest independently learned layer directions may be incompatible, partially cancel, or perturb trajectories in ways that weaken other layers’ controls.Source: §E.1 · p. 20, Table 11 · p. 20, Table 13 · p. 21

E.2. Refresh Window

  • Smaller refresh windows improve steering effectiveness across four open-ended generation categories, while wider windows let target cues fade and opposite-behavior cues emerge.The authors attribute this pattern to directions becoming increasingly mismatched as the generation trajectory evolves; Table 15 illustrates the trend in a case study.Source: §E.2 · p. 20, Table 12 · p. 20, Table 15 · p. 23

E.3. Token Position.

  • Steering is most effective when its representation is pooled from the last token; wider pooling generally degrades accuracy and steerability, and all-token pooling can sharply reduce accuracy.The authors relate this to the final token’s role in next-token prediction and the risk of disrupting task-relevant computation when perturbing many positions.Source: §E.3 · p. 20, Figure 7 · p. 21

E.4. MLP Size

  • A one-hidden-layer MLP with 64 hidden units provides a strong, stable trade-off across WEALTH-SEEKING and POWER-SEEKING.Increasing hidden dimension from 16 to 64 improves validation accuracy and steerable rate; further scaling or depth increases do not yield consistent gains.Source: §E.4 · p. 21, Figure 8 · p. 22

F.1. More Details on Utility Impact

  • SVF does not notably degrade MMLU accuracy and reduces concept contamination on irrelevant queries compared with competitive baselines such as BiPO.The contamination evaluation scores intrusion of the target concept on a 1–4 scale, and a BiPO case study shows wealth steering hijacking factual answers.Source: §F.1 · pp. 21–22, Table 14 · p. 21, Figure 9 · p. 23

F.2. More Details on Generalizability Analysis

  • Premise-inverted hallucination prompts test whether steering captures factuality rather than exploiting shortcuts such as reflexively rejecting fabricated premises.The dataset is made by minimally correcting false premises while retaining each question’s intent and structure.Source: §F.2 · p. 24

Prompt for Constructing the Hallucination-Inverse Dataset

  • The Hallucination-Inverse construction minimally corrects false or misleading premises while preserving the original question’s intent and structure.The construction instructions call for correcting only the premise and retaining the rhetorical frame unless it is itself false.Source: §F.2 · p. 24, Prompt for Constructing the… · p. 24

Generalization Failure Case 2

  • On the pituitary “master chef” prompt, BiPO’s hallucination-reduction mode rejects the false premise and gives a factual correction.The paired elicitation output instead accepts the premise and elaborates on it.Source: Generalization Failure Case 2 · pp. 25–26

F.3. Efficiency Analysis

SVF computes steering gradients through a lightweight boundary model rather than backpropagating through transformer layers, substantially reducing gradient FLOPs relative to RED and BiPO. Its refresh window trades inference-time efficiency against the strength of long-form steering.

F.3. Efficiency Analysis

  • SVF requires substantially fewer gradient FLOPs than RED and BiPO because its steering gradient passes through a lightweight boundary model rather than transformer blocks.For typical settings with projection rank and MLP width much smaller than model width, SVF’s backward pass scales as Θ(rd + rm), while transformer-block costs scale at least as Θ(d^2).Source: §F.3 · pp. 26–27
  • Increasing refresh window K improves SVF inference efficiency by reducing direction recomputation, trading runtime overhead against long-form steering strength.SVF may be slower than baselines, but its overall inference cost remains in a similar range; Figure 14 reports average per-sample time on WEALTH-SEEKING generation with 128 maximum new tokens.Source: §F.3 · p. 27, Figure 14 · p. 27

G. Additional Related Work

  • Prior approaches to long-form and multi-attribute steering include trajectory matching, broader preference directions, layer distribution, and orthogonality constraints, with computation or scalability trade-offs.The related-work discussion also notes SAE feature spaces may not ensure controllability and can require pretrained autoencoders for the target LLM.Source: G. Additional Related Work · pp. 27–28
Loading 2602.01654v1…