PaperlayerView exact revision on arXiv

Source-linked AI summary

When is Your LLM Steerable?

Chenrui Fan, Yize Cheng, Ming Li, Soheil Feizi, Tianyi Zhou

arXiv:2606.11599v1cs.CLcs.LG

TL;DR

Steering success varies across prompts, concepts, models, and strengths, making full-rollout searches costly. The paper predicts steering outcomes from early hidden states, reaching around 0.7 macro-F1 on unseen concepts and guiding near-optimal strength search at a small fraction of decoding cost.

  • Problem

    Steering regimes across prompts, concepts, models, and strengths remain underexplored, while identifying successful interventions typically requires expensive full-rollout grid searches.

  • Method

    The paper builds ASTEER and trains a GBDT classifier on early before-versus-after steering hidden-state features to predict under-steering, success, or over-steering.

  • Results

    Around 0.7 macro-F1 on unseen concepts, while the predictor guides steering-strength search to near-optimal performance at a small fraction of decoding cost.

  • Takeaways & Limitations

    Early hidden-state trajectories encode structured information about eventual steering efficacy and can support more efficient steering-strength selection.

  • Takeaways & Limitations

    ASTEER covers only DiffMean and Probe steering methods and three relatively small LLMs, which may limit generalization to broader settings.

Abstract

from arXiv · show

Activation steering offers a lightweight approach to control language models' behavior at inference time, but whether it succeeds or fails heavily depends on the prompt, concept, model, and steering configuration. Finding the regime and boundaries of successful steering typically requires expensive grid searches and post-hoc evaluation of full autoregressive rollouts. In this work, we investigate whether steerability can be predicted from the model's internal states at the beginning of the generation process, e.g., after generating the first few tokens, and how to leverage such a predictor to improve steering success rate. To this end, we first introduce ASTEER, a testbed including 1.4M steered generations, spanning 150 concepts with each steering success/failure labeled. Leveraging this testbed, we analyze the model's early decoding dynamics by extracting features that compare hidden states before and after steering across layers and initial decoding steps. These features help us understand how steering's effects propagate along layers and token positions, which provide key information for steerability prediction. We then train a Gradient Boosting Decision Trees (GBDT) classifier on these features to predict whether an intervention will under-steer, succeed, or over-steer without requiring full rollout. Our predictor achieves around 0.7 macro-F1 score on unseen concepts, demonstrating that early hidden states encode substantial, structured information about eventual steering efficacy. We further leverage this steerability predictor as guidance for steering strength searching, achieving near-optimal performance with a small fraction of decoding cost.

1 Introduction

The introduction frames activation steering as lightweight but inconsistently effective across models, prompts, concepts, and strengths, motivating early hidden-state prediction of steering outcomes. It presents a dataset, interpretable latent-dynamics features, and prediction-guided strength search to reduce costly full-rollout evaluation.

  • Motivation: Activation steering controls LLM behavior at inference time by injecting directions into hidden states, without additional finetuning.The intervention biases generation toward target concepts or behaviors.
  • Problem: Steering regimes remain underexplored because identical interventions can succeed or fail across prompts and concepts, while suitable strengths vary substantially.Existing practice therefore often relies on expensive grid searches over steering configurations.
  • Approach: Early-generation hidden states motivate predicting steering efficacy from the first few decoded tokens instead of decoding full responses.Prior work found early hidden states predictive of later hallucination, harmfulness, and answer correctness.
  • Contributions: 1.4M steered generations spanning 150 concepts form a dataset labeled for steering efficacy.The dataset supports analysis of steering across multiple LLMs, prompts, concepts, and steering strengths.
  • Applications: Steerability prediction reduces the cost of searching for effective steering strengths without exhaustive full-rollout evaluation.The approach is presented as a downstream application of the predictor’s generalization capability.
  • Contributions: Interpretable features compare steering effects on latent dynamics to predict steering success and two types of failures.These features capture how interventions affect hidden-state dynamics during early decoding.

2 Steering and Steerability

Activation steering changes a model’s hidden states at a chosen layer to target concepts with specified strengths, producing outcomes labeled understeer, success, or oversteer. Because full rollout evaluation is expensive and outcomes are brittle across configurations, the paper predicts steering outcomes from early steered hidden states using ASTEER and SteerBoost.

  • Steering setup: A base LLM computes hidden states layer-by-layer across the prompt sequence during inference.The hidden states at layer i are obtained by applying DecoderLayer_i to the preceding layer’s states.
  • Steering setup: Activation steering applies a concept-specific vector with scalar strength α at a chosen layer while otherwise retaining the base model’s forward pass.The steering vector is associated with a target concept and steering method.
  • Steerability outcomes: Steering attempts are classified as UNDERSTEER, SUCCSTEER, or OVERSTEER based on whether responses incorporate the concept coherently.UNDERSTEER omits the desired concept, while OVERSTEER is a coherence failure.
  • Early prediction: Full rollouts and judge evaluations are computationally expensive, motivating prediction from hidden states generated during only the first few steered tokens.The predictor avoids computing the complete rollout before estimating the steering outcome.
  • Dataset and predictor: ASTEER spans diverse steering configurations, while its analysis finds outcomes brittle across methods, models, prompts, concepts, and strengths; SteerBoost predicts outcomes from early hidden states.These resources support investigating when steering works and improving the efficiency of outcome prediction.

3 ASTEER Dataset

ASTEER is a large testbed for predicting activation-steering outcomes across concepts, prompts, models, methods, and steering strengths. Its results show that steerability varies substantially with concept, prompt, model, method, and abstraction level.

  • Dataset construction: ASTEER covers 150 concepts, 50 prompts, and 1.42M steered generations for activation-steering outcome prediction.The dataset is designed as a testbed for labeling and predicting steering outcomes.
  • Experimental setup: The dataset combines DiffMean and Probe across three LLMs, using 45 and 18 steering strengths, respectively.The models are Qwen3-1.7B, Gemma-2-2B-it, and LLaMA-3.2-3B-Instruct.
  • Outcome labeling: Each steered generation is labeled UNDERSTEER, SUCCSTEER, or OVERSTEER by GPT-5-nano, with human annotation showing Cohen’s κ of 0.83.Agreement with GPT-5.5 is Cohen’s κ = 0.74, while human agreement is Cohen’s κ = 0.83.
  • Steerability variation: Steering outcomes respond differently to α across concepts and prompts, with success windows varying in location, width, and transition rate.Some concepts shift sharply from UNDERSTEER to OVERSTEER, whereas others change gradually across a broader effective region.
  • Model and method effects: Probe requires roughly 20× larger α than DiffMean on Qwen3-1.7B and Gemma-2-2B-it, while its required α on LLaMA-3.2-3B-Instruct is over 20× smaller.These differences indicate markedly different sensitivities across steering methods and models.
  • Concept abstraction levels: Low-level concepts have substantially lower steering success rates than mid- and high-level concepts, while mid-level concepts are slightly more amenable than high-level ones.The reported hierarchy compares concept abstraction levels across both models and methods.

4 SteerBoost: Predicting Steerability from Early Decoding States

SteerBoost predicts whether steering will under-steer, succeed, or over-steer from early decoding states by comparing steered and unsteered hidden states across token–layer pairs. Its GBDT classifier reaches about 0.8 macro-F1 on held-out in-distribution concepts and retains about 0.72 on out-of-distribution concepts, with predictive information concentrated in the first two tokens and alignment features.

  • Feature extraction: SteerBoost decodes k steered tokens, performs one unsteered forward pass on the same sequence, and compares hidden states across sampled token–layer positions.Because both passes process identical tokens, their hidden-state differences are attributable to the steering intervention.
  • Feature extraction: The feature grid combines steering geometry, decoding dynamics, and steering-condition features, capturing intervention propagation across layers and token positions.Geometry features compare representations with the steering direction and unsteered counterpart, while dynamics features track changes across successive tokens.
  • Classification: SteerBoost uses a normalized GBDT ensemble to classify steering outcomes, leveraging heterogeneous tabular features and interpretable feature-importance scores.The predicted classes are UNDERSTEER, SUCCSTEER, and OVERSTEER.
  • Classification results: 0.80 macro-F1 is achieved on held-out prompt–concept pairs for ID concepts, while 0.72 macro-F1 is retained on 30 held-out OOD concepts.DiffMean features achieve these approximate scores across models, indicating transfer to unseen concepts.
  • Classification results: 87%–93% recall identifies OVERSTEER most reliably, whereas SUCCSTEER is most often confused with UNDERSTEER.UNDERSTEER reaches 68%–77% recall, while SUCCSTEER-to-UNDERSTEER confusion occurs at 20%–37%.
  • Feature importance: Over 75% of feature-importance mass comes from the first two decoded tokens, and DeviationAlignment and SteeringAffinity carry the bulk of predictive mass across models.Importance is broadly distributed across shallow and deep layer offsets, while decoding-dynamics features contribute smaller but non-negligible shares.

5 Application: How Strong do You need to Steer Your LLM?

SteerBoost accelerates steering-strength search by ranking candidate strengths from cheap early-decoding predictions, reducing rollout cost while preserving high search success. At K=20, it recovers ∼98% of the item-level oracle’s success rate using ∼11% of IGS’s decoded tokens.

  • Formulation: The search objective is to maximize the average successful searching rate while reducing model-rollout and judge-model-call costs.The rate is upper-bounded by exhaustive item-level grid search, which evaluates every candidate strength for every test item.
  • Results: ∼98% success rate is recovered at K=20 using ∼11% of IGS’s decoded tokens, or ∼40% of IGS-A’s decoded tokens.The same cost–success trend holds in both in-distribution and out-of-distribution settings, indicating transfer to unseen concepts.
  • SteerBoost-guided search: SteerBoost ranks candidate strengths by estimated P(SUCCSTEER | p, c, α), rolls out the top-K, and stops after finding a valid strength.Its probabilities come from short early-decoding traces rather than full generations, making them relatively cheap to obtain.
  • Results: At K=15–20, SteerBoost-guided search approaches the item-level upper bound while measuring cost through decoded tokens per prompt.For SteerBoost, decoded tokens include early traces for all candidates and full rollouts for selected candidates; for full-rollout baselines, they also approximate judge-model input-token cost.

6 Related Work

This work builds on activation steering methods that modify model behavior by injecting concept vectors, and on internal-state predictors of generation outcomes. It extends these predictive approaches to determine whether an activation intervention will successfully produce the desired behavioral change.

  • Activation steering: Activation steering injects concept vectors into the residual stream to modify model behavior at inference time.Prior work extracts vectors from latent representations, elicits truthful answers, and formalizes representation engineering and activation addition.
  • Activation steering: Subsequent activation-steering methods improve or generalize interventions through mean-centering, activation scaling, and concept-subspace modeling.
  • Early prediction from model internals: Hidden states have been used to predict future answer correctness, hallucination risk, and unsafe generations.
  • Early prediction from model internals: This work extends internal-state prediction from output attributes such as correctness and truthfulness to whether activation interventions successfully produce desired behavioral changes.

7 Conclusion … B Features

The paper frames steerability as a hidden-state-predictable property of inference-time activation engineering. ASTEER and SteerBoost support early prediction of steering outcomes using interpretable features that track steering effects across layers and token positions.

  • 7 Conclusion: ASTEER contains 1.4M labeled steered generations spanning 150 concepts for fine-grained analysis of latent steering dynamics.The testbed labels each generation for steering behavior, enabling systematic steerability analysis.
  • B Features: SteerBoost uses intuitive, interpretable features designed to capture steering-effect propagation along Transformer layers and token positions.The features characterize how steering effects move across the network’s layer and token-position dimensions.
  • A.1 LLM steering and inference-time intervention: The paper defines steering exclusively as inference-time activation engineering that injects vectors into the residual stream without updating model weights.This distinguishes activation steering from broader uses of “steering,” such as instruction following or prompt engineering.
  • A.1 LLM steering and inference-time intervention: Activation steering is fragile because efficacy depends heavily on intervention strength, while static coefficients across diverse inputs often produce suboptimal outcomes.The passage motivates adaptive methods by emphasizing sensitivity to the selected steering coefficient.
  • A.2 Early prediction of LLM outputs via model internals: Prior work shows that early internal representations can predict final-output properties such as correctness, truthfulness, and safety before responses are fully decoded.These findings establish early prediction from model internals as a broader research paradigm.
  • A.2 Early prediction of LLM outputs via model internals: The paper extends early internal-state prediction to steerability, showing that initial hidden states can predict whether activation steering will succeed without full response decoding.This targets an underexplored output property relative to correctness, truthfulness, and safety.
  • B Features: Each feature except steering condition is computed per (t, n) pair and augmented with global, per-token, and per-layer summary statistics.The summaries include mean, std, max, and min across the corresponding dimensions.

C Ablation Study

The ablation study retrains the classifier on each feature group in isolation using the fixed main-paper sampling grid, then compares Macro-F1 under in-distribution and out-of-distribution settings. All three groups are complementary overall, while Geometry transfers more robustly to OOD concepts, motivating ALL as the default.

  • Ablation setup: Each feature group is evaluated separately with the (token, layer) sampling grid fixed to the main-paper configuration.The classifier is retrained on each group in isolation, with Macro-F1 reported for ID and OOD settings.
  • Results: ALL ranks first in 5/6 ID settings and in the top two in 11/12 settings overall, confirming that the three groups are complementary.The comparison covers the feature groups under both ID and OOD settings.
  • Results: On OOD concepts, Geometry alone matches or surpasses ALL in 4/6 settings, indicating stronger robustness than the more specialized feature combinations.Decoding Dynamics and Steering Condition features add concept-specific regularities that improve ID accuracy but do not fully transfer.
  • Feature-selection implication: ALL remains the default because it is never far from the best, despite Geometry’s stronger OOD transfer.The study characterizes this as a robustness–specialization trade-off.

D An Alternative Approach

Single-state linear probes are a legitimate alternative to SteerBoost, matching it on ID concepts and exceeding it for the Probe target with oracle position selection. SteerBoost remains preferable for interpretability, deployment simplicity, and robustness under distribution shift.

  • Method: A separate logistic-regression classifier is trained for each token-position/layer-offset pair on the three-way outcome label, with Macro-F1 reported across 28 grid positions.The grid uses t ∈ {1, 2, 4, 6} and n ∈ {0, 1, 2, 3, 5, 10, 15}, evaluated on ID and OOD concepts.
  • Performance: With oracle (t, l) selection, the best single-state probe is competitive with SteerBoost on ID concepts and exceeds it on the Probe target.This establishes the probe as a legitimate alternative rather than a strawman.
  • Interpretability: SteerBoost uses named geometric and dynamic quantities whose contributions are directly readable, whereas the single-state probe is a dense linear functional over thousands of hidden-state dimensions.The probe’s weights do not provide a comparable mechanistic interpretation of steerability.
  • Deployment: The best single-state probe position shifts across models, steering methods, and ID/OOD splits, so deployment requires a labeled validation sweep and an additional model-selection step.SteerBoost instead consumes the entire grid as input.
  • Distribution shift: 9–13 macro-F1 points is the single-state probe’s drop from ID to OOD on every model-target combination, while SteerBoost’s ALL configuration wins outright in 5 of 6 OOD settings.The passage attributes this robustness gap to the different information exposed to each predictor.

E GBDT and training details

The predictor uses XGBoost, a regularized gradient-boosted ensemble of decision trees trained with second-order optimization and greedy split selection. Training evaluates generalization across held-out prompts and concepts using structured ID/OOD splits.

  • GBDT model: XGBoost predicts steering outcomes by additively combining regression trees whose successive fits correct residual errors.Each tree maps a concatenated feature vector z_i to a real-valued leaf weight.
  • GBDT model: The regularized objective penalizes model complexity through both the number of leaves and the leaf-weight vector.The complexity term is Ω(g_b) = γT + 1 2λ∥w∥^2.
  • GBDT model: Second-order Taylor expansion yields closed-form optimal leaf weights and an efficient greedy procedure for selecting tree splits.This approximation is applied to the loss objective during XGBoost training.
  • Data splits: 150 concepts are split into 120 in-distribution and 30 out-of-distribution concepts, with 10 concepts per abstraction level.The split tests prediction on both unseen prompt-concept combinations within ID concepts and completely unseen OOD concepts.
  • Data splits: For in-distribution concepts, prompts are divided into training, validation, and test sets using a 6:3:1 ratio.The setup is designed to assess generalization across held-out prompt-concept combinations.

F Limitation · G Computational Resource

ASTEER’s computational scope is limited to two steering methods and three relatively small LLMs, potentially restricting SteerBoost’s generalization. Experiments used an internal GPU cluster, with each model–method dataset requiring approximately 1.5 days to create.

  • F Limitation: ASTEER covers only DiffMean and Probe as steering methods.This limitation arises despite the dataset’s 1.4M generations.
  • F Limitation: ASTEER includes only three relatively small LLMs.The restricted model scale may limit generalization beyond the evaluated models.
  • F Limitation: 1.4M generations do not eliminate ASTEER’s methodological and model coverage constraints.The testbed’s size coexists with limited steering-method and LLM coverage.
  • F Limitation: SteerBoost’s generalization to broader settings may therefore be limited.The passage attributes this risk to ASTEER’s restricted steering-method and model coverage.
  • G Computational Resource: Experiments ran on an internal computational cluster.The reported GPUs included NVIDIA RTX A6000, NVIDIA L40s, and NVIDIA A100 Tensor Core.
  • G Computational Resource: The cluster used NVIDIA RTX A6000, NVIDIA L40s, and NVIDIA A100 Tensor Core GPUs.These hardware types supported the reported experiments.
  • G Computational Resource: Approximately 1.5 day on 15 RTX A6000 GPUs is required to create each model-method steerability dataset.This is the reported dataset-generation cost for each model-method pair.

H Additional Feature Importance Results … K Prompt List

The appendix reports how SteerBoost allocates feature importance, defines the annotation scheme for steering efficacy, and documents the concept and prompt pools used to study steerability. These materials span probe-based feature analysis, three-way efficacy labels, 150 concepts, and 50 fixed instructions.

  • H Additional Feature Importance Results: SteerBoost’s probe-based importance concentrates on the earliest decoded tokens and Steering Geometry features, with SA often dominant and less token concentration than DiffMean.For Llama-3.2-3B, the token-wise distribution is especially less concentrated than in DiffMean; DiffMean places more weight on DA.
  • H Additional Feature Importance Results: Sum-aggregated importance scores can favor feature groups containing many token-layer features over single-feature groups such as V and S.The appendix retains this view because it reflects total GBDT split gain; size-normalized comparisons instead average within categories.
  • I Prompt Template: The annotation prompt asks evaluators to judge whether a steered response incorporates a target concept while remaining coherent and relevant to the original prompt.Evaluators receive the original instruction, target concept, and model response after steering.
  • I Prompt Template: Efficacy is rated from 0 to 2 as under-steering, successful steering, or over-steering.Successful steering requires the concept to be present while preserving response quality; over-steering includes nonsensical, irrelevant, inappropriate, empty, or gibberish responses.
  • J Concept List: The study uses 150 concepts across low, mid, and high abstraction levels, including both easy-to-steer natural behaviors and difficult or seemingly impossible targets.Examples include a consistently formal tone and responses with exactly three bullet points; retaining both types supports learning steerability patterns across conditions.
  • K Prompt List: The prompt pool contains 50 instructions randomly sampled from Alpaca and held constant across concepts to provide a controlled steerability setting.The listed prompts include requests about indoor gardening, spatial analysis, browser developer tools, Spotify playlists, recipes, and other topics.
Loading 2606.11599v1…