Source-linked AI summary
Steer2Adapt: Dynamically Composing Steering Vectors Elicits Efficient Adaptation of LLMs
Pengrui Han, Xueqiang Xu, Keyang Xuan, Peiyang Song, Siru Ouyang, Runchu Tian, Yuqing Jiang, Cheng Qian, Pengcheng Jiang, Jiashuo Sun, Junxia Cui, Ming Zhong, Ge Liu, Jiawei Han, Jiaxuan You
TL;DR
Existing steering methods often rely on single static directions, limiting adaptation to varying tasks and coordinated capabilities. STEER2ADAPT composes reusable semantic concept vectors by searching a low-dimensional subspace with few examples, achieving an average 8.2% improvement across nine tasks and three models. The method provides inference-time adaptation that the paper characterizes as data-efficient, stable, and transparent.
Problem
Existing steering methods rely on single static directions that can be ineffective across tasks and inadequate for tasks requiring coordinated control over multiple capabilities.
Method
STEER2ADAPT constructs reusable domain-specific semantic subspaces and uses Bayesian optimization to discover task-specific linear combinations from a handful of examples.
Results
8.2% average improvement across nine reasoning and safety tasks and three models demonstrates consistent inference-stage adaptation.
Takeaways & Limitations
Composing reusable semantic concept vectors provides a data-efficient, stable, and transparent approach to inference-time adaptation across diverse tasks within a domain.
Takeaways & Limitations
The formulation assumes that task-domain performance is governed by k underlying behavioral concept dimensions represented by reusable steering vectors.
Abstract
from arXiv · showhide
Activation steering has emerged as a promising approach for efficiently adapting large language models (LLMs) to downstream behaviors. However, most existing steering methods rely on a single static direction per task or concept, making them inflexible under task variation and inadequate for complex tasks that require multiple coordinated capabilities. To address this limitation, we propose STEER2ADAPT, a lightweight framework that adapts LLMs by composing steering vectors rather than learning new ones from scratch. In many domains (e.g., reasoning or safety), tasks share a small set of underlying concept dimensions. STEER2ADAPT captures these dimensions as a reusable, low-dimensional semantic prior subspace, and adapts to new tasks by dynamically discovering a linear combination of basis vectors from only a handful of examples. Experiments across 9 tasks and 3 models in both reasoning and safety domains demonstrate the effectiveness of STEER2ADAPT, achieving an average improvement of 8.2%. Extensive analyses further show that STEER2ADAPT is a data-efficient, stable, and transparent inference-time adaptation method for LLMs.
1. Introduction
Existing steering approaches are costly, inflexible, or insufficiently expressive for task variation and coordinated capabilities. STEER2ADAPT instead composes reusable semantic vectors, searching a low-dimensional recipe with few examples for data-efficient inference-time adaptation.
- Motivation: Existing task-vector steering is computationally intensive and generalizes poorly across tasks, while semantic-driven steering can lack expressiveness for complex tasks.Task-vector methods learn directions from downstream data; semantic-driven methods use predefined concept vectors.
- Motivation: Single static steering directions can be ineffective or harmful across tasks and cannot flexibly coordinate multiple capabilities.These limitations motivate composing existing steering vectors rather than refining individual vectors.
- Approach: STEER2ADAPT represents domain dimensions in a reusable, low-dimensional semantic concept subspace and searches a task-specific linear combination of basis vectors.The framework shifts activation steering from finding one direction to discovering a compact steering recipe.
- Approach: Bayesian optimization with a stability-aware objective searches subspace coefficients from only a handful of examples before injecting the composed vector at inference time.The objective rewards correcting wrong predictions while penalizing flips from correct to incorrect.
- Results: Across nine reasoning and safety tasks and three models, STEER2ADAPT achieves an average 8.2% improvement.The reported framework is intended to provide data-efficient, stable, and transparent adaptation.
2. Related Works
Prior LLM adaptation work spans pre-training, fine-tuning, inference-stage methods, and model composition. STEER2ADAPT differs by composing domain-relevant activation vectors rather than merging discrete model components or updating model weights.
- LLM Adaptation: LLM adaptation generally includes pre-training, fine-tuning, and inference-stage adaptation.Inference-stage adaptation adjusts models for novel tasks without prohibitive retraining costs.
- LLM Adaptation: Context-based augmentation uses in-context learning and few-shot capabilities, potentially incorporating external knowledge or past experience.This is one major direction in inference-stage adaptation.
- Composition in LLM Adaptation: Model composition can statically fuse parameters or dynamically route inputs among experts, scaling capacity but potentially increasing memory requirements.These approaches compose model components in parameter space or through expert routing.
- Composition in LLM Adaptation: STEER2ADAPT composes domain-relevant activation vectors to synthesize a vector for novel tasks within the same domain, rather than merging discrete model components.Its composition occurs in activation space instead of parameter space.
3. Methodology
STEER2ADAPT constrains adaptation to a reusable semantic subspace and searches its coefficients with Bayesian optimization. A stability-aware objective favors correcting errors while preserving correct behavior, producing an inference-time composed vector without gradient updates.
- Task Formulation: The method hypothesizes that domain performance is governed by k behavioral concept dimensions represented by steering vectors.For a task, coefficients are searched so their combined vector improves performance using only a few examples.
- Prior Semantic Subspace Construction: A domain-specific concept dictionary is extracted with representation engineering and used to form a reusable semantic prior subspace.The vectors correspond to behavioral concepts selected for the domain.
- Composed Vector Search: STEER2ADAPT searches coefficient space with Bayesian optimization and composes the learned coefficients into the final steering vector for inference.The search occurs within a low-dimensional space rather than over the full activation dimension.
- Composed Vector Search: The stability-aware objective maximizes gains on initially incorrect examples while penalizing prediction flips and confidence degradation on initially correct examples.Its penalty hierarchy makes the optimization risk-averse.
- Inference-Time Steering: The optimized vector is injected only at inference time, requires no gradient updates, and can be reused across inputs from the same target task.This makes the procedure a plug-in intervention during inference.
4. Experiment Setup
The evaluation covers reasoning and safety tasks using three open-source model families and compares prompting and representation-based baselines. Results are summarized at task, model, and reliability levels across both domains.
- Domains: The study evaluates reasoning and safety, two domains requiring complex, multifaceted capabilities.These domains are treated as distinct settings for evaluating adaptability.
- Tasks and Datasets: The reasoning evaluation covers five tasks, while the safety evaluation covers four benchmarks.Reasoning includes Code, Social, Arithmetic, Logic, and Game; safety includes SaladBench, FaithfulQA, TruthfulQA, and BBQ.
- Models and Calibration: The evaluation uses three open-source backbones and a balanced calibration set of n = 12 examples for baseline comparison.The models are Llama-3.1-8B-Instruct, Qwen-2.5-7B-Instruct, and Mistral-7B-Instruct-v0.1.
- Evaluation Views: Figure 3 aggregates average percentage improvement over baseline by task and model, alongside gain distributions across reasoning and safety evaluation scenarios.The reliability views cover 5 tasks × 3 models for reasoning and 4 tasks × 3 models for safety.
- Baselines: The baselines include few-shot prompting, in-context learning, Contrastive Activation Addition, and Single-Direction Steering.Representation-based baselines use static task vectors or fixed coefficient sweeps over basis vectors.
5. Experiment Results
Across reasoning and safety tasks, STEER2ADAPT consistently improves performance across tasks and models, with stable gains and a favorable performance–efficiency trade-off.
- STEER2ADAPT yields positive improvements across all evaluated tasks, leading reasoning gains and ranking best on three of four safety tasks.Baselines frequently show task-dependent regressions or ineffectiveness.
- STEER2ADAPT achieves the strongest or near-strongest improvements across all evaluated backbone models in both domains.It improves reasoning performance across Llama-3.1, Qwen-2.5, and Mistral-7B while avoiding the cross-model regressions seen in several baselines.
- Figure 4 tests whether basis directions remain effective under mismatched, noisy, or task-vector-based subspaces.These conditions distinguish the effects of relevance, moderate noise, and alternative subspace construction.
- STEER2ADAPT produces compact, positively centered gain distributions without negative outliers across reasoning and safety scenarios.Baseline methods show higher variance and severe regressions, including drops exceeding 30% in some safety scenarios.
- STEER2ADAPT achieves the highest efficiency score, outperforming prompting-based methods on performance gain per unit of inference cost.The score divides normalized performance improvement by inference cost.
6. Analysis
Analyses show that STEER2ADAPT benefits from relevant basis directions, tolerates moderate subspace imperfections, and exposes interpretable but entangled trade-offs while preserving linguistic competence.
- Basis Directions Matter: Using a safety-related subspace for reasoning causes substantial performance degradation and significantly higher variance across reasoning tasks.The result supports the importance of semantic relevance between the subspace and target domain.
- STEER2ADAPT is tolerant to Imperfect Basis: Adding weakly related or unrelated directions to the reasoning subspace causes only minor changes in average performance and variance.The tested additions include safety-derived vectors and a generic optimistic direction.
- Task Vectors can be Used as an Alternative Subspace: Task vectors from relevant safety tasks form a competitive steering subspace, though semantic vectors retain a modest performance advantage.The comparison suggests related task vectors can substitute for semantic vectors with some loss in performance.
- Transparency: Coding gains align with higher Conscientiousness and lower Openness, while safety objectives show entangled, non-uniform trade-offs.The coding alignment corresponds to more structured and less exploratory behavior, whereas basis directions are not fully disentangled.
- Transparency: Basis combinations vary across tasks and models, motivating adaptive search rather than fixed or intuitive combinations.This variability limits the reliability of manually selecting one combination for every setting.
- Preserving Linguistic Competence: +7.5% average task improvement accompanies a −2.37% average BLiMP change, yielding a 3.9× performance–preservation trade-off.The averages cover nine vectors across reasoning and safety domains and five BLiMP syntactic benchmarks.
7. Conclusion
STEER2ADAPT reframes inference-time LLM adaptation as dynamically composing reusable semantic vectors into task-specific steering recipes. Across reasoning and safety experiments, it consistently improves downstream performance and remains robust to subspace noise and entanglement.
- STEER2ADAPT discovers task-specific steering recipes by composing a small set of reusable semantic concept vectors instead of learning directions from scratch.The framework targets efficient and transparent inference-time adaptation.
- Across reasoning and safety domains, STEER2ADAPT consistently improves downstream LLM performance while remaining robust to noise and entanglement within the vector subspace.The conclusion presents vector composition as a scalable direction for diverse and evolving tasks.
A.1. Limitations and Future Work
The paper identifies limitations around basis-direction quality, disentanglement, search dimensionality, and evaluation scope, while proposing structured and broader future extensions.
- Basis directions: The method assumes reasonably relevant basis directions; completely irrelevant or adversarial bases may degrade performance.The authors identify systematic construction of high-quality candidate directions as an open problem.
- Basis directions: Interactions among non-disentangled basis directions can introduce trade-offs, particularly in safety-related settings.The paper motivates richer models of interactions beyond simple linear interpretations.
- Search space: Adaptive search currently operates within a fixed, low-dimensional subspace, while larger or dynamically constructed spaces may increase search complexity.More efficient search strategies for higher-dimensional steering spaces remain future work.
- Evaluation scope: The evaluation focuses on reasoning and safety benchmarks, leaving performance in other adaptation-demanding domains open.Examples include long-horizon planning, culturally rich language understanding, socially grounded interaction, and embodied agents.
- Evaluation scope: Behavioral and psychological evaluations are identified as another setting for extending the framework beyond standard benchmarks.The paper presents this as a promising direction for future research.
- Future directions: Future work includes automatically discovering task-relevant basis directions and adding sparsity or hierarchical structure to manage representational interactions.These directions aim to reduce reliance on manual construction and better organize the steering space.
A.2. Preliminary
The preliminary formulation treats inference-time adaptation as controlling a fixed pretrained model without parameter updates. It focuses on internal activation interventions that perturb hidden states and propagate the modified activations through later layers.
- Problem formulation: Inference-stage adaptation seeks an inference-time control signal that maximizes expected task utility without updating model parameters.The intervention space determines the class of test-time adaptation strategy.
- Control signals: A control signal specifies an intervention applied to a fixed pretrained model to modulate behavior during inference.The paper focuses on signals operating at internal representational levels.
- Activation-level interventions: An activation-level intervention perturbs a hidden activation at a chosen layer before the modified state is processed onward.The perturbation is applied to the layer hidden state associated with the input.
- Activation-level interventions: The modified activation is propagated through subsequent layers to produce the model output.
A.3. Optimization Objective Details
STEER2ADAPT uses a risk-averse objective to improve initially incorrect examples while preserving correct predictions and confidence. Bayesian optimization searches a bounded coefficient space with a Gaussian-process surrogate and expected improvement, while accompanying analyses examine subspaces, task-vector alternatives, and model-dependent basis combinations.
- Stability-Aware Objective: The objective prioritizes preserving correct examples over acquiring improvements on incorrect ones.Its design is explicitly risk-averse.
- Adaptation Gain: Initially incorrect examples receive continuous rewards for increases in the correct answer’s log-probability.The reported gain for fixing one error is typically +1.0 to +3.0 in log-probability mass.
- Hierarchical Safety Regularization: Initially correct examples incur hierarchical penalties for prediction flips and confidence-margin degradation.The framework uses a prohibitive flip penalty and a substantial degradation penalty.
- Risk-Averse Condition: The hierarchy λflip > λdrop > max(Ggain) favors lossless directions that improve errors without eroding robustness.A confidence loss of 10.0 can outweigh an approximately 2.0 gain from correcting an error.
- Bayesian Optimization: Bayesian optimization models the objective with a Gaussian process and selects candidates by maximizing expected improvement.The surrogate uses a constant mean and Matern-5/2 kernel, while expected improvement balances exploration and exploitation.
- Search Setup: The coefficient search explores the bounded hypercube [−2, 2]^k through 50 Sobol initialization points and 350 optimization iterations.Objective values are standardized for numerical stability, yielding 400 evaluations per seed.