PaperlayerView exact revision on arXiv

Source-linked AI summary

IDEEA: training-free Input-Dependent stEEring via Activation cluster matching

Zheng Wang, Muchen Li, Renjie Liao, Yan Leng

arXiv:2609.02089v1cs.CLcs.LG

TL;DR

Pre-training gives LLMs broad knowledge without guidance about how to behave. IDEEA provides training-free input-dependent steering by selecting directions conditioned on input activations, achieving the highest TruthfulQA truth × info rate gain over the unsteered model and generalizing across other concepts.

  • Problem

    Pre-training gives LLMs broad knowledge but no guidance on whether to be truthful, helpful, or safe.

  • Method

    IDEEA clusters positive and negative activation supports, constructs cluster-conditional steering directions through optimal matching, and selects a direction based on the input activation at inference time.

  • Results

    IDEEA achieves the highest TruthfulQA truth × info rate gain over the unsteered model, roughly doubling the strongest baseline, and generalizes to other steerable concepts.

  • Takeaways & Limitations

    IDEEA preserves original input representations, avoids refusal-collapse failures, and exploits multimodal activation structure that static directions average away.

  • Takeaways & Limitations

    Interventions across multiple layers can cause inter-layer activation drift, potentially making per-head cluster fits suboptimal.

Abstract

from arXiv · show

Steering aligns large language models (LLMs) by injecting a bias into selected activations at inference time, offering a far cheaper alternative to weight-update methods such as supervised fine-tuning or reinforcement learning. However, most existing training-free steering methods are input-independent: a single direction is fitted once and shared across all inputs. This is fundamentally limiting as different inputs occupy different regions of the activation space and admit different optimal steering directions toward the same target concept, much as the gradient with respect to a fixed loss varies from input to input. We close this gap with IDEEA (Input-Dependent stEEring via Activation cluster matching), a training-free framework for input-dependent steering. IDEEA clusters the positive and negative activation supports per attention head, and solves an optimal-matching problem to construct a set of cluster-conditional directions, all about the target concept. At inference time, it picks from this pool of directions and uses the one that best matches the input's own activation for steering. IDEEA aligns the model toward the target concept while preserving the input's original representation, evidence that activations encoding a concept occupy several distinct sub-regions of the representation space rather than a single one. IDEEA improves the truth $\times$ info rate in TruthfulQA by an average of 9.9% (up to 23.5%) over the best input-independent baseline.

1 Introduction

LLM alignment can be achieved through inference-time activation steering, but static training-free directions are limited because concept-related activations occupy multiple sub-regions. IDEEA selects an input-conditioned direction and reports gains across steering tasks.

  • Inference-time steering adds a linear bias to selected activations without modifying model weights.It targets a small subset of the model, such as one layer or selected attention heads.
  • Static training-free methods use one steering operator for every input, despite concept activations spanning distinct sub-regions and causing unexpected failures.The limitation motivates conditioning the intervention on each input’s activation.
  • IDEEA is a training-free framework that selects a steering direction conditioned on the input at inference time.It clusters positive and negative activation supports and chooses the cluster-conditional direction best matching the input representation.
  • IDEEA preserves original input representations while avoiding refusal-collapse failures during inference.This supports steering toward the target concept without collapsing outputs into uninformative refusals.
  • IDEEA generalizes across truthfulness, social behavior, political polarity, and toxicity mitigation, with consistent gains over training-free baselines.The four evaluated steering tasks cover distinct target concepts.

2 Related Work

Training-free steering methods derive directions from contrastive activation spaces, but most apply one direction to every input. IDEEA addresses this input-dependence gap through activation clustering and input-conditioned selection.

  • Optimization-based steering can be effective but carries compute overhead and lacks a transparent geometric interpretation of its learned direction.Training-free steering instead extracts directions directly from contrastive activation space.
  • Mass-mean steering computes the difference between mean positive and negative activations and applies the resulting invariant direction across inputs.ITI applies it to selected attention heads, while CAA applies it across an entire layer.
  • SAE steering recovers decoder directions for monosemantic features from pretrained sparse autoencoders and adds them to the residual stream.Public feature suites make this practical without retraining the SAEs per task.
  • Existing training-free methods commit to one direction or projection regardless of an activation’s location in the representation manifold.SEA maximizes cross-covariance with positive activations and minimizes it with negative ones, but applies the same projections to all inputs.
  • IDEEA extends training-free steering with input-dependent direction selection at inference time.This distinguishes its framework from methods that keep the steering direction fixed across inputs.

3 Preliminaries

Training-free steering collects labeled activations, derives directions, and injects them during generation. The preliminaries define per-head activation data, baseline intervention, and head selection based on concept separability.

  • Training-free steering typically collects labeled activations, derives steering directions, and injects them during the forward pass at inference time.IDEEA adopts this pipeline and adds input-dependent direction selection.
  • 3.1 Dataset Setup: The labeled dataset pairs conversation histories with y_i ∈ {+1, −1}, indicating whether the target concept is preserved or violated.Examples of concepts include truthfulness, toxicity, and safety.
  • 3.2 Activation Collection: Activations are collected from multi-head attention outputs, with per-head activations indexed by layer and head location.The model maintains token-level embedding streams through layered attention and residual computations.
  • 3.2 Activation Collection: The activation dataset records each head’s activation at the residual-stream position corresponding to the last input token.These labeled per-head activations provide the basis for forming steering directions.
  • 3.3 Input-Independent Steering: The baseline steering direction is the difference between mean positive and mean negative activations, normalized and scaled before being added to the residual stream.Its strength uses a global α and head-wise standard deviation σ_l,h.
  • 3.3 Input-Independent Steering: Heads are ranked by held-out linear-probe accuracy, and intervention is restricted to the top-K heads because many heads encode little concept signal.Chance-level heads can inject noisy directions and degrade model utility.
  • 3.3 Input-Independent Steering: Input-independent steering uses the same direction v_l,h regardless of the input.This input-agnostic construction is the central limitation addressed by IDEEA.

4 IDEEA

IDEEA addresses the limits of one static steering direction by clustering activation supports, optimally matching cluster directions, and selecting an input-conditioned direction at inference time.

  • Framework: IDEEA selects a steering direction conditioned on each input at inference time, rather than applying one static direction to every input.The framework extends the training-free pipeline with clustering, optimal matching, and per-input direction selection.
  • Clustering: K-means clusters positive and negative activation supports separately for each attention head, with equal cluster counts required for bijective matching.The method uses K-means for simplicity, while allowing other unsupervised clustering algorithms.
  • Optimal Matching: Each matched positive-negative cluster pair defines a steering direction, and the optimal bijection selects mutually coherent directions across the concept’s activation-space partition.Coherence is measured by average pairwise cosine similarity among the cluster-conditional directions.
  • Optimal Matching: The matching problem is an NP-hard Quadratic Assignment Problem, but exact enumeration remains tractable for the small cluster counts typical of semantic clustering.For larger cluster counts, the paper discusses FAQ approximation and a Linear Assignment relaxation with a polynomial-time Hungarian-algorithm solution.
  • Steering: The method operates on a sparse subset of attention heads, whose inter-head geometry can be sensitive to additive perturbations.The selected heads rarely lie in the same layer, motivating attention to the discrepancy introduced by steering.
  • Steering: At inference time, IDEEA chooses a direction aligned with the current activation or associated with its nearest cluster, and applies it at the last-token position.Both strategies condition the direction on the current activation, allowing the intervention to adapt to the processed input.

5 Experiments

Experiments evaluate IDEEA across six instruction-tuned models and four target concepts, comparing it with established training-free baselines. IDEEA generally improves TruthfulQA and transfers beyond truthfulness while reducing refusal-driven failures.

  • 5.1 Setup: Top-K head selection is supported by probe accuracy concentrated in middle layers, with close agreement between train and held-out validation splits.Many heads remain near chance, while a smaller subset shows substantially higher concept-related linear separability.
  • 5.2 Main Results: IDEEA variants achieve the best TruthfulQA T×I score on four of six models and consistently outperform the unsteered base.Across models, min-perp gains 34.2% over base and nearest-cluster gains 28.5%, compared with 17.4% for SEA.
  • 5.2 Main Results: +292.1% is nearest-cluster’s average gain over the unsteered model across four dictator-game traits, exceeding all training-free baselines and sys prompt weak at +271.5%.The stronger system-prompt condition is treated as an artificial upper bound because it leaks the persona’s decision criteria.
  • 5.3 Analysis: On Llama2 7B, ITI’s refusal rate reaches .31, whereas min-perp reduces refusals to .08 while retaining or improving truth rate.The same pattern of ITI and CAA entering refusal modes while cluster-based steering avoids them appears across other models.
  • 5.4 Ablations: Nearest-pos-neg trails nearest-cluster on all six models by .033 on average and up to .066, while auto-nc underperforms nearest-pos-neg across all six models.These ablations attribute useful regularization to the QAP-optimal matching and symmetric cluster-count structure.

6 Conclusion

IDEEA is a training-free, input-dependent steering framework that selects directions conditioned on input activations. Across models and tasks, it improves steering while avoiding refusal collapse and exploiting multimodal activation structure.

  • IDEEA selects a steering direction conditional on the input activation at inference time.
  • Across six open-weight language models, IDEEA roughly doubles the strongest baseline’s TruthfulQA truth × info rate gain over the unsteered model.
  • IDEEA’s superior steering effect generalizes to the dictator game, political polarity steering, and toxicity mitigation.
  • IDEEA avoids ITI’s refusal-collapse failure mode and exploits multimodal activation structure that static directions average away.

Limitations

The method’s multi-head, multi-layer intervention can induce activation drift that makes deeper heads’ cluster fits suboptimal. Activation steering also carries misuse risks requiring constrained deployment and safety evaluation.

  • Perturbing early residual-stream layers can cause deeper heads to see activations that drift from their calibration distribution.This drift may render per-head cluster fits no longer optimal.
  • Quantifying inter-layer drift could inform methods that condition on both inputs and earlier perturbations.
  • Unlike IDEEA’s multi-layer intervention, CAA sidesteps this drift by intervening at only a single layer.
  • Inference-time activation steering could be misused to elicit unsafe, biased, deceptive, or otherwise undesirable behavior.The paper recommends restricting target concepts, auditing outputs, and pairing steering with safety evaluation.

A Compute Cost

The reported reproduction sweep requires substantial evaluation compute, but IDEEA’s intervention-specific GPU cost is paid once and reused, leaving per-configuration steps CPU-only.

  • 1,200 L40S GPU-hours are required to reproduce the reported sweeps.This estimate excludes additional exploratory development-time compute and is dominated by evaluation-time generation.
  • Each cluster-count setting takes under 30 minutes for CPU K-means clustering and exact QAP matching.Activation collection and each steering or evaluation hyperparameter setting take approximately 30 minutes per L40S GPU setting.
  • IDEEA collects activations in one GPU forward pass, saves them, and reuses them across the sweep.Every per-configuration intervention step is CPU-only, unlike fine-tuning, which requires heavy GPU usage for every configuration.
  • Evaluation-time generation is excluded from the GPU-cost comparison because both steering and weight-update methods would incur it identically.

B Implementation and Seeding

Experiments use fixed-seed implementation settings, while the sweep-cost comparison treats IDEEA’s activation collection as a one-time expense and fine-tuning as configuration-dependent.

  • All reported runs use random seed 0, with deterministic clustering settings including random_state=0 and n_init=10.Models use bfloat16 weights and greedy decoding for TruthfulQA; exact QAP is exhaustively solved over nc ∈ {2, ..., 6}.
  • IDEEA’s GPU cost remains constant as sweep size A grows because activation collection is paid once and reused.Fine-tuning instead pays for a full training run per configuration.
  • Evaluation-time generation is excluded from Table 6 because it is identical across the compared methods.

C Artifacts and Data

The evaluation uses public datasets and open-weight models under stated licenses, with English-only prompts and no new human-subject data. TruthfulQA analyses compare protocols, hyperparameter configurations, and clustering effects across six models.

  • Public datasets and open-weight models are used only for research evaluation under stated licenses or access terms.
  • All language data and prompts are English, and no new human-subject data are collected.
  • Table 7 optimizes TruthfulQA T×I over the full evaluation set without cross-validation; its protocol agrees exactly with Table 1 on 26 of 39 cells.
  • The alternative protocol differs from Table 1 by a mean absolute T×I difference of .007 and a maximum difference of .038.
  • Hyperparameter searches differ by method, while min-perp and nearest-cluster additionally sweep the number of clusters, nc.
  • Across six models, clustering gains increase with nc before reaching a plateau.

G Full Structural-Constraint Ablation Results

The additional evidence examines structural constraints, geometric projections, safety scoring, and comparison baselines. Cluster-based shifts consistently align activation distributions across modes and projections, while evaluation protocols distinguish valid baselines from artificial upper bounds.

  • H Additional Geometric Evidence: Cluster-shifted negatives align with positive activation support across multiple modes, whereas one ITI direction rigidly translates the negative cloud.
  • H Additional Geometric Evidence: Across PCA, t-SNE, and UMAP projections, cluster-shifted distributions overlap the positive support more than ITI-shifted distributions.
  • H Additional Geometric Evidence: The projection comparison indicates that the qualitative geometric pattern does not depend on a single dimensionality-reduction method.
  • Safety Evaluation: TET responses count as successful defenses only when an LLM judge returns SAFE; INVALID outputs are excluded before SAFE scoring.
  • Safety Evaluation: The safety judge labels harmful compliance UNSAFE, unusable or looping text INVALID, and coherent safe redirection SAFE.
  • Baselines: Strong system-prompt baselines state the decision rule and are reported as artificial upper bounds because they leak evaluation criteria.

K Dictator Game Synthetic Dataset

The synthetic dictator-game dataset probes four target traits through contrastive contexts and open-ended scenarios. Llama3 8B generates repeated responses for each scenario-context pairing to support trait evaluation.

  • Dataset Construction: Four target traits each receive a hand-written positive context, a contrastive negative context, and ten open-ended scenarios.
  • Dataset Construction: For every scenario-context pair, Llama3 8B generates ten responses at temperature 1, producing 100 responses per trait-context pair.
  • Competitive Trait: The competitive trait contrasts relative advantage with cooperative shared success across promotion, scholarship, sales, and team-competition scenarios.
  • Inequality-Aversion Trait: The inequality-aversion trait probes equalization and disparity reduction through pay, resource allocation, recognition, and workplace scenarios.
  • Self-Interest Trait: The self-interest trait contrasts personal gain with altruism using dilemmas involving wallets, donations, mispricing, credit, and shared resources.
  • Social-Welfare Trait: The social-welfare trait emphasizes collective well-being through vaccination, housing, assistance, recycling, community action, and charitable giving scenarios.
Loading 2609.02089v1…