PaperlayerView exact revision on arXiv

Source-linked AI summary

Steering Interference Reflects the Model's Defaults, Not the Behavior Directions

Srikanth Malla, Chiho Choi, Joon Hee Choi

arXiv:2609.06951v1cs.LGcs.AI

TL;DR

Activation steering is intended to control one behavior without disturbing others, but the determinants of its side effects remain unclear. The paper measures effects in generated text across 24 behaviors and ten instruction-tuned models, comparing real and random pushes and testing geometry on fully held-out behaviors. It finds that side effects are primarily set by model defaults rather than the steered direction, so disentangling directions alone does not make steering modular.

  • Problem

    Activation steering assumes modular control, but it is unclear whether collateral behavioral changes are determined by the targeted behavior’s direction or by the model itself.

  • Method

    The paper reads behavioral effects from judged generated text, compares real steers with matched random pushes, and evaluates direction geometry with leave-one-behavior-out tests across 24 behaviors and ten models.

  • Results

    Steering profanity raises toxicity by 3.4 standard deviations while reverse steering changes profanity by 0.0, and matched random pushes reproduce real-steer rankings at rank correlation 0.90.

  • Takeaways & Limitations

    Steering behaves as a perturbation whose endpoint is fixed by the model’s default behaviors, so direction disentanglement alone cannot ensure modular control.

  • Takeaways & Limitations

    The default set is model-specific, why its pull weakens at the largest scale is unexplained, and all ten models use the same 24 behaviors.

Abstract

from arXiv · show

Activation steering promises modular control of language model behavior: a behavior such as politeness corresponds to a direction in a model's activations, and adding that direction while it generates should switch the behavior on and leave everything else alone. It does not. We ask what decides which other behaviors move, and by how much, and find that it is the model rather than the behavior being steered. A steer relaxes the model toward a small set of behaviors it already favors, chiefly refusal, sycophancy, and poeticism, and that set is much the same whatever is steered. Three results across 24 behaviors and ten instruction-tuned models support this, every effect read off the generated text by a language-model judge rather than off a probe. That readout matters: all 24 behaviors are linearly decodable, but only 20 change what the model writes. First, a direction carrying no behavioral content, matched to a real steer only in the size of the vector it adds, moves the same behaviors in the same order as real steers do, while producing none of the behaviors that need a specific direction. Second, most interference runs one way, so it cannot be an overlap between two directions: steering profanity makes the model toxic, while steering toxicity leaves profanity untouched. Third, with a behavior held out entirely, geometry measured on the others explains almost none of the interference it takes part in. The account holds on all ten models, the pull toward defaults strongest below 10B parameters and weakening in each family's largest. Reading a steer as a perturbation whose endpoint the model fixes implies that disentangling behavior directions cannot by itself make steering modular.

1 INTRODUCTION

The paper tests whether activation-steering side effects are determined by behavior-direction geometry or by the model itself. Across its argument, it finds that arbitrary pushes move similar default behaviors, while interference is asymmetric and poorly predicted by geometry.

  • Motivation: Activation steering assumes that adding a behavior direction switches on its target while leaving the rest of the output unchanged.The technique adds a fixed residual-stream direction during generation and is valued as a cheap, reversible control surface.
  • Research question: The paper asks whether a steer’s collateral effects depend on the targeted behavior or on the model receiving the perturbation.Its central account treats the endpoint of a steer as model-determined rather than as a direct consequence of the edited behavior.
  • Competing account: The geometric account predicts symmetric interference because overlap assigns one value to each behavior pair, and it should transfer to behaviors excluded from fitting.The paper tests both equal bidirectional effects and held-out predictive transfer.
  • Measurement: Generated text, rather than activation probes, is used to measure behavioral effects because projections would rise from the added direction even without output change.The measurement therefore requires a language-model judge and behavior-specific steering calibration.
  • Core result: Steering profanity raises toxicity by 3.4 standard deviations, whereas steering toxicity changes profanity by 0.0, demonstrating one-way interference.This asymmetry cannot be represented by a single symmetric overlap value.
  • Core result: A random push matched only in vector size reproduces the ordering of side effects at rank correlation 0.90 while producing none of the direction-specific behaviors.The contribution summary identifies the model’s default set as the source of common side effects.

2 BEHAVIOR DIRECTIONS, STEERING, AND WHAT GEOMETRY PREDICTS

This section defines behavior directions, steering, and the interference matrix, then states two commitments of the geometric hypothesis: symmetry and transfer to held-out behaviors.

  • Behavior directions: A behavior direction is the difference between mean residual-stream activations for responses with and without that behavior.The direction is one activation-space vector per behavior.
  • Steering: Steering adds a behavior direction to the residual stream during generation, scaled by a behavior-specific steering coefficient.The premise is that this activates the target behavior without changing the rest of the output.
  • Interference matrix: The interference matrix M records how steering behavior i changes the judged expression of behavior j, with diagonal entries representing target effects and off-diagonal entries representing side effects.Each ordered pair contributes one matrix entry.
  • Geometric hypothesis: The geometric hypothesis says that Mij depends on the cosine overlap between behavior directions i and j.This makes direction overlap the proposed determinant of cross-behavior interference.
  • Symmetry: Because cosine overlap is symmetric, geometry predicts that steering i disturbs j exactly as much as steering j disturbs i.The matrix can be decomposed into symmetric and one-way components to measure departures from this commitment.
  • Transfer: A geometric account must predict interference for a behavior held out together with all 38 of its behavior-pair entries.This leave-one-behavior-out test prevents a predictor from exploiting that a behavior is generally easy to induce.

3 METHOD

The method measures behavior changes from generated text, constructs an interference matrix, analyzes its structure, compares real and random pushes, tests geometry on held-out behaviors, and replicates the pipeline across models.

  • Pipeline: The pipeline extracts one direction per behavior, calibrates steering, judges generated text, and repeats across ordered pairs to fill matrix M.It is presented on Qwen2.5-7B and then repeated unchanged on nine further models.
  • Direction and calibration: Each behavior direction is formed from 12 contrastive prompt pairs and evaluated on held-out pairs using AUROC against shuffled-label controls.Steering coefficients are calibrated with judge ratings on 32 neutral prompts while monitoring repetition.
  • Interference measurement: Each matrix entry Mij is the change in the judge’s rating of behavior j when behavior i is steered, normalized by a pooled standard deviation.An entry is significant when a prompt bootstrap interval excludes zero.
  • Matrix structure: The analysis decomposes M into ordered patterns and assigns each behavior scores for driving others and being induced, whose difference defines source strength.These quantities organize the matrix’s dominant structure.
  • Random push: Random directions replace behavior directions in a matched-push comparison using the same added size and coherence floor.This tests whether side-effect rankings require behavior-specific content.
  • Geometry prediction: Geometry is evaluated with ridge and boosted-tree predictors under leave-one-behavior-out validation that removes all 38 pairs involving the held-out behavior.The predictors use overlap, directional features, norms, and on-target effects.
  • Figure 2: The judge-grounded matrix has rows for steered behaviors and columns for judged behaviors, with diagonal entries as on-target effects and off-diagonal entries as side effects.Both axes are ordered by source strength, from resistant behaviors to sinks.
  • Replication: The full pipeline is replicated on ten instruction-tuned models spanning four families and 1.5B to 72B parameters.Layers, coefficient grids, and thresholds remain fixed to the Qwen2.5-7B settings.

4 LINEAR DECODABILITY DOES NOT IMPLY STEERABILITY

Linear decodability does not guarantee that steering changes model behavior: only 20 of 24 decodable behaviors are steerable, and the resulting interference is widespread, asymmetric, low-rank, and organized by a model-level ranking.

  • Steerability: All 24 behaviors are linearly decodable, but only 20 change what the model writes under judge-in-the-loop calibration.Every direction reaches held-out AUROC 1.000, while shuffled-label controls average 0.50.
  • Interference prevalence: 149 of 380 off-diagonal entries are significant, and the average side effect is 0.37 pooled-standard-deviation units versus 2.07 for the average on-target effect.The number of significant effects is far above the roughly 19 expected by chance.
  • Asymmetry: Steering profanity moves toxicity by +3.4 pooled standard deviations, whereas steering toxicity moves profanity by +0.0.The one-way component carries 0.68 of total entry magnitude, showing that the asymmetry is widespread.
  • Default ranking: A random push reproduces the model’s default behavior ranking, while remaining unable to produce lexically specific behaviors.This supports model-level defaults rather than a claim that every behavior moves indiscriminately.
  • Geometry: Two thirds of the interference lies outside the reach of symmetric geometric overlap.The result follows because overlap cannot represent the observed one-way component.
  • Source strength: A single ranking organizes much of the one-way interference, from behaviors that drive others to behaviors that are easily induced.The matrix’s dominant structure is therefore directional rather than pairwise symmetric.
  • Behavior classes: Ten of twenty behaviors are significant sinks led by sycophancy, refusal, and poeticism, while six significantly resist induction.Prosocial and positive behaviors occupy the resistant end; failure, negative, and filler behaviors occupy the sink end.

6 A RANDOM PUSH REPRODUCES THE SAME RANKING

A behavior-free random push reproduces the ordering of real steering side effects, while default behaviors emerge independently of the targeted direction. Held-out tests further show that geometry explains little of the interference and does not predict its magnitude.

  • 0.90 rank correlation shows that a behavior-free random push reproduces the ordering induced by real steers.Across values, the correlation is 0.92 Pearson; the random push is matched to real steers by residual-stream norm and coherence.
  • A random push significantly induces ten behaviors and suppresses seven, with refusal, numeric, poeticism, and sycophancy among the leading induced behaviors.Five of the six resistant behaviors are suppressed, along with confidence and verbosity.
  • 0.16 pooled-standard-deviation units describes random-push toxicity, while profanity remains exactly zero, revealing a one-way default drift rather than symmetric overlap.The toxicity interval is [0.07, 0.28]; profanity is among the lexically specific behaviors requiring a targeted push.
  • 0.75 rank correlation links unsteered behavior expression to resistance, not inducibility, across 20 behaviors.Fourteen non-resistant behaviors average 0.04 judge points out of 10 on neutral prompts, versus 3.78 for six resistant behaviors.
  • Held-out geometry predicts interference weakly: the best Spearman correlation is 0.41, while no variant reaches held-out R2 of 0.10.The best predictor of the one-way structure reaches only 0.16, and richer probes or concept cones do not rescue the result.
  • Orthogonalized and anchored steering leave side effects essentially unmoved while reducing the on-target effect from 2.2 judge points to 1.2 and 1.8.Their pooled-standard-deviation effects are −0.03 [−0.08, 0.02] and +0.02 [−0.02, 0.06], respectively.

8 THE ACCOUNT GENERALIZES ACROSS MODELS AND SCALE

The account generalizes across all ten models, although the pull toward default behaviors weakens in each family’s largest model. Which defaults appear varies with model and alignment style.

  • All ten models show one-way, low-rank interference, while held-out geometry fails to predict it across scales from 1.5B to 72B.The one-way component is 0.63 to 0.69 of total entry magnitude, the top three patterns capture 0.65 to 0.93 of energy, and held-out R2 ranges from −0.11 to 0.13.
  • The random-push correlation is positive in every model but weakens toward each family’s largest, reaching 0.54 for Qwen2.5-72B, 0.38 for Llama-3.3-70B, and 0.15 for Gemma-2-27B.The correlation is as high as 0.90 in smaller and mid-size models.
  • The default set tracks alignment style: refusal leads induced behaviors in most refusal-trained families but is weakest among Mistral’s seven.Thus the mechanism generalizes while the particular behaviors forming the set remain model-specific.

9 RELATED WORK

Prior work documents steering side effects and often explains interference geometrically, but this paper argues that text-based, held-out evaluation instead identifies the model’s default set as the key pattern. The distinction also preserves a gap between behaviors that are decodable and behaviors that steering changes.

  • Prior studies document unreliable, entangled steering effects that vary with behavior and depth, including non-isolated safety directions.The paper positions its contribution as identifying something that predicts these side effects.
  • Geometric accounts explain cross-behavior interference through behavior-subspace overlap, but this paper tests those predictions using generated text and held-out behaviors.The paper contrasts its measurement with prior analyses based on supplied-answer probabilities or fitted behavior geometry.
  • Holding out all 38 pairs involving each behavior changes the interpretation: the observed ranking is attributed to the model’s default set rather than representation geometry.The defaults are behaviors the model settles into when pushed in any direction.
  • All 24 behaviors are linearly detectable, but only 20 are confirmed by a judge to change what the model writes.The paper treats this dissociation as a measurement issue that a readout blind to behavioral change would miss.
  • The account complements work on off-distribution drift and default assistant modes by identifying a broader default-behavior pattern across steering directions.The cited related work includes coherence degradation, safety bypass, and persona drift under strong or shifted steering.
  • The claim is bounded because the default set is model-specific, the scale-related weakening is unexplained, and most pairwise interference remains outside both geometry and the one-way ranking.All ten models use the same 24 behaviors, while judge robustness and cross-model structural replication address two additional concerns.

AI USE STATEMENT

The authors used language models to generate contrastive behavior examples and scaffold code, then reviewed and tested all AI-assisted work before release and analysis.

  • AI-assisted materials: 288 contrastive response triples defining 24 behaviors were generated with a language model under a shared-user-turn specification.The full set is released with the code and listed by behavior in Appendix B.
  • Review and release: Authors read the generated triples before use and released them so any effects could be checked.
  • Review and release: Authors read and tested AI-scaffolded code, and regenerated every reported number and figure from saved judge ratings through the released pipeline.
  • Scope: The experiments use language models as both the in-the-loop judge and a research aid, while the authors claim responsibility for experiments, analyses, and claims.

A METRICS USED IN THIS PAPER

The paper defines metrics for reading behavior directions, interference structure, prediction, uncertainty, and output coherence. Several metrics distinguish symmetric from directional effects and evaluate results out of sample.

  • Readout and decodability: Held-out AUROC evaluates whether projections onto a behavior direction separate behavior-present from behavior-absent responses without fitting and testing on the same pairs.A same-pair AUROC of 1.0 is uninformative, so held-out and shuffled controls are used.
  • Association metrics: Spearman ρ measures agreement between orderings, while Pearson r measures closeness to a straight-line relationship.
  • Prediction: Out-of-sample R2 measures explained variance relative to predicting the mean, and can be negative when predictions are worse than that baseline.The paper gives Qwen2.5-3B’s out-of-sample value as −0.11.
  • Interference structure: Singular-value energy fractions quantify how much of the interference matrix is carried by leading low-rank patterns.When the top three of twenty components carry 0.71 of the energy, the matrix is largely organized by three patterns.
  • Interference structure: The symmetric matrix component captures mutual effects, whereas the antisymmetric component captures one-way effects unreachable by cosine-overlap accounts.The decomposition is M = S + A, with cosine similarity predicting no antisymmetric component.
  • Interference scale: Interference entries standardize each target behavior by its pooled judge-score standard deviation across baseline and steered conditions.An entry of 0.37 therefore represents 0.37 natural-spread units for that behavior.

B BEHAVIOR SET

The study defines 24 behaviors with contrastive response triples spanning register, affect, interpersonal stance, and content style. Shared prompts within each triple isolate behavioral expression from topic.

  • Behavior definitions: Each behavior is represented by 12 contrastive triples containing a shared user prompt, behavior-present response, and behavior-absent response.
  • Behavior definitions: Shared user prompts make the difference of class means target the behavior expressed in the response rather than the topic.
  • Examples: The examples include refusal about lock picking and archaism in a tea-making response, each contrasted with an ordinary alternative.
  • Behavior coverage: The behavior set spans register, affect, interpersonal stance, and content style instead of concentrating in one category.The full definitions are provided in Table 2 and the complete triples are released with the code.

C DIRECTION EXTRACTION AND LINEAR DECODABILITY

The paper extracts behavior directions from residual-stream activations and evaluates them with held-out linear decodability and judge-based steering. It separately models directional interference and tests predictors under held-out behaviors.

  • Direction extraction: Behavior directions are formed from difference-of-means residual-stream activations collected across assistant-response tokens and layers.Held-out projection AUROC uses stratified five-fold cross-validation with a label-shuffled chance control.
  • Judge readout and calibration: Steering adds each raw direction at its selected layer across coefficients {1, 2, 3, 4, 6, 8}, with 32 neutral prompts judged for every coefficient.A distinct-bigram coherence score is also computed during calibration.
  • Judge readout and calibration: Table 3 compares per-behavior held-out AUROC, shuffled controls, steering coefficients, judge self-effects, coherence, and steerability.Four behaviors are decodable but not steerable.
  • Interference prediction: Ordered-pair predictors use symmetric geometry features and directional features, with ridge and gradient-boosted regressors evaluated by leave-one-behavior-out cross-validation.Directional features are required to predict the antisymmetric component.
  • Replication: The cross-model replication covers ten instruction-tuned models from 1.5B to 72B parameters across four families, with each model self-judged.Table 4 reports replication over all 24 readouts, unlike Table 5’s steerable subset.

F COMPUTE AND REPRODUCIBILITY

A judge-ablation study shows that the paper’s main conclusions remain stable when self-judged generations are re-read by a fixed strong judge. Continuous rankings and asymmetric structure persist, while exact top-sink membership is more model-specific.

  • Experimental setup: The cross-model pipeline reuses identical hyperparameters, with each model self-judging its own generations.The primary experiments use Qwen2.5-7B-Instruct; the cross-model study reruns the pipeline on nine additional models.
  • Judge robustness: Jaccard agreement for the steerable-versus-nonsteerable boundary ranges from 0.81 to 1.00 under fixed-judge re-reading.Disagreements involve only one or two marginal behaviors in the smallest and largest models.
  • Judge robustness: The per-behavior coefficient response agrees at Spearman 0.77, while target-susceptibility rankings agree at Spearman 0.74 to 0.83.Agreement is strongest for the largest model on the susceptibility hierarchy.
  • Judge robustness: The asymmetry fraction changes only from 0.69 to 0.64, 0.65 to 0.62, and 0.63 to 0.64 across the three tested models.This indicates that the asymmetric low-rank structure is essentially unchanged under the alternate judge.
  • Judge robustness: Top-six attractor-sink overlap is lower, ranging from 0.71 to 0.33, showing that cutoff-based membership is softer than the continuous ranking.The reported set overlaps are 0.71, 0.50, and 0.33, while the mechanism remains stable.
Loading 2609.06951v1…