PaperlayerView exact revision on arXiv

Source-linked AI summary

Compositional Multilingual and Behavioral Attribute Steering

Hyun Gu Kang, Daniil Gurgurov, Tanja Baeumel, Josef van Genabith, Simon Ostermann

arXiv:2609.08410v1cs.CL

TL;DR

The paper investigates whether independently extracted steering vectors can be composed additively without losing their intended effects. It evaluates language, jailbreak, and conciseness across instruction-tuned models, finding that composition works when vectors use suitable layer–strength configurations and that approximate orthogonality is consistent with this behavior.

  • Problem

    Existing evidence on additive, training-free composition is mixed, leaving unclear when independently extracted steering vectors can control multiple attributes simultaneously.

  • Method

    The study evaluates single-attribute, pairwise, and three-way additive steering across four instruction-tuned models, sweeping layers and strengths and analyzing vector geometry.

  • Results

    Single-attribute steering works for all three attributes within suitable configurations, while two-attribute composition succeeds when each vector is injected at its own best-performing layer.

  • Takeaways & Limitations

    Approximate orthogonality in the residual stream is consistent with successful per-layer additive composition and helps explain why it can avoid destructive interference.

  • Takeaways & Limitations

    The experiments use instruction-tuned models and fixed intervention strengths, so results may not directly generalize to base models, different prompting formats, or dynamically tuned settings.

Abstract

from arXiv · show

This study examines the compositionality of steering vectors for language and behavioral control in large language models. Focusing on language, jailbreak, and conciseness, we investigate whether additive, training-free composition of attribute steering vectors can preserve the intended steering effect of each attribute, across four instruction-tuned models from two model families and two size scales. We find that single-attribute steering is reliable for all three attributes, but only within an appropriate combination of intervention layer and steering strength, with abstract behaviors (jailbreak, conciseness) favoring middle layers and language favoring earlier layers. We show that additive composition of two attribute vectors succeeds in steering both attributes simultaneously when each is injected at its own best-performing layer, and that this partially extends to three simultaneously composed attributes, addressing an inconsistency left open by prior work on training-free composition. We further analyze the geometric properties of these steering vectors, finding that they are approximately orthogonal in the residual stream, consistent with their compositional behavior.

1 Introduction

The paper asks whether independently extracted steering vectors can be added during one forward pass without degrading one another. It argues that additive composition works when each attribute uses an appropriate layer–strength configuration, with vector geometry helping explain the result.

  • Activation steering adds contrastively extracted vectors to the residual stream at inference time, offering a lightweight alternative to prompting or fine-tuning.
  • Prior evidence on additive, training-free composition is mixed, with reports ranging from destructive interference to mild success.
  • Additive composition succeeds reliably when each attribute vector is injected at its own best-performing layer and strength.
  • The study tests language, jailbreak, and conciseness steering across four instruction-tuned models from two families and two sizes.
  • The experiments extend pairwise composition to a triple and analyze whether approximate orthogonality relates to successful composition.

2 Related Work

Prior work establishes reliable single-attribute steering but leaves additive composition under-characterized. This paper addresses that gap by studying when training-free vector addition succeeds and relating outcomes to vector geometry.

  • Single-attribute steering has controlled properties including sentiment, toxicity, refusal, and language, but does not establish multi-attribute compositionality.
  • Additive composition injects multiple attribute vectors without further optimization, but existing studies report mixed outcomes.
  • Some prior work reports destructive interference, whereas separate-layer injection has shown limited success for two instruction-following constraints.
  • Most subsequent approaches learn additional parameters at the input, output, or activation level instead of resolving plain additive composition.
  • The paper claims novelty in characterizing additive composition conditions, testing multilingual–behavioral combinations, and scaling across model families and sizes.

3 Methodology

The methodology extracts normalized contrastive steering vectors from final post-instruction residual activations and applies them at selected layers. Compositional steering injects multiple vectors in one forward pass, using each vector’s corresponding layer.

  • Steering vector extraction: The procedure extracts activations only at the final post-instruction token position rather than pooling across the sequence.
  • Steering vector extraction: DiffMean steering vectors are constructed from differences between mean activations of two balanced conditions.
  • Steering vector extraction: Each resulting vector is normalized so that intervention strength α controls magnitude independently of the raw vector norm.
  • Steering vector application: A steering vector modifies the hidden representation at a chosen layer, and the altered activation is passed to subsequent layers.
  • Compositional steering: Compositional steering applies multiple attribute vectors within the same forward pass, assigning each vector to its corresponding layer.
  • Compositional steering: When multiple vectors share a layer, their scaled vectors are summed before being added to the representation.

4 Experimental Setup

The study extracts language, jailbreak, and conciseness steering vectors from contrastive conditions and evaluates them across four instruction-tuned models. It combines objective-specific metrics using harmonic means while sweeping intervention layers and steering strengths.

  • Steering objectives: The three steering objectives are language, jailbreak, and conciseness, each extracted as a DiffMean vector from paired contrastive instructions.Language contrasts English with target-language instructions; jailbreak contrasts harmful with harmless instructions; conciseness contrasts instructions with and without a shortening instruction.
  • Evaluation metrics: Each objective is evaluated with objective-specific metrics that are aggregated across generations and combined using the harmonic mean.Language uses LFS and OR, jailbreak uses JBS and OR, and conciseness uses CCS and OR.
  • Models: The evaluation uses Llama-3.1-8B-Instruct, Llama-3.1-70B-Instruct, Qwen2.5-14B-Instruct, and Qwen2.5-32B-Instruct.These models span two model families and two size scales.
  • Evaluation metrics: LFS measures target-language success, while OR scores whether outputs remain relevant, coherent, and interpretable.LFS counts responses whose fastText target-language probability exceeds 95%; OR uses normalized scores from 0 to 1.
  • Evaluation metrics: JBS measures harmful-instruction compliance, and CCS measures the relative reduction in output length caused by conciseness steering.JBS maps Safe, Controversial, and Unsafe classifications to normalized scores; CCS is 1 − min(L_steered/L_baseline, 1).
  • Intervention settings: Single-attribute experiments sweep four intervention layers and five steering strengths, with different α ranges for the Llama and Qwen model families.Llama uses α ∈ {1.0, 2.0, 4.0, 6.0, 8.0}; Qwen uses α ∈ {10.0, 20.0, 40.0, 60.0, 80.0}, reflecting different vector magnitudes.
  • Baselines and generation: Prompt baselines append explicit objective instructions, using three suffixes per single attribute and best-performing combinations for compositional settings.Conciseness conditions allow 512 new tokens, while other conditions allow 64.

5 Experimental Results

Single-attribute steering is effective but depends on the objective-specific layer–strength configuration. Additive composition is strongest for language plus jailbreak, weaker and more variable for language plus conciseness, and exceeds prompt baselines in the exploratory three-attribute setting.

  • Single-Attribute Steering: Language steering is effective, but performance varies substantially across models, intervention layers, and steering strengths.Scores at the latest layer are consistently close to zero, indicating that success requires an appropriate layer–strength combination.
  • Single-Attribute Steering: Jailbreak steering generally peaks at the second-earliest evaluated layer across models.Qwen2.5-32B is a minor exception, with a slightly higher score at layer 44 than at layer 28.
  • Single-Attribute Steering: Conciseness steering is generally strongest at the second-earliest evaluated layer across models.Qwen2.5-32B is a small exception, achieving a slightly higher score at layer 44 than at layer 28.
  • Two-attribute Steering: Two-attribute experiments select each attribute’s best steering layer and strength before testing additive composition against prompt baselines.The study evaluates language plus jailbreak and language plus conciseness across models and language pairs.
  • Two-attribute Steering: Language plus jailbreak composition exceeds prompt-based baselines on average for all four models.Llama-3.1-70B shows the largest and most consistent margin, while Llama-3.1-8B has the widest spread across language pairs.
  • Two-attribute Steering: Language plus conciseness composition generally falls below prompt baselines and has greater variance across language pairs.Llama-3.1-8B is weakest, while Qwen2.5-14B is strongest with most language pairs scoring high; the paper relates the variability to less consistent orthogonality.
  • Three-attribute Steering: Three-attribute composition achieves higher overall steering success than prompt-based steering across all models.Performance is more variable across language pairs than in either two-way composition, consistent with allocating three attributes across a fixed set of intervention layers.

6 Geometry of Steering Vectors

The steering vectors occupy depth-dependent geometric structures: language vectors diverge across depth, while language, jailbreak, and conciseness vectors are generally near-orthogonal. Vector magnitudes also grow with depth, but raw magnitude does not determine intervention strength because vectors are normalized before use.

  • Language-vector geometry: Language-vector similarity is high early (≈0.6-1.0) and decreases toward later layers.This indicates increasing divergence among language attributes toward the output.
  • Cross-attribute geometry: Language, jailbreak, and conciseness vectors remain close to orthogonal across depths and models.The main exception is a slight late-layer positive correlation between language and conciseness, higher for languages tokenized into more tokens.
  • Vector magnitude: Vector magnitude grows sharply with depth, with language and jailbreak vectors increasing fastest while conciseness remains comparatively small.Raw magnitudes differ substantially between Llama and Qwen models, and normalization means they do not directly determine steering strength.
  • Compositionality: Orthogonality is consistent with successful additive composition, while less-orthogonal language–conciseness vectors correspond to less consistently successful composition.The three-way result also follows this pattern, succeeding with more variance when all pairwise similarities are near zero.

7 Conclusion

The study evaluates compositional steering for language, jailbreak, and conciseness across four models. It finds that reliable single-attribute control requires attribute-specific layer–strength settings, while additive multi-attribute steering can work when vectors are injected at their respective best-performing layers.

  • The study examines language, jailbreak, and conciseness steering across four models.
  • Single-attribute steering is effective only with an appropriate layer–strength configuration.Jailbreak and conciseness favor middle layers, whereas language favors earlier layers.
  • Additive, training-free composition of two attributes can succeed when each vector is injected at its own best-performing layer.This addresses an inconsistency left open by prior work.
  • The vectors are approximately orthogonal, offering a possible explanation for why per-layer additive composition avoids destructive interference.

Limitations

The experiments are limited to instruction-tuned models and fixed intervention strengths, constraining direct generalization to other prompting formats and dynamic contexts.

  • The experiments are limited to instruction-tuned models and may not directly generalize to base models or different prompting formats.The activation-extraction setup relies on the post-instruction token position, particularly for jailbreak steering.
  • The study uses a constant intervention strength for each condition, although optimal strength may vary by context and require dynamic tuning.

Ethics Statement

The ethics statement frames jailbreak steering as a characterization of non-refusal behavior rather than a tool-building effort. It uses an existing public harmful-prompt dataset and flags that generated outputs and evaluation materials may contain harmful content.

  • Purpose and data: Jailbreak steering is included to characterize a failure mode, not to build a jailbreak tool.The study uses harmful prompts from an existing, publicly available dataset.
  • Prompt-based baselines: The combined prompt suffix requests a target language, a direct non-refusal answer, and brevity.
  • Output evaluation: The judge prompt evaluates relevance and coherence without lowering scores for language, brevity, safety, harmfulness, legality, refusal, or compliance.
  • Output evaluation: The relevance criterion asks whether the response is a relevant answer to the question.
  • Output evaluation: The coherence criterion distinguishes coherent text from gibberish, repetition, or useless content.
  • Output evaluation: Scores range from 0 for unrelated or useless text to 2 for clearly relevant, coherent, and meaningful responses.A concise response can receive 2 if it adequately addresses the question.
  • Output evaluation: Judge outputs use the format Rating: [[score]].
  • Reporting: English translations are provided for the non-English generated responses shown in Table 1.

E Disaggregated Single-Attribute Steering Results

The section disaggregates single-attribute steering results across intervention layers and steering strengths for language, jailbreak, and conciseness. It also reports aggregate language–behavior steering scores using harmonic means of the relevant metrics.

  • Single-attribute metrics: Single-attribute results are shown across intervention layers and steering strengths for all four models, with standard-deviation shading and prompt-based baselines.The underlying metrics are reported separately for each steering attribute.
  • Language steering: Language steering is evaluated with Language Forcing Success (LFS) and Output Relevance (OR).Figure 10 summarizes language-pair performance using the harmonic mean of LFS and OR.
  • Jailbreak steering: Jailbreak steering is evaluated with Jailbreak Success (JBS) and Output Relevance (OR).Figure 13 includes JBS, OR, LFS, and aggregated performance for language–jailbreak–conciseness steering.
  • Conciseness steering: Conciseness steering is evaluated with Conciseness Control Score (CCS) and Output Relevance (OR).Figure 11 aggregates LFS, OR, and CCS for language–conciseness steering.
  • Composed aggregate scores: Language–jailbreak and three-attribute aggregate scores combine LFS, OR, JBS, and, when applicable, CCS through harmonic means.The corresponding figures report results by language pair across the four models.

G Disaggregated Compositional Steering Results

The section examines individual metrics and geometric relationships in compositional steering across language–conciseness, language–jailbreak, and three-attribute settings. It compares composition conditions by language pair and investigates whether cosine similarity relates to compositional steering degradation.

  • Disaggregated compositional results: Compositional results are reported separately for language–conciseness, language–jailbreak, and three-attribute steering across all four models.The individual metrics underlying the aggregate compositional scores are presented separately for each composition setting.
  • Metric distributions: Individual metric distributions, language-pair means, and prompt-baseline means are shown for language–conciseness, language–jailbreak, and three-attribute steering.Figures 14–16 use boxes, individual points, and white diamonds to distinguish these summaries.
  • Vector geometry: Cosine similarity is measured across depth between jailbreak and conciseness vectors and each language vector for four instruction-tuned models.Figures 17–20 report these comparisons for Llama-3.1-8B-Instruct, Llama-3.1-70B-Instruct, Qwen2.5-14B-Instruct, and Qwen2.5-32B-Instruct.
  • Composition comparisons: Figure 21 compares steering scores across all composition conditions by language pair and model.The scores are computed from aggregated evaluation metrics.
  • Per-language analysis: The analysis tests whether lower compositional steering success is linked to higher cosine similarity between language and behavioral-attribute vectors.Figure 21 compares per-language steering success for single-, two-, and three-attribute steering in this analysis.
Loading 2609.08410v1…