PaperlayerView exact revision on arXiv

Source-linked AI summary

AutoLexSteer: Automatic Contrast Construction for Lexical Activation Steering

Shuhe Wang, Lachlan Cowley, Eduard Hovy, Jey Han Lau

arXiv:2609.06879v1cs.CLcs.AI

TL;DR

Accurate lexical steering vectors are difficult to construct manually because lexical concepts lack natural response contrasts and prompt differences can introduce artifacts. The paper introduces Hangman contrasts and AutoLexSteer, which automatically builds clue-based vectors from WordNet-related words; these vectors steer among lexical meanings and show preliminary behavioral applications. The framework is limited by its dependence on lexical resources, small lexical-word coverage, and partially manual, preliminary behavioral steering.

  • Problem

    Manual construction of precise lexical steering vectors is difficult because words lack natural response contrasts and prompt differences can introduce unintended artifacts.

  • Method

    AutoLexSteer retrieves related words from WordNet, generates and filters Hangman-style association contrasts, and constructs steering vectors for target and source meanings.

  • Results

    Hangman vectors steer from related or multiple source words toward target words, including unseen related words, and show preliminary success for behavioral steering.

  • Takeaways & Limitations

    The Hangman framework supports automated lexical steering at the level of words and related word senses, with extensions to behaviors such as refusal and anti-sycophancy.

  • Takeaways & Limitations

    The framework requires lexical resources and sense definitions, is evaluated on only 15 lexical words, and behavioral steering remains preliminary and partly manual.

Abstract

from arXiv · show

Steering vectors have rapidly emerged as a popular and effective method for guiding the output of LLMs in very specific ways. But constructing accurate steering vectors is a difficult manual process due to the opacity of embeddings. We introduce Hangman, a novel type of steering vector that operates using word senses, as well as AutoLexSteer, the first fully automated process for building steering vectors. AutoLexSteer employs families of closely-related words extracted from WordNet to specify both the steering source to be avoided and the desired steering target. The steering vectors are quite precise, can be used to steer at the level of words and sets of word senses (meanings), and are able to steer certain LLM behaviors like sycophancy. The dataset and code can be found at https://github.com/ShuheWang1998/autolexsteer.

1 Introduction

The paper introduces Hangman contrastive prompts and AutoLexSteer, an automated pipeline for constructing lexical steering vectors from word-sense clues. These vectors steer from source meanings toward target meanings, including related and unseen words, with preliminary behavioral applications.

  • Motivation: Steering vectors bias generation away from a source continuation toward a target continuation, but manually constructing precise lexical contrasts is difficult.Lexical concepts lack the natural response pairs available for behavioral concepts, and loosely specified vectors can cause unwanted side effects.
  • Hangman Contrastive Pairs: Hangman contrasts represent each side with association clues, sharing matched clues while using diagnostic clues to distinguish source and target words.For TREASURY→BANK, shared clues include money and finance, while teller is diagnostic for BANK and bonds for TREASURY.
  • Results: Hangman vectors steer reliably with relatively little damage, consistent with word senses occupying stable directions across contexts.The evaluation uses an LLM judge for source-to-target completion and perplexity for damage on unrelated text.
  • Results: Averaging vectors from multiple source words can steer toward BANK and can also affect related words absent from construction, such as MORTGAGE.The authors interpret this as consistent with overlap in the representation of semantically related senses.
  • AutoLexSteer: AutoLexSteer automatically retrieves related words from WordNet, generates and filters contrastive clue lists, constructs vectors, and evaluates them without manual prompt authoring.The pipeline is designed to apply to arbitrary word pairs.
  • Behavior Steering: The framework extends beyond lexical steering to behaviors including refusal, anti-sycophancy, and sycophantic-to-honest steering.Behavioral applications are presented as a smaller set of experiments or extension with a manual related-word fix.

2 Related Work

Prior activation-steering research established inference-time latent interventions and expanded them to high-level features and refusal. Much existing work still relies on human-designed contrasts, which can capture format or label shortcuts rather than the intended concept.

  • Activation Steering: Activation steering can alter generated text without updating model weights through inference-time interventions on the residual stream.Subsequent work connected steering to representation engineering, refusal localization, and concept erasure.
  • Contrast Construction: Most existing steering methods use human-designed question-answer contrasts with hand-written positive and negative sides around an explicit answer label.Related work reports that such settings may capture heuristic shortcuts rather than causal mechanisms.
  • Contrast Construction: In-context learning can be driven by format and label space rather than the ground-truth mapping, motivating care in contrast construction.

3 Background

The standard steering formulation averages hidden-state differences between positive and negative prompts and injects the resulting vector during generation. Because real prompt pairs differ along many axes, contrast construction is central to isolating the intended concept.

  • Steering Formulation: Given positive prompt P and negative prompt N, the steering vector is computed from the difference between their mean hidden representations.The vector is used as an inference-time intervention in a decoder-only language model.
  • Steering Formulation: The intervention strength α is nonnegative, and the vector is added to the model’s hidden state during generation.
  • Contrast Construction: If positive and negative prompts differ only in the target concept, the extracted direction ideally isolates that concept.In practice, differences in length, syntax, lexical overlap, answer format, and task framing can introduce artifacts.
  • Contrast Construction: Contrast construction is therefore treated as a central problem in activation steering.

4 Single-source Steering

Single-source Hangman steering vectors are constructed from automatically generated and filtered contrastive clue lists, then evaluated for target steering, selectivity, and fluency. They reliably shift completions toward target words, but single-source vectors remain narrowly focused on the trained source word.

  • Contrastive prompt construction: AutoLexSteer retrieves related source words from WordNet and uses GPT-4o to generate shared and source/target-specific association clues for each target-source pair.The pipeline generates 500 paired lists per pair before filtering.
  • Contrastive prompt construction: Four sequential filters remove lexical leakage, invalid or unbalanced clues, poorly verified pairs, and pairs whose representations fail a model-separability test.Pairs with probe AUC below 0.70 are dropped; surviving pairs are deduplicated and reduced to approximately 130 per source-target pair.
  • Vector construction: The steering vector averages per-pair hidden-state differences at a selected layer and is added to the residual stream during inference.The layer and intervention strength are selected on a held-out validation split.
  • Single-source results: 1.28 same-concept S and 0.66 related-concept S show reliable target steering, while unrelated-concept S = 0.18 and ∆PPL = 0.08 indicate limited collateral impact.The reported average perplexity increase is no larger than 0.28 across the evaluated conditions.
  • Single-source results: Single-source vectors have a coverage limitation: a TREASURY-trained vector converts TREASURY-prone completions to BANK but can drift toward another neighbor for LENDER- or BRANCH-prone prompts.This motivates extending steering from one source word to sets of related source words.

5 Multi-Source Steering

The paper extends Hangman steering from one source word to source-word sets by averaging individual vectors into a super-vector. These super-vectors improve steering across related contexts and generalize to unseen related source words while preserving selectivity.

  • Construction: The super-vector averages individual steering vectors to steer from a set of source words toward one target.The experiment uses five source words per target and applies the same inference-time injection process.
  • Construction: The individual source-to-target vectors have similar but distinct directions, with cosine similarities around 0.3–0.4 for BANK, CLINIC, and GUILT.Their non-zero similarity supports averaging them into a shared direction without implying they are identical.
  • Results: 1.46 same-concept S and 0.79 related-concept S exceed the single-pair values of 1.28 and 0.66, while unrelated-concept S and ∆PPL remain similar.The comparison is averaged over 15 targets on Llama-3.1-8B.
  • Results: 0.59 unseen-related-concept S exceeds unrelated-concept S of 0.23, showing some generalization to held-out related source words.It remains below the related-concept score of 0.79.
  • Results: The super-vector successfully steers the held-out source word REMORSE toward GUILT across distinct continuations sharing a broad context.REMORSE was not used to construct the super-vector.

6 Beyond Lexis: Steering LLM Behavior

The paper applies multi-source Hangman steering to refusal and honesty behaviors using manually specified source sets and automatically generated association contrasts. On Llama-3.1-8B, the method strongly increases target behavior on related prompts while remaining comparatively selective on unrelated prompts.

  • Setup: The behavioral study targets comply→refusal and sycophantic→honest using multi-source vectors built from three manually specified source words per behavior.For honesty, the source set is {SYCOPHANTIC, HEDGE, SIDESTEP}; for refusal, it is {COMPLY, EXPLAIN, ROLEPLAY}.
  • Contrast construction: Approximately 165 filtered association-list pairs remain for each source-target pair after generating 500 candidates and applying the behavioral filtering procedure.The path-similarity sub-check is disabled for behavioral pairs because it removes legitimate clues such as truthful for HONEST.
  • Setup: The behavioral super-vectors are injected at layer 14 with α = 1.3 for refusal and layer 13 with α = 1.7 for honesty, selected on held-out prompts.These settings are slightly deeper than the lexical optimum.
  • Results: Unrelated-concept S reaches only 0.28 for refusal and 0.30 for honesty, while CAA reaches 1.36 and 1.24 on related prompts but is less selective.CAA’s unrelated-concept S is roughly twice Hangman’s and its ∆PPL is about 1.5× higher.

7 Conclusion

AutoLexSteer automates lexical steering-vector construction by retrieving related words, generating and filtering contrastive association lists, and constructing the resulting vector. Hangman vectors steer toward target words from related, multiple, and unseen source words, with preliminary behavioral-steering success.

  • AutoLexSteer automatically retrieves related words, generates and filters contrastive association lists, and constructs Hangman-style steering vectors.
  • Hangman vectors steer generation from related source words, multiple source words, and unseen related source words toward a desired target word.These findings suggest that the interrelated lexical concepts share a local subspace.
  • Behavior steering showed preliminary success beyond lexical concepts.

8 Limitations

The framework’s scope is constrained by its dependence on lexical resources and by evaluation on only 15 lexical words. Behavioral steering remains preliminary and requires manual source-word specification, although the automated lexical procedure is designed to scale.

  • AutoLexSteer works only for high-resource languages with the required lexical resources and sense definitions.
  • The experiments cover only 15 lexical words, although the authors report relatively small result variance and believe the findings are robust.
  • The lexical framework is designed to scale to more words because its procedure is automated.
  • Behavioral steering results are preliminary and not fully automated because source words must still be specified manually.

D Sentence-Completion Task: Prompt and Examples

The sentence-completion evaluation generates prefixes for each target and condition, then uses filtering and target-model separability checks to construct the evaluation set. Concrete targets generally separate more clearly than abstract targets, while removing F3 most harms steering quality.

  • For every target-condition tuple, GPT-4o generates 80 candidate prefixes and retains 50 that pass the baseline-frequency filter.
  • Each target uses three conditions with the same sentence-prefix form while evoking different concepts.Table 10 provides two examples per condition for three representative targets.
  • Concrete targets generally achieve higher linear-probe AUC than abstract targets across the evaluated models.Concrete targets include BANK, TEACHER, AIRPORT, CLINIC, and MACHINE; abstract targets include BELIEF, AMBIGUITY, MORAL, GUILT, and LUCK.
  • Removing F3 causes the largest drop in S and the largest rise in ∆PPL among the filter ablations.The ablation averages results over 15 targets on Llama-3.1-8B.

E LLM-judge Calibration

The LLM judge is calibrated against human annotations on a representative stratified subset using a shared three-level rubric. Its aggregate agreement is strong but lower on abstract targets, while steering performance is highest at mid-layers and an intermediate strength.

  • The calibration set contains 200 items sampled from the main evaluation prefixes, stratified across 15 targets and three prefix conditions.
  • Three blind graduate-student annotators independently assign scores from 0 to 2 using the same rubric given to the LLM judge.
  • The judge reaches precision 0.84, recall 0.79, and instance-level agreement 0.87 against majority-vote human labels.
  • Per-target agreement is lowest for abstract targets and highest for concrete targets, partly because abstract paraphrases produce human disagreement.
  • Steering success peaks at mid-layers and at α = 2.0 for both models, while larger strengths collapse generation fluency.The selected peak layers are l = 9 for Llama and l = 8 for Qwen.

G Per-target Results and Additional Cases

The appendix expands the qualitative evaluation of super-vectors and tests whether steering remains localized to the intended lexical neighborhood. It also documents the rubric used to judge related-word candidates and cross-format behavior.

  • Additional Cases: The appendix provides the full collection of super-vector cases for BANK, CLINIC, and GUILT, extending the representative examples in the main text.Table 12 includes all base-neighbor continuations.
  • Additional Cases: Off-target prompts show only surface paraphrases, indicating that super-vectors leave math, code, and natural-prose generation essentially unchanged outside the target lexical neighborhood.Reported differences include wording substitutions such as “and therefore admit” for “and admit.”
  • Evaluation Materials: The related-word rubric asks GPT-4o to rate candidates by semantic relatedness while excluding synonyms and derivative forms.The rubric evaluates candidate words relative to a target word.

H Qwen-2.5-7B Results

The appendix documents Qwen-2.5-7B evaluation materials alongside Hangman steering configurations, contrastive-list construction, judge prompts, and behavioral examples. These materials cover single-source and super-vector steering plus on-target and off-target behavior tests.

  • Qwen-2.5-7B Results: Tables 16 and 17 report Qwen-2.5-7B results for single-source steering and a five-source Hangman super-vector using Qwen-selected layer and strength settings.The super-vector uses the (l, α) selected for Qwen in §4.4.
  • Construction and Evaluation: The contrastive construction prompt creates matched positive and negative association lists for target and related words within the same semantic domain.Lists share common clues but include diagnostic clues distinguishing the intended words.
  • Construction and Evaluation: The construction rubric excludes the target and related words, morphological variants, and overly explicit definitions while requiring balanced lists with shared and diagnostic clues.It also specifies similar style, length, and difficulty for the two lists.
  • Behavioral Steering: The appendix includes additional on-target examples for refusal and sycophancy, plus off-target examples testing benign requests and reasonable opinions.These examples assess both intended behavioral steering and over-firing.
  • Construction and Evaluation: The cross-format judge focuses on semantic movement from a related-word concept to a target concept while preserving local fluency, not on isolated target-word mentions.The prompt compares base and steered continuations.
  • Configuration Sensitivity: Layer and strength sensitivity peaks at l = 9 for Llama-3.1-8B and l = 8 for Qwen-2.5-7B, with optimal α = 2.0 for both models.The super-vector dominates at every tested α in the reported comparison.
Loading 2609.06879v1…