PaperlayerView exact revision on arXiv

Source-linked AI summary

Toward universal steering and monitoring of AI models

Daniel Beaglehole, Adityanarayanan Radhakrishnan, Enric Boix-Adserà, Mikhail Belkin

arXiv:2502.03708v2cs.CLcs.AIstat.ML

TL;DR

The paper addresses the limited understanding of how large AI models represent general concepts and develops scalable linear concept representations using Recursive Feature Machines. Across language, vision-language, and reasoning models, these representations support steering and monitoring, with broad transferability and stronger monitoring performance than direct judge models. The authors also report that newer, larger models are more steerable and that steering can improve capabilities beyond prompting.

  • Problem

    Internal representations of model knowledge remain elusive, limiting understanding of concept structure and the development of model capabilities and safeguards.

    Source: Introduction · pp. 1–2

  • Method

    The paper uses Recursive Feature Machines to extract linear concept representations from activations for steering and monitoring across large language, vision-language, and reasoning models.

    Source: Introduction · pp. 2–3, Concept steering and monitoring… · p. 4

  • Results

    Across hundreds of concepts, steering transferred across languages and combined concepts, while concept-feature monitors outperformed direct judge models for hallucination and toxic-content detection.

    Source: Introduction · p. 3

  • Takeaways & Limitations

    Internal concept representations provide scalable tools for steering model behavior, improving capabilities, and monitoring misaligned content.

    Source: Introduction · p. 3

    In one steering example, prompting produced similar behavior, although the responses were less extreme.

    Source: Steering large models · p. 5

Abstract

from arXiv · show

Modern AI models contain much of human knowledge, yet understanding of their internal representation of this knowledge remains elusive. Characterizing the structure and properties of this representation will lead to improvements in model capabilities and development of effective safeguards. Building on recent advances in feature learning, we develop an effective, scalable approach for extracting linear representations of general concepts in large-scale AI models (language models, vision-language models, and reasoning models). We show how these representations enable model steering, through which we expose vulnerabilities, mitigate misaligned behaviors, and improve model capabilities. Additionally, we demonstrate that concept representations are remarkably transferable across human languages and combinable to enable multi-concept steering. Through quantitative analysis across hundreds of concepts, we find that newer, larger models are more steerable and steering can improve model capabilities beyond standard prompting. We show how concept representations are effective for monitoring misaligned content (hallucinations, toxic content). We demonstrate that predictive models built using concept representations are more accurate for monitoring misaligned content than using models that judge outputs directly. Together, our results illustrate the power of using internal representations to map the knowledge in AI models, advance AI safety, and improve model capabilities.

Introduction

Understanding internal concept representations remains an open problem, limiting both model capability improvement and safeguard development. This work introduces scalable feature extraction for concept steering and monitoring, demonstrating broad applicability and practical benefits across models and tasks.

Motivation

  • Internal concept representations remain difficult to characterize, while prompting can unpredictably change accuracy and safety-related behavior.Prior work includes unsupervised feature discovery and supervised probing, but their effectiveness for systematically extracting concepts for steering and monitoring remains unclear.Source: Introduction · pp. 1–2

Approach

  • The paper develops scalable linear concept representations from large-model activations using Recursive Feature Machines (RFMs).RFMs provide a feature-extraction approach for representing general concepts and support both steering and monitoring across models.Source: Introduction · pp. 2–3

Scope and capabilities

  • Concept steering and monitoring are demonstrated across LLMs, vision-language models, and reasoning models spanning 8 billion to 90 billion parameters.The approach exposed and mitigated vulnerabilities, enhanced coding and reasoning, transferred across languages, and combined multiple concept features for mixed-concept steering.Source: Introduction · p. 3

Scalability and capability improvement

  • Steering scaled to 512 concepts across three Llama models, with newer, larger, and better-performing models more steerable.A coding-language benchmark also showed that steering could improve performance over prompting in a single language.Source: Introduction · p. 3

Monitoring

  • Concept-feature predictive models monitored hallucinations and toxic content more effectively than models judging outputs directly across six benchmarks.Models built from smaller open-source LLM representations outperformed judge models based on GPT-4o and task-specific fine-tuned LLMs.Source: Introduction · p. 3

Practical contribution

  • The method integrates into standard inference pipelines without separate concept-specific fine-tuning and can identify and steer toward a concept in under one minute using fewer than 500 samples on one A100 GPU.The authors present this as a scalable, cost-efficient alternative for mapping model representations and steering behavior.Source: Introduction · p. 3

Results

RFM extracts concept vectors from model activations for steering and monitoring. Across models, steering exposes vulnerabilities, changes model stances or behavior, transfers across languages, and supports multi-concept control.

Concept steering and monitoring through Recursive Feature Machines.

  • RFM uses supervised feature learning with nonlinear predictors to extract per-block concept vectors from activations, which can then steer or monitor model behavior.The method uses kernel ridge regression and Average Gradient Outer Products, treating feature learning as a supervised analogue of PCA.Source: Concept steering and monitoring… · pp. 4–5
  • Concept vectors steer activations toward or away from concepts, while linear combinations enable simultaneous steering of multiple concepts.The control coefficient determines steering direction and strength; monitoring instead uses multiple extracted eigenvectors as predictive features.Source: Concept steering and monitoring… · p. 5

Steering large models.

  • RFM steering exposed vulnerabilities by making Llama provide thermite instructions and enabled Llama-Vision to adopt liberal or conservative stances after initially refusing.The thermite example identified iron(III) oxide, aluminum, and a 3:1 mixture ratio; stance steering suggests linear representations of political leanings.Source: Steering large models · p. 5, Figure 2 · p. 6
  • Steering deception vectors with a negative coefficient made DeepSeek responses more honest, complementing quantitative evaluation across 75 RolePlaying scenarios.Positive coefficients steer toward deceptive behavior, whereas negative coefficients steer away from it.Source: Steering large models · p. 5
  • English-trained concept vectors transferred to Chinese and French prompts, and combining concept vectors enabled multi-concept steering.The paper also reports steering across human languages and for non-binary concepts such as review ratings.Source: Steering large models · pp. 5–6
  • The quantitative evaluation automated concept generation, training-data creation, and evaluation across 512 concepts, five concept classes, four extraction algorithms, and three Llama models.The supplied passages describe the evaluation design but do not provide its full comparative outcome table.Source: Steering large models · p. 6

A Transferability of concept vectors across languages in Llama-3.3-70b-4-bit

The study evaluates scalable concept steering across hundreds of concepts, languages, coding tasks, and monitoring benchmarks. It finds that newer models and RFM-based representations support effective steering and that probes built from internal representations outperform direct LLM judges for detecting hallucinations and toxicity.

A Transferability of concept vectors across languages in Llama-3.3-70b-4-bit

  • The automated pipeline generated and evaluated concept vectors for 512 concepts across five concept classes using GPT-4o judgments over fixed steering coefficients.It compared PCA, difference of means, logistic regression, and RFM across three Llama models.Source: A Transferability of concept vectors… · pp. 7, 9
  • Newer and larger Llama models appeared more steerable across 512 concepts, while RFM provided a substantial overall advantage over PCA, difference of means, and logistic regression.Topophile steerability increased from 21.3% to 90.8% as model size and version increased, although gains were not consistent for every concept class.Source: Figure 4 · p. 8, A Transferability of concept vectors… · p. 9
  • Steering to C++ improved Llama-3.3-70b-4-bit coding performance over both original responses and direct C++ prompting across 50 HackerRank problems.The evaluation covered 25 Easy and 25 Medium problems and 732 total test cases.Source: A Transferability of concept vectors… · pp. 9–10, Figure 5 · p. 10
  • Steering toward chain-of-thought concept vectors improved performance on quantitative reasoning tasks such as counting letters in words.The study learned concept vectors for chain-of-thought reasoning and evaluated them on quantitative reasoning tasks.Source: A Transferability of concept vectors… · p. 10

Accurate monitoring via probing internal representations.

  • Probing activations was consistently better than direct LLM judges across hallucination and toxic-content benchmarks, with RFM probes also outperforming GPT-4o and ToxicChat-T5-Large overall.RFM was the only probing method to outperform GPT-4o on FAVABENCH.Source: Accurate monitoring via probing… · p. 11, Figure 6 · p. 11

Discussion

Directly perturbing internal activations with concept vectors can expose vulnerabilities, mitigate deception, and improve coding and reasoning, while revealing linear structure in model representations. The discussion argues that concept transferability and mixing support linear representation hypotheses, but the origins and extraction of these structures remain unresolved.

Discussion

  • Concept-vector perturbations exposed vulnerabilities, mitigated deception, and improved coding and reasoning, while activation-based monitoring outperformed models judging outputs directly.Source: Discussion · p. 12
  • Concept vectors can represent surprisingly complex operations, including personification and cross-language translation, through additive linear perturbations.Source: Discussion · p. 12
  • Transferability and concept mixing provide evidence for the linear representation hypothesis, which proposes that semantic relationships are encoded by linear transformations in network representations.Source: Discussion · p. 12
  • Classification can separate language-related prompts without recovering the steering direction, showing that classification and steering vectors may differ substantially.Source: Discussion · p. 12, Figure 7 · p. 12
  • The nature of linearly represented concepts and why classification extracts steering vectors remain identified as key unresolved questions.Source: Discussion · p. 12

A Hyperparameter selection for RFM concept vector extraction

RFM hyperparameters are selected using held-out validation, correlation with labels, and a fixed search over iterations, bandwidth, normalization, and regularization choices.

A Hyperparameter selection for RFM concept vector extraction

B Concept extraction and steering details

The appendix describes concept-vector extraction datasets and steering coefficients for safety, stance, language, programming, style, and multi-concept experiments. These setups include labeled prompts, held-out scenarios, translated statements, and linear combinations of concept vectors.

B Concept extraction and steering details

C Quantitative evaluation of steered DeepSeek models

The quantitative evaluation steers two distilled DeepSeek models across deception coefficients and judges their outputs as deceptive, honest, nonsensical, or overthinking. Steering successfully shifts behavior toward honesty at lower coefficients and deceptiveness at higher coefficients.

C Quantitative evaluation of steered DeepSeek models

D Prefixes for automated steering of 512 concepts

The steering prompts use concept-class-specific prefixes to instantiate each of 512 concepts. These prefixes ask the model to personify fears or locations, or adopt expert, mood, and persona attributes.

  • 512 concepts are prompted with prefixes tailored to five classes: fears, experts, moods, locations, and personas.The prefixes provide a uniform way to instantiate each concept before steering.Source: D Prefixes for automated steering of… · p. 19
  • Fear and location concepts use personification prompts centered on terror or love, respectively.Fear prompts say “Personify someone who is terrified of F,” while location prompts say “Personify someone who loves L.”Source: D Prefixes for automated steering of… · p. 19
  • Expert, mood, and persona concepts are elicited by asking the model to adopt the corresponding role or state.The prompts use “Take on the role of an expert E,” “Take on a M mood,” and “Take on the role of P.”Source: D Prefixes for automated steering of… · p. 19

E Questions for evaluation of 512 concepts

Evaluation questions are organized by concept class and designed to elicit responses indicative of each concept. Responses are constrained to 20 words or fewer to reduce generation cost.

  • Five questions were formulated for each concept class to elicit responses related to the target concept.The questions were selected from prompts produced by GPT-4o.Source: E Questions for evaluation of 512… · pp. 19–20
  • Fear questions probe phobias, secret fears, fear-themed villains, nightmares, and amusement-park rides.Examples include asking for the scariest thing in the world and what form a fear would take in sleep.Source: E Questions for evaluation of 512… · p. 19
  • Expert questions ask about subjects, difficult problems, resource allocation, work, and what the model would teach others.These prompts are intended to elicit professional interests and expertise-related responses.Source: E Questions for evaluation of 512… · pp. 19–20
  • Mood questions ask about feelings, current vibe, daily experience, song titles, and desired destinations.The prompts target direct descriptions of the model’s current emotional state or mood-associated preferences.Source: E Questions for evaluation of 512… · p. 19
  • Location and persona questions elicit favorite places, desired skills, imagined shrines, life settings, work, interests, power, and teaching.These questions are designed to reveal preferences associated with the relevant location or persona concept.Source: E Questions for evaluation of 512… · p. 20
  • Each evaluation question ends with “Answer in 20 words or less” to reduce response generation time and length.The questions were selected specifically to elicit responses related to the concept of interest.Source: E Questions for evaluation of 512… · p. 20

F Steering coefficients for 512 concepts

Steering is evaluated across six control coefficients per model. The coefficient ranges differ by model, and larger coefficients can produce nonsensical, repetitive outputs.

G Evaluation prompts for 512 concepts

Steering success is judged by GPT-4o using structured prompts that compare each response with the intended concept and the question. The evaluator requires relevance, specificity, and an answer to the question.

  • For each of the 25 questions in SI Appendix E, the study used detailed evaluation prompts with example evaluations, including an example for question 1.Source: G Evaluation prompts for 512 concepts · p. 20
  • Each evaluation receives the intended phobia and the steered response to a specified question, then assesses personality consistency.The evaluator is explicitly asked to determine whether the response matches someone with the given phobia.Source: G Evaluation prompts for 512 concepts · p. 21
  • A successful response must mention themes relevant to the phobia, avoid vague or generic content, and answer the question asked.The evaluator outputs score 1 for successful steering and 0 otherwise, with a concise explanation.Source: G Evaluation prompts for 512 concepts · p. 21
  • Examples score responses as failures when they are unrelated or nonsensical, and as successes when they directly connect the answer to the intended phobia.Responses about terrifying smiles or children laughing receive score 1, while unrelated darkness receives score 0.Source: G Evaluation prompts for 512 concepts · p. 21

H Description of baseline methods for steering

The section describes three baseline approaches for extracting concept vectors from labeled activation data: PCA, difference in means, and logistic regression.

  • PCA pairs examples from opposite classes, places their differences into a matrix, and uses its oriented top eigenvector as the concept vector.The data are represented as labeled activation vectors, with S0 and S1 denoting the two classes.Source: H Description of baseline methods for… · p. 21
  • Difference in means computes the concept vector from the mean activations of the two labeled classes before Pearson-correlation orientation.Source: H Description of baseline methods for… · p. 21
  • Logistic regression uses a grid search over C ∈ {1000, 100, 1, 0.1}, the liblinear solver, and 10^-3 tolerance, then normalizes its coefficients to unit length.The normalized coefficients define the concept vector before orientation.Source: H Description of baseline methods for… · p. 21

I Details regarding evaluation of steered code

The evaluation of steered code responses uses 50 HackerRank algorithm challenges spanning easy and medium difficulty.

J Improving reasoning by steering toward chain-of-thought

The section describes how chain-of-thought concept vectors are extracted and evaluated, then specifies prompt-based monitoring tasks for toxicity, hallucinations, factuality, and query categories.

J Improving reasoning by steering toward chain-of-thought

  • Chain-of-thought vectors are extracted from 1,000 label −1 and 1,000 label 1 GSM8K questions appended with a step-by-step reasoning suffix.Source: J Improving reasoning by steering… · p. 23
  • On 887 letter-counting questions, excessive chain-of-thought steering can degrade performance by causing models to overthink instead of answering.The evaluation uses steered Llama-3.1-8B models and reports performance in Table S6.Source: J Improving reasoning by steering… · p. 23

K Prompts for monitoring

  • Queries that induce hallucinations are assigned one of six categories, including erroneous queries, inappropriate content, complex reasoning, out-of-scope information, beyond-modality interaction, and other types.The prompt requires returning only a category number from 1 to 6.Source: K Prompts for monitoring · p. 24
  • Monitoring prompts classify toxicity, hallucinations, response factuality, contextual correctness, and hallucination-inducing query categories.Hallucination categories include contradictory entity, relation, and sentence errors, invented entities, subjective errors, and unverifiable facts.Source: K Prompts for monitoring · p. 24

L Monitoring using concept vectors

This section presents the procedures, materials, and evaluation configurations for monitoring with concept vectors, including probe and judge-model comparisons for hallucination and toxicity detection. It also includes illustrative examples of concept-based steering and monitoring-related outputs.

Hyper-parameter tuning for probing models

  • Probes used held-out validation data to tune method-specific hyperparameters, including regularization, bandwidth, mean-centering, and iteration counts.RFM used λ = 10−3 with bandwidths L ∈ {1, 10, 100}; iteration counts varied by benchmark, while linear and logistic regression searched predefined regularization grids.Source: L Monitoring using concept vectors · p. 25

Aggregating features for probing models

  • Aggregated probes projected activations onto concept vectors and tuned AUROC directly across blocks, or through an OLS fit to RFM projections.The same validation sets were used for aggregation and per-block models, providing an extension of AUROC from a single vector to multiple concept vectors.Source: L Monitoring using concept vectors · p. 25

Validation and test splits

  • Evaluation used cross-validation or predefined splits to generate non-overlapping test predictions for all judge models and supervised probes.ToxicChat used 10-fold training cross-validation with a fixed test set; other datasets used five-fold procedures, with 70/30 or 80/20 training-validation splits where needed.Source: L Monitoring using concept vectors · p. 25

Generating AUROCs for LLMs

  • LLM AUROCs were computed from token probabilities, using normalized Yes/No probabilities for binary tasks and normalized 1–6 probabilities for multiclass HaluEval-Wild.GPT-4o probabilities came from the top 20 API log-probabilities, while open-source models used full logits followed by softmax.Source: L Monitoring using concept vectors · p. 25

Prompt materials

  • Table S3 lists 200 statements for label 0 and 200 statements for label 1 used as prompts in Fig. 4.The statements span philosophical, scientific, social, and everyday claims.Source: Table S3 · p. 46

Steering evaluation

  • Table S4 provides 25 additional samples comparing RFM-based steering across model sizes, with GPT-4o judgments marking successful and unsuccessful steering.The samples contain five examples per concept class.Source: Table S4 · p. 47

Illustrative outputs

  • The examples pair concepts such as forensic scientist, mathematical logician, and proteomics researcher with responses illustrating concept-conditioned outputs across Llama model sizes.Other examples condition responses on moods such as hostile or nostalgic and on locations such as Istanbul.Source: L Monitoring using concept vectors · pp. 48–49

Qualitative and supplementary comparisons

  • The examples show concept-conditioned responses spanning locations, expertise, personas, moods, skills, and values, including topophile, expert, and persona prompts.These examples include concepts such as peace, proof theory, structural biology, synthetic biology, city identity, and women’s rights, but they do not establish monitoring accuracy or method superiority.Source: L Monitoring using concept vectors · pp. 49, 52, 53
  • Table S5 compares RFM, Logistic, Diff. Means, PCA, and prompting across 25 examples on Llama-3.3-70b-4-bit at ϵ = 0.4.The table uses blue and red text to mark responses judged successfully or unsuccessfully steered by GPT-4o, but the supplied passage does not provide the cell-level outcomes.Source: Table S5 · p. 51, L Monitoring using concept vectors · p. 52

Dataset statistics

  • Table S7 summarizes the statistics of the datasets used for monitoring.Source: Table S7 · p. 56

Monitoring comparison

  • Table S8 compares AUROC scores for probing and judge models across five hallucination datasets and ToxicChat for toxicity monitoring.The hallucination datasets are FAVABENCH [19], HaluEval [13], HaluEval-Wild [37], PubMedQA, and RAGTruth from HaluBench [27].Source: Table S8 · p. 57
  • The comparison reports RFM, linear regression, and logistic regression using either block aggregation or the best single block selected on a validation set.The LLM providing activations is indicated in parentheses, and logistic-regression entries can be unavailable because of convergence or runtime limits.Source: Table S8 · p. 57
Loading 2502.03708v2…