Taxonomy, Opportunities, and Challenges of Representation Engineering for Large Language Models

Introduction

• RepE can reveal how human-understandable concepts appear in activations and steer behavior without training, potentially improving control, data efficiency, flexibility, and performance preservation.The survey describes it as a way to influence models’ behavior, personality, encoded beliefs, and performance toward safe and desired outcomes.Source: §1 · p. 1
• Initial Activation Steering assumed concepts were linear activation-space directions; later methods explored output-driven identification (Cao et al., 2024a), unsupervised features, non-linear representations (Qiu et al., 2024), alternative operators (Postmus & Abreu, 2024), steering functions, and adapters (Wu et al., 2024c).Early work compared activations for positive and negative concept inputs and added a concept vector to modulate its intensity (Li et al., 2023a).Source: §1 · pp. 1–2
• The survey unifies RepE as a pipeline for identifying, operationalizing, and controlling representations, and contrasts methods at each step.This framework addresses the lack of an overview and conceptual clarity in the literature.Source: §1 · p. 2
• It reviews RepE evaluation methodologies and benchmarks, proposes evaluation best practices, and surveys applications in AI Safety, Ethics, and Interpretability, including which concepts are more amenable to control.Source: §1 · p. 2
• The survey compares RepE with fine-tuning, prompting, and decoding-based methods, suggests why RepE works, and catalogs its advantages and challenges.Source: §1 · p. 2
• It reviews >130 papers and includes weight-modifying, output-optimized, and non-vector methods that control intermediate representations, while excluding soft-prompting, decoding-based methods, and Activation Patching.The survey notes that over 100 papers appeared in the preceding year without a systematization of the literature.Source: §1 · pp. 2–3
• Future work should improve benchmarking and explore non-linear and multi-concept representations, layer interactions, temporal trajectories, and new applications, while strengthening identification and control.Practical use also requires better multi-concept steering, long-form generation, reliability, out-of-distribution robustness, and preservation of general capabilities.Source: §1 · pp. 2–3

2 What is Representation Engineering?

• RepE manipulates model representations to control behavior related to a concept, with goals of behavior steering and understanding internal computations.Potential uses include guarding against jailbreaks, adapting style or personality, improving reasoning, and investigating how concepts influence outputs.Source: §2 · p. 5
• RepE identifies a concept operator and uses a steering function to manipulate activations or weights, assuming concepts are represented within the model.Concepts can include behaviors, tasks, entities, knowledge, or personality traits; the operator’s intended relation to a concept representation entails assumptions about its geometry and location.Source: §2 · pp. 5–6

2.1 Strategies for RepE

• Common identification strategies include linear activation addition, sparse autoencoders, and tuning a concept operator toward desired outputs; control can alter activations or weights.Linear directions may be found by contrasting activations with and without a concept, while sparse autoencoders project features into a higher-dimensional sparse space.Source: §2.1 · p. 6
• Truthfulness steering contrasts activations from prompts to be honest or lie, while preference steering optimizes an activation vector using desired and undesired outputs (Zou et al., 2023a; Cao et al., 2024a).The truthfulness direction can guide LoRA training toward more truthful answers; the preference vector can be added during inference to favor desired outputs.Source: §2.1 · p. 6

2.2 Advantages of Representation Engineering

• RepE can work with few training examples, may not require labeled data, and allows steering to be switched off or adjusted per request for personalization and context-dependent control.The cited flexibility examples include Cao et al. (2024a), Stickland et al. (2024), Guo et al. (2024), Chen et al. (2024a), and Lucchetti & Guha (2024).Source: §2.2 · p. 6
• RepE enables precise single-concept steering with negligible added inference cost, no extra inference steps, and typically little degradation in model capabilities.The capability-preservation finding is attributed to Panickssery et al. (2024), Stickland et al. (2024), Guo et al. (2024), Qiu et al. (2024), and van der Weij et al. (2024).Source: §2.2 · p. 7
• Steering can provide causal evidence that an identified representation produces an expected output change, while user-accessible controls can give non-experts tacit understanding of model internals.Causal verification is associated with Marks & Tegmark (2023) and Arora et al. (2024); tacit understanding with Chen et al. (2024a).Source: §2.2 · p. 7
• A meta-survey indicates RepE tends to control behavior more effectively than prompting, fine-tuning, or decoding-based methods, and can be combined with other methods for defense in depth.Combining methods can strengthen control by layering multiple imperfect safeguards.Source: §2.2 · p. 7

3.1 Representation Identification

• Identification elicits concept-related stimuli, collects activations, and derives a concept operator through input reading, output optimization, or unsupervised feature learning.Stimuli may have positive, negative, neutral, or unknown relationships to the concept, and must activate its representation in the model.Source: §3.1 · pp. 7, 9

3.2 Operationalizing Representations

• Operationalizing representations requires assumptions about their structure, such as linear directions or nonlinear manifolds, and a concept operator linking identification to control.The operator’s intended relationship to the model’s actual concept representation can vary.Source: §3.2 · p. 9

3.3 Representation Control

• Representation Control applies a steering function and concept operator to modify activations or weights, aiming for outputs more aligned with the target concept.Activation steering is repeated for new inputs, whereas weight steering is applied once; the distinction can be blurry.Source: §3.3 · p. 10

3 Framework and Notation

• RepE is a two-step pipeline: Representation Identification finds a concept operator, and Representation Control uses it to modify model activations or weights.The framework also classifies pipelines by their identification and control methods and the assumptions used to operationalize representations.Source: §3 · p. 7, Figure 2 · p. 8, §3.3 · p. 10

4.1.1 Constructing Inputs

• Input Reading constructs concept-related input groups and derives an operator from their activations; contrastive pairs can isolate the concept better but may be harder to obtain.Inputs may come from existing datasets, manual creation, or synthetic generation; synthetic inputs can target concepts without datasets but may introduce confounders. Groups can reflect positive, negative, or neutral examples and other concept relationships.Source: §4.1.1 · pp. 11–12

4.1.2 Collecting Activations

• Activation collection varies by layer, model component, and token position; researchers may test these choices for operator effectiveness or focus computation on likely concept-bearing layers.Activations can come from residual streams, attention heads, or MLP and attention outputs; token choices include significant individual tokens or averages across all tokens.Source: §4.1.2 · p. 12

4.1.3 Calculating the Concept Operator

• Input Reading derives operators through difference-in-means, probes, dimensionality reduction, or selecting relevant neurons and attention heads.Larger datasets can improve difference-in-means operators, while probes may capture direction without magnitude and PCA’s highest-variance direction may miss the positive-negative shift; neuron and head selection aims to reduce side effects.Source: §4.1.3 · pp. 12–13

4.2.2 Output Scoring

• Output Optimization collects activations from inputs intended to elicit relevant outputs, scores generated text against the concept, and turns scores into a loss.Inputs may be queries or text for transformation; outputs can be single tokens, sentences, or long-form generations. Scoring can use ground-truth likelihoods, desired-versus-undesired likelihood differences, or an LLM judge; losses include cross-entropy, preference-optimization objectives, and regularized variants.Source: §4.2.1 · pp. 13–14, §4.2.2 · p. 14

4.2.3 Optimizing Activations

• The concept operator is optimized against the output-scoring loss while model activations and weights remain frozen; shared operators can generalize better than input-specific ones.Gradient descent with AdamW is common, while another approach samples outputs and backpropagates toward a vector favoring high-scoring outputs. Averaging per-input operators is one way to form a generalized operator.Source: §4.2.3 · p. 15

4.3.2 Discovering Features

• Unsupervised Feature Learning discovers many features before matching them to concepts: SAEs learn sparse, disentangled directions, while DCT seeks meaningful activation perturbations with predictable downstream effects.SAEs reconstruct activations using a small subset of features; DCT optimizes vectors whose downstream effects are large and predictable, and vectors can serve directly as concept operators.Source: §4.3 · p. 15, §4.3.2 · p. 16

4.3.3 Identifying Concepts

• SAEs generally need much larger activation datasets than DCT, and learned features are matched to concepts by inspecting activations, testing effects, probing, or using labels.Feature labels may be judged manually, by LLMs, or through pre-labeled datasets; contexts where a feature activates do not always predict its effect.Source: §4.3.1 · p. 16, §4.3.3 · pp. 16–17

4.3.4 Applying SAEs

• SAE features can serve as operators or inform steering vectors, but a target concept is not guaranteed to be found and feature steering should be evaluated on model outputs.Applying an SAE operator requires translating activations into and out of SAE space; steering vectors may fall outside the SAE training distribution, complicating their decomposition and reconstruction.Source: §4.3.4 · p. 17

5 Representations Operationalization

• RepE assumes concepts are represented in activation space, but methods must choose an operator shape and relationship without a clear account of representation structure.These choices bake assumptions about the geometry of internal representations into identification and control.Source: §5 · p. 17

5.1 Assumed Geometry

• Geometry assumptions shape identification and steering: a linear probe can miss nonlinear concept features, and linear-direction steering can fail for non-Euclidean representations.The choice of representation geometry is embedded in both the identification method and steering function.Source: §5.1 · p. 17
• The Linear Representation Hypothesis treats concepts as activation-space directions whose magnitude indicates intensity, but alternatives include fuzzy clusters (Postmus & Abreu, 2024) and non-linear manifolds (Qiu et al., 2024).Other proposed geometries include circles, onions (Csordás et al., 2024), non-linear paths, and vector-space subsets.Source: §5.1 · p. 18

5.2 Concept Operators

• Concept operators link representation identification to control, most often as vectors but also as matrices or matrix–vector combinations.Vectors commonly encode linear directions; matrices can capture correlations or other information about a concept, while matrix–vector forms can support affine steering.Source: §5.2 · pp. 18–19

6.1 Modifying Activations

• Activations can be steered during inference by modifying an intermediate layer’s activations, which then shape later computations and outputs.In the paper’s example, steering activations toward truthful and away from dishonest patterns is intended to make subsequent outputs more honest.Source: §6.1 · p. 19

6.1.1 Activation Steering Functions

• Linear addition increases a concept direction’s magnitude, while combining steering vectors can cause concept interference and reduce model quality.Methods also renormalize after addition, target individual attention heads, or add noise; alternatives include removing a direction by projection, scaling aligned components, and soft projection that preserves non-concept directions.Source: §6.1.1 · pp. 19–20
• Other activation functions rotate activations while retaining their magnitude, use affine transformations, or project activations before editing.Rotation uses a reflection and learned MLP angle adjustment; affine steering can match representation statistics or manipulate facts, transfer vectors across models, and create counterfactuals. Projection into and out of an SAE space incurs reconstruction-related capability loss and higher computational cost.Source: §6.1.1 · pp. 20–21

6.1.2 Location & Time

• Steering can target one or many layers, components, or token positions; input-dependent decisions can adjust whether, where, or how strongly to intervene.Dynamic steering is motivated by differing input needs and may improve effectiveness while reducing side effects; methods use similarity, probes, divergence, activation norms, or output evaluation to choose interventions or strengths.Source: §6.1.2 · p. 21

6.2 Modifying Weights

• Weight steering changes model weights so they produce activation patterns and outputs aligned with a target concept.Weights can be modified through fine-tuning or direct editing.Source: §6.2 · p. 22, §6.2.1 · p. 22, §6.2.2 · p. 23

6.2.1 Training Weights

• Weight training can target desired activations, desired outputs, or both, and output-oriented methods lie on a spectrum between representation control and parameter-efficient fine-tuning.Some methods train on activations produced by activation steering or noise-like activations; others optimize outputs, or jointly target concept-aligned activations and output performance.Source: §6.2.1 · p. 22

6.2.2 Editing Weights

• Direct weight editing can remove a concept direction from representations or shift hidden-state activations away from undesirable regions and toward desirable ones.The section’s takeaway is that representation control operates through activation steering or weight steering, with weights controlled by training or editing.Source: §6.2.2 · p. 23

7.1 Prototypical RepE Pipelines

• Table 8 classifies RepE pipelines by identification strategy, operator shape, activation versus weight control, data and optimization needs, linearity, input dependence, retraining, and intended concepts.The selected pipelines were chosen to represent diverse approaches, with preference for highly cited papers published at top conferences.Source: §7.1 · p. 23, Table 8 · p. 24
• CAA, LoRRA, ITI, SAEs, LoReFT, and BiPO illustrate distinct ways to identify or learn concept operators and control model representations.CAA uses a mean activation difference; LoRRA uses a principal component of contrasting activations and trains a LoRA adapter; ITI uses probe-selected attention heads; SAEs learn sparse features unsupervised; LoReFT trains a low-rank activation adapter; and BiPO optimizes a preference vector. These pipelines were proposed for behavioral steering, honesty and ethical values, truthfulness, broad feature discovery, downstream tasks, and human-preference alignment, respectively.Source: §7.1 · pp. 23–24

7.2 Which Methods Work Better?

• Comparisons of RepE methods often cannot establish superiority because unbiased evaluations are lacking.The section summarizes previously reported empirical comparisons rather than presenting a definitive head-to-head evaluation.Source: §7.2 · p. 25

7.2.1 What Works for Representation Identification?

• Which identification approach works best depends on the concept: unbiased comparisons favor input reading for sentiment and output optimization for concrete topics.SAE steering results are mixed: some studies find it stronger than input-reading methods, another finds the reverse, and other findings suggest SAEs can improve input-reading operators. On AxBench, SAEs trail LoReFT and class means in concept detection and steering but outperform probing and dimensionality-reduction methods.Source: §7.2.1 · p. 25
• Evidence on identification prompts, readout positions, and activation-difference functions is mixed, though Difference-in-Means performed strongest in 6 of 7 studies.Choice prompts can outperform free-form answers; evidence on token readout varies. Difference-in-Means usually leads, while probes are next best and PCA or CCS tend to do worse; DAS performed strongest in its single test, and mean-centering can help steering.Source: §7.2.1 · pp. 25–26

7.2.2 What Works for Representation Operationalization?

• Matrices can improve steering over vectors but may raise cost or reduce output quality, while direct comparisons favor non-linear over linear operators.The matrix-versus-vector evidence is mixed in Zou et al. (2023a), where a matrix slightly outperforms a vector but loses to an input-specific vector; non-linear SEA outperforms linear SEA, and non-linear ITI improves effectiveness at a higher capability cost.Source: §7.2.2 · p. 26

7.2.3 What works for Representation Control?

• Whether weight or activation control is more effective remains unclear, and activation-steering functions have inconsistent rankings across studies.Wang et al. (2024a) find weight control more effective but with a larger output-quality reduction. Piece-wise operations or projection outperform linear addition in some comparisons, but other findings favor linear addition over orthogonal projections at a capability cost; more expressive functions offer higher performance ceilings but need more data.Source: §7.2.3 · p. 26
• Applying steering at every inference step is generally more effective, while the best component and layer coverage remain uncertain.Continuous steering can counter diminishing effects during generation, but first-token-only steering can remain effective with less output-quality impact. Steering both attention and MLP blocks beats steering either alone in one evaluation, while steering at the LM head or embedding layer is ineffective; all-layer control is assumed to improve steering at a capability cost but lacks experimental confirmation, and many concepts steer best in middle layers.Source: §7.2.3 · pp. 26–27

8.1.1 Evaluation Task and Dataset

• Evaluations compare model outputs before and after intervention on concept-related tasks, measuring concept alignment and general-quality degradation using established or purpose-built datasets.TruthfulQA, AdvBench, and BOLD are common datasets; multiple-choice tests are convenient but less faithful to downstream use than longer generations.Source: §8.1 · p. 27, §8.1.1 · p. 27

8.1.2 Measuring Steerability

• Steerability measures vary by free generation versus predefined answers, label source, and metric, including judge scores, ground truth, word counts, accuracy, likelihood, logit differences, and causal efficacy.Labels may come from LLM judges, dataset labels, or concept-related word occurrences; human evaluation is rare, and the cited metrics include multiple-choice accuracy, top-k probability, and log-odds.Source: §8.1.2 · pp. 27–28

8.1.3 Measuring Changes in Model Quality

• Model-quality evaluations use capability benchmarks, general-dataset next-token measures, and targeted checks such as generation diversity, fluency, and instruction following; some combine quality with steering effect.Combined metrics include perplexity-normalized effect size and the harmonic mean of concept alignment, instruction following, and fluency.Source: §8.1.3 · pp. 28–29

8.1.4 Other Experimental Details

• RepE studies vary steering strength, intervention layers and token positions, identification sample counts, and model family or size, and compare against the unsteered model and other controls.Steering-strength sweeps reveal the tradeoff between concept alignment and model quality; common additional baselines include earlier RepE, prompting, and fine-tuning methods.Source: §8.1.4 · p. 29

8.2 Best Practices for Evaluating RepE Methods

• Best practices favor open-ended and difficult realistic tests, specialized validated judges when possible, steering-strength sweeps, capability reporting, generalization tests, and diverse concepts and model sizes.Open-ended generations better resemble downstream settings than multiple-choice tests, and varied capability tests can assess both fluency and question-answering accuracy.Source: §8.2 · pp. 29–30
• Authors should compare against current state-of-the-art baselines, report null results to counter survivorship bias, and show result distributions rather than average steerability alone.Neutral evaluations can reveal weaknesses absent from original method papers, while variance reporting helps readers assess steering reliability.Source: §8.2 · p. 30

8.3 RepE benchmarks

• On AxBench, prompting outperformed all nine tested RepE methods at steering concepts, while Difference-in-Means, Logistic Regression, and ReFT-r1 led concept detection.AxBench uses synthetic positive and negative examples for 500 concepts and scores generated responses for concept presence, instruction following, and fluency; its concepts test narrow topics.Source: §8.3 · p. 30
• Im & Li (2025) found Difference-in-Means best for steering-vector identification, while the survey calls for a unified, fair benchmark spanning practical concepts, models, dataset sizes, and steering strengths.The benchmark should compare the full RepE pipeline and expose current weaknesses; fair comparisons must account for methods’ different data, compute, and concept requirements.Source: §8.3 · pp. 30–31

9.1 Types of Concepts

• RepE has been applied across six concept categories, with high-level behavioral characteristics such as harmfulness and honesty the most popular.These abstract, potentially context-dependent concepts concern the behavior exemplified by the model.Source: §9.1 · p. 31
• RepE can induce tasks such as reasoning, classification, and associations by triggering an input-output function, for example with a function vector.Source: §9.1 · p. 31
• RepE controls language and style, including sentiment, genre, and language, as well as encoded world knowledge and beliefs, including factual knowledge and situational beliefs.Source: §9.1 · p. 31
• RepE adjusts output content such as topics, and has been used to alter encoded values, goals, and ethical principles, including human preferences (Cao et al., 2024a) and ethical theories (Tlaie, 2024).Linguistic features such as verb conjugations and noun gender are also controlled, for interpretability (Hao & Linzen, 2023) or evaluation (Arora et al., 2024).Source: §9.1 · pp. 31–32

9.2 Commonly Controlled Concepts

• Truthfulness, Harmfulness, and Toxicity are the most common controlled concepts, and their reported high success rate suggests controllability but may be inflated by publication bias.The section also identifies Truthfulness, Harmfulness, and Fairness as popular high-level behavioral concepts.Source: §9.2 · p. 32

10.1 AI Safety

• RepE methods prevent harmful outputs more effectively than prompting or fine-tuning safeguards, and researchers operationalize harm with datasets including AdvBench, HarmfulQA, and HarmEval.RepE has also been used to reduce toxicity, though strengthening toxicity representations can instead make models more toxic and enable attacks (Wang & Shu, 2023).Source: §10.1 · p. 33
• RepE steers truthfulness, often outperforming fine-tuning but sometimes prompting, and has been used to mitigate or induce hallucinations and control refusal behavior.TruthfulQA is the most commonly used truthfulness dataset (Lin et al., 2022); hallucinations can be more effectively identified before answering and controlled in attention-head activations. Refusal-direction interventions can strengthen safety training or prevent refusals to benign queries (Yu et al., 2024; Cao et al., 2024b).Source: §10.1 · p. 33
• RepE has been used to prevent private-data leakage, although securing specific data can increase leakage of other information; tested techniques can even outperform Differential Privacy methods.Researchers have also used RepE to control sycophancy and model motivations, while attempts to uncover and steer deception have been unsuccessful (Clymer et al., 2024).Source: §10.1 · p. 34

10.2 Ethics

• RepE identifies and controls bias-sensitive attributes, sentiment toward social groups, and counterfactual outputs that can expose differences in how models respond to users.It has also been used to align models with preferences using DPO-based steering or synthetic preference data, and to steer general or theory-specific ethical reasoning (Cao et al., 2024a; Liu et al., 2023; Adila et al., 2024a; Pham & Nguyen, 2024; Tlaie, 2024; Zou et al., 2023a; Xu et al., 2024a).Source: §10.2 · pp. 34–35

10.3 Knowledge Editing

• RepE can edit factual associations, detect and alter entity attributes, and control whether answers draw on parametric or prompt-provided knowledge; unlearning results remain mixed.RMU pushes harmful representations toward a random vector while retaining harmless ones, but unlearning methods remain vulnerable to harmful steering vectors and SAE steering is not yet competitive (Li et al., 2024b; Xu et al., 2024b; Farrell et al., 2024). RepE also controls beliefs about the user, current year, and other agents’ beliefs (Chen et al., 2024a).Source: §10.3 · p. 35

10.4 Task Execution

• Comparisons find that in-context vectors steer behavioral tasks better, while function vectors control functional tasks more effectively, generalize better, and reduce model-quality deterioration.Function vectors are identified via attention heads with causal task effects, whereas in-context vectors come from contrastive few-shot activations; additional work improves these vectors and identifies representations that trigger unprompted Chain-of-Thought reasoning (Liu et al., 2024b; Zhang & Viteri, 2024).Source: §10.4 · pp. 35–36

10.5 Controlled Text Generation

• RepE controls sentiment, emotions, personality, style, genre, and topic; moderating personality traits has also improved reasoning and conversation while reducing sycophancy.Work also steers honesty and creativity, and uses language-pair inputs to improve cross-lingual retrieval coherence; English safety steering vectors can transfer to other languages (Weng et al., 2024).Source: §10.5 · p. 36

10.6 Performance

• RepE has been used to elicit unprompted Chain-of-Thought and improve social reasoning, but attempts to steer the faithfulness of Chain-of-Thought to model decisions were unsuccessful.Other reported applications include improvements to language understanding and generation, code, type prediction, answer consistency, order bias, multilingual retrieval, and agent exploration (Tanneru et al., 2024; Wu et al., 2024c; van der Weij et al., 2024; Lucchetti & Guha, 2024; Rahn et al., 2024).Source: §10.6 · p. 36

11.1 Related Methods

• Prompting changes text without altering model internals, soft prompts modify input embeddings, and fine-tuning changes weights without targeted, isolated concept control.RepE avoids prompting’s context-window token cost, can provide interpretability benefits, and often enables continuous control; fine-tuning overlaps with RepE methods that optimize outputs and apply weight steering.Source: §11.1 · pp. 38–39
• Decoding shifts token probabilities during inference; MI studies bottom-up circuits while RepE takes a top-down view, and activation patching replaces activations rather than modifying them.Probes can identify concepts, detect task drift or deceptive reasoning, and support RepE identification or steering modulation (Abdelnabi et al., 2024a; MacDiarmid et al., 2024; Goldowsky-Dill et al., 2025).Source: §11.1 · p. 39

11.2 Meta-study comparing to Other Methods For Behavior Control

• The meta-survey compared per-paper control effectiveness, capability retention, and sample counts where available, averaging multi-task results and choosing each category’s best comparator.The authors caution that most available comparisons come from RepE papers, potentially biasing results in RepE’s favor.Source: §11.2 · p. 39

11.2.1 Prompting

• Across 24 papers, RepE steers target concepts better than prompting in 75% of cases, though AxBench (Wu et al., 2025) found prompting better for concrete content concepts.RepE also often had less impact on general capabilities than prompting.Source: Table 13 · p. 40, §11.2.1 · p. 40

11.2.2 Fine-tuning

• RepE controls concepts more effectively in 72% of comparisons versus fine-tuning’s 20%, and fine-tuning more often harms general capabilities.RepE outperformed fine-tuning in every comparison under 500 samples, suggesting it may be more sample-effective.Source: Table 13 · p. 40, §11.2.2 · p. 40, Figure 3 · p. 40

11.2.3 Decoding-based Methods

• Across 7 papers, RepE outperformed decoding-based methods in 71% of cases and had less impact on general capabilities.Source: §11.2.3 · p. 41

11.2.4 Combining Methods

• All four evaluated papers found combining RepE with prompting, fine-tuning, or decoding controlled behavior better than either approach alone (Li et al., 2023a; Wang et al., 2024c).This was typically tested when RepE alone did not outperform the respective baseline; the meta-survey’s overall takeaway is that RepE offers more effective control with less capability loss.Source: §11.2.4 · p. 41

• LLMs already represent concepts such as refusal and honesty, allowing RepE to identify and promote existing representations rather than learn them anew.The section’s takeaway is that these concepts are structured and possibly linear, making latent-variable manipulation a route to behavioral control.Source: §12 · pp. 41–42
• The Linear Representation Hypothesis (LRH) proposes that concepts can be linear activation-space directions; supporting evidence coexists with nonlinear features.Linear probes and RepE successes provide further evidence for linear representations, while circular representations of weekdays illustrate a nonlinear alternative (Li et al., 2024e; Abdou et al., 2021).Source: §12 · p. 41
• Models can remain functional after activation manipulation, helping RepE preserve general capabilities despite internal edits.Small random activation perturbations do not strongly affect performance; resilience may relate to dropout during training or operators derived from the network’s own activations, which may remain within its usual activation distribution.Source: §12 · pp. 41–42
• Because activations function as intermediate memory variables, changing concept representations can alter subsequent computation and ultimately the model’s output.RepE therefore resembles editing semantically meaningful variables in a neural network’s intermediate memory.Source: §12 · pp. 41–42

13.1 Empirical Weaknesses

• Simultaneously steering concepts often weakens each effect and harms capabilities, though separate-layer, adaptive, or sparse orthogonal interventions aim to reduce interference.The section identifies multi-concept control as a major challenge because edits can interfere; some proposed approaches seek to mitigate the problem.Source: §13.1 · p. 42
• RepE’s control of short answers lacks evidence for long-form and multi-turn use, and can weaken across contexts, prompt templates, tasks, or languages.Operators identified from short-answer training data have preliminary evidence of failing to steer long generations; differing system prompts and answer-position changes also reduce effectiveness.Source: §13.1 · p. 42
• Steering can reduce language-model quality, diversity, and instruction following while affecting unrelated concepts or missing contextual facets, creating practical side effects.Stronger interventions can worsen capabilities; examples of unwanted effects include reduced refusal rates and increased age-based bias.Source: §13.1 · pp. 42–43
• Steering outcomes vary with hyperparameters, concepts, and inputs, and successful interventions do not establish interpretive validity; RepE also requires internal access and entails derivation costs.Some concepts cannot be steered, and interventions can have negative effects on individual inputs even when average behavior improves.Source: §13.1 · pp. 43–44

13.2.1 Challenges in Representation Identification

• Identification may entangle concepts, mis-specify targets through scoring or prompts, and depend on suitable data and learned representations, with no ground truth to verify discoveries.A prompt may activate a representation of what the human wants rather than the intended concept, and methods cannot control concepts the model has not represented.Source: §13.2.1 · pp. 44–45

13.2.2 Challenges in Representation Operationalization

• Operationalization can fail when representations are nonlinear, context-dependent, dynamic, or distributed across layers; concepts may also have multiple distinct steering directions.A static or single-layer operator may capture only a subset of situations or aspects of a concept.Source: §13.2.2 · p. 45

13.2.3 Challenges in Representation Control

• Manipulating activations can shift them off their natural distribution and disturb later computations, potentially reducing capabilities, especially with stronger or multi-concept interventions.The section’s takeaway links control weaknesses to difficulty retaining the model’s natural activation distribution.Source: §13.2.3 · pp. 45–46

14.1 Opportunities to Improve Representation Identification

• Representation Identification could improve through refined operators, stronger datasets, better output scoring, method comparisons, failure analysis, and automated interpretability.Refinement could fine-tune Input Reading operators or use an SAE to remove unrelated features; Wu et al. (2025) propose jointly learning detection and steering. Operator quality depends on dataset quality (Braun et al., 2025) and improves with more data; investigate dataset properties, human or model judges, reward models, context-dependent or environment-based metrics, cross-method comparisons, hypothesized spurious correlations and misspecification, and LLM review of activating inputs and outputs.Source: §14.1 · p. 46

14.2 Opportunities to Improve Representation Operationalization

• Operationalization should move beyond static linear, single-concept representations toward nonlinear, context-dependent, time-varying, and cross-layer models.Potential approaches include situation-specific operators or a model that selects operators, representation trajectories across inference steps, and jointly optimized operators across layers; Lindsey et al. (2024) report concepts spread over multiple layers.Source: §14.2 · pp. 46–47
• RepE could model interactions between concepts and test richer operators and feature geometries rather than assuming isolated concepts and fixed representational forms.Interaction targets could include oA∧B or oB|A; richer operators might use input-conditioned neural networks or ensembles. Testing multiple geometries per concept may identify effective forms and deepen understanding of Transformer concept geometry (Jiang et al., 2024a; Csordás et al., 2024).Source: §14.2 · p. 47

14.3 Opportunities to Improve Representation Control

• Control research could combine weight and activation steering, sequence interventions over generation, and explore more expressive functions and training-time representation changes.Krasheninnikov & Krueger (2024) found in a toy setting that more expressive functions raised the performance ceiling and controlled more complex tasks while requiring more data. Yu et al. (2024) ablated refusal features during training; Cloud et al. (2024) used Gradient Routing to localize capabilities and enable inference-time unlearning.Source: §14.3 · p. 47

14.4 Possible Applications of Representation Engineering

• Possible applications include steering agent goals, preventing deception, studying in-context learning, and shaping cooperation in multi-agent settings.RepE might directly edit an agent’s pursued goal, support more honest behavior when a system attempts deception, and help study the hypothesis class and algorithms underlying in-context learning.Source: §14.4 · p. 48
• Further applications include personalizing agents’ ethical values, red-teaming models, and tracking how concept representations develop across training and model scales.The section proposes applying RepE in red-teaming open-weight models and evaluating worst-case jailbreaks of closed-weight models, as well as studying concepts across pre-training, post-training methods, and models of different sizes.Source: §14.4 · p. 48

14.5 Building a More Rigorous Science of Representation Engineering

• A widely accepted benchmark, rigorous evaluation standards, and theoretical frameworks are needed to strengthen RepE’s scientific foundation.Causal frameworks could clarify objectives and estimate intervention effects, but DAG-based approaches require mapping activations to graph nodes and PO approaches assume observed covariates adequately control confounding. Causal dataset design may improve isolation of mechanisms and out-of-distribution generalization.Source: §14.5 · pp. 48–49
• Toy testbeds, a library of concept representations, accessible best practices, and dedicated software could support understanding, comparison, and adoption of RepE.Toy settings can test effects of model size, data quality, and assumed geometry; a concept library could support operator prediction and serve as a baseline. Gemma Scope exemplifies tooling for analyzing and steering Gemma-2-2b SAE features.Source: §14.5 · p. 49

15 Conclusion

• The survey reviews over 100 recent studies and frames Representation Engineering as a pipeline of representation identification, operationalization, and control.It presents RepE as a paradigm for controlling and interpreting LLMs by manipulating their internal representations.Source: §15 · p. 50
• Most RepE methods identify representations with contrasting inputs and control them by adding activation vectors, often relying on the Linear Representation Hypothesis.Applications include AI safety, ethics, and interpretability, steering concepts such as harmfulness, fairness, and truthfulness.Source: §15 · p. 50
• RepE tends to outperform prompting, fine-tuning, and decoding-based methods at lower cost to generation quality, but reliability, robustness, and quality remain challenges.Challenges include long-form generation, multi-concept control, spurious concept relations, feature-geometry assumptions, and distribution shifts; the survey calls for more rigorous evaluations and study of nonlinear, dynamic, cross-layer, and interacting representations.Source: §15 · p. 50

15.1 Limitations

• The survey is comprehensive only for RepE papers published before 31.8.2024, and limited empirical comparisons leave many questions about best practices unanswered.It includes some papers published up to 31.2.2025, but does not cover them comprehensively.Source: §15.1 · p. 50

Broader Impacts

• RepE has dual-use risks: steering can reduce harmful outputs or align models with human intent, but can also enable jailbreaks, deception, misalignment, and stronger societal biases.The survey notes prior use of RepE to bypass safety mechanisms and produce more harmful outputs.Source: Broader Impacts · p. 50
• RepE can support interpretation of model internals, but it does not provide perfect explainability and may create an unwarranted illusion of understanding.Users may trust explanations more than is warranted.Source: Broader Impacts · p. 50

A.1 Commonly Used Models

• The meta-survey, covering papers published before 31.8.2024, finds that Llama-2-7B, Llama-2-13B, Mistral-7B, and Llama-3-8B base and instruction-tuned models are most popular.It also collected model sample sizes and statistics on authors and publication dates.Source: A Meta-survey · p. 64, §A.1 · p. 64
• 164 of 229 experiments used base models, while 65 used preference-tuned models; findings on whether instruction-tuning changes steering effectiveness are mixed.Dong et al. (2024b) and von Rütte et al. (2024) report larger effects on base models, while Bortoletto et al. (2024) and Wang et al. (2024c) find no clear relationship. Most studied models have 3–10 billion parameters; 9 of the 10 most popular have 7–13 billion, and 8 are Llama-family or derived models. Four papers find no clear relationship between scale and steerability, and only Templeton et al. (2024) study a proprietary state-of-the-art model. RepE has also been applied to RNNs and a vision-language model, suggesting architecture agnosticism.Source: §A.1 · p. 64

A.2 Number of Samples

• Most RepE papers use 100–1,000 samples for representation identification, and RepE effectiveness can improve with more samples.Figure 5 summarizes experiments by sample amount.Source: §A.2 · p. 64, Figure 5 · p. 65, §A.3 · p. 66
• Qiu et al. (2024), Yin et al. (2024), and Wang et al. (2024c) report improved steering with more identification samples, unlike Adila et al. (2024b); expressive operators benefit more from abundant data.More expressive steering functions with more operator parameters and operations work less well with little data but can better use more data.Source: §A.2 · p. 64
• There is no general recommendation for sample quantity because it depends on the model, target concept, and method; sample-efficient methods and RepE scaling laws remain open opportunities.Source: §A.2 · p. 65

A.3 Statistics about Publications on Representation Engineering

• Of 87 papers, 60 focus mainly on methodological improvements, 22 on applying RepE to problems, and 3 on evaluating RepE methods.Source: §A.3 · p. 65
• Of 87 papers, 58 have only academic co-authors, 19 are academia–industry collaborations, and 8 are authored solely by industry researchers.Source: §A.3 · p. 65
• Most models used have 3–10 billion parameters, and current literature shows no apparent relationship between model scale and RepE effectiveness.Source: §A.3 · p. 66
• The RepE literature is very new but growing rapidly.Source: §A.3 · p. 66

B Survey Methodology

• The structured survey produced a taxonomy and application summary while also comparing RepE with other methods and gathering meta-statistics.Source: B Survey Methodology · p. 66

B.1 Literature Search

• The literature search began from three seminal papers, used forward citation search and combinations of RepE-related and model-related terms, then checked relevant papers’ citations.Source: §B.1 · p. 66
• Included studies had to concern LLMs, identify a concept representation, control behavior, identify representations post hoc, and steer intermediate representations.The criteria exclude image-generation steering, interpretation without representation modification, retraining, and changes to token embeddings or logits.Source: §B.1 · pp. 66–67
• The search was conducted in September 2024 and covered papers published through August 31, 2024, without guaranteeing complete coverage of later work.Source: §B.1 · p. 67

B.2 Extracting Information from Papers

• After an initial non-exhaustive read, exhaustive reading classified methods in the taxonomy and recorded application targets, comparisons, experimental settings, publication metadata, and authors’ views.The extracted details supported mapping methodological choices and clustering papers by application.Source: §B.2 · p. 67
• Claude-3.5-Sonnet assisted with information extraction, but the authors read every paper and verified generated answers because the model was not sufficiently reliable.Source: §B.2 · p. 67

C Information Extracted from Papers

• The extraction catalog recorded identification methods, information sources, activation elicitation, vector construction, supervision, and identification data.It also tracked prompting formats and where activations were read.Source: C Information Extracted from Papers · p. 68
• The catalog tracked control mechanisms and static versus input-adaptive changes, alongside steering successes, notable results, weaknesses, and future work.Source: C Information Extracted from Papers · pp. 67–68

D Papers that compare RepE to Prompting, Fine-tuning and Decoding-based methods

• The appendix lists prompting comparisons including Zhang et al. (2024c), Cai et al. (2024), Konen et al. (2024), Arditi et al. (2024), and Ghandeharioun et al. (2024).The full list also includes Zhang & Viteri (2024), Chen et al. (2024b), Stickland et al. (2024), Guo et al. (2024), Qiu et al. (2024), Leong et al. (2023), Li et al. (2024c), Luo et al. (2024), Cao et al. (2024b), Scalena et al. (2024), Wang et al. (2024b), Liu et al. (2024b), Li et al. (2024e), Liu et al. (2024a), Wang et al. (2024c), and Li et al. (2024a).Source: D Papers that compare RepE to… · p. 69
• Fine-tuning comparisons include Lucchetti & Guha (2024), Li et al. (2024d), Yang et al. (2024), Price et al. (2024), and Pham & Nguyen (2024).The list also includes Liu et al. (2024b), Wang et al. (2024b), Wu et al. (2024b), Yu et al. (2024), Leong et al. (2023), Qiu et al. (2024), Guo et al. (2024), Stickland et al. (2024), Qian et al. (2024), Chen et al. (2024b), Zhang et al. (2024b), Liu et al. (2023), Wang et al. (2024a), Ackerman (2024), Li et al. (2024b), Yin et al. (2024), Cao et al. (2024a), and Wu et al. (2024a;c).Source: D Papers that compare RepE to… · p. 69
• The listed decoding-based comparisons include Li et al. (2024a;d), Cao et al. (2024b), Banerjee et al. (2024), Leong et al. (2023), Qiu et al. (2024), and Zhang et al. (2024b).Source: D Papers that compare RepE to… · p. 69
• Combination comparisons are listed for Wang et al. (2024c), Li et al. (2023a), Stickland et al. (2024), and Chen et al. (2024b).Source: D Papers that compare RepE to… · p. 69

D.1 Table of empirical comparisons

• Tables 14 and 15 report steering effect and, when available, model quality for concepts compared with prompting and fine-tuning methods.Scores across multiple models or test-set subsets are averaged; studies reporting only graph-based results are excluded because reading graph values may be imprecise.Source: §D.1 · p. 69
• Table 15 collects studies comparing RepE and fine-tuning on steering effect and model quality.Source: Table 15 · p. 70