Locate, Steer, and Improve: A Practical Survey of Actionable Mechanistic Interpretability in Large Language Models

Foundations

• The foundation defines token embeddings, transformer blocks and residual streams, multi-head attention, feed-forward networks, and sparse autoencoder features as core interpretable objects.These components are introduced in §2 as the basis for subsequent analysis.Source: Paper Outline · p. 4

Improve Alignment and Capability

• Model-improvement applications address safety and reliability, fairness and bias, persona and role, multilingualism, knowledge management, and logic and reasoning.These application areas are organized under alignment and capability improvements in §5.Source: Paper Outline · p. 4

Improve Efficiency

• Efficiency improvements are divided into efficient training and efficient inference.The outline presents these as the two subsections of §5.3.Source: Paper Outline · p. 4

Localization

• Localization methods in §3 comprise magnitude analysis, causal attribution, gradient detection, probing, and circuit discovery.Figure 1 characterizes §3 as the paper’s localization stage.Source: Paper Outline · p. 4, Figure 1 · p. 5

Steering

• Steering methods in §4 cover amplitude manipulation and targeted optimization.These methods constitute the intervention stage following localization.Source: Paper Outline · p. 4, Figure 1 · p. 5

Discussion and Challenges

• The paper concludes with a discussion of challenges in §6 after defining interpretable objects, introducing localization and steering methods, and showing their applications to model improvement.Figure 1 summarizes this progression from foundations through analysis and intervention to improvement.Source: Paper Outline · p. 4, Figure 1 · p. 5

Motivation

• Existing MI research explains model mechanisms but underexplores translating microscopic insights into practical interventions for improving model behavior.The introduction distinguishes foundational work focused on scientific understanding from applied surveys that still lack clear method categories and sufficiently concrete application guidance.Source: §1 · pp. 5–6
• The survey transforms MI from passive observation into an actionable discipline by structuring model improvement as “Locate, Steer, and Improve.”The pipeline addresses the lack of a unified guide for categorizing internal methods and applying them to model optimization.Source: §1 · p. 6

Framework

• The framework defines interpretable objects and separates Localizing for causal diagnosis from Steering for manipulating components and model outputs.For each technique, the survey specifies its methodological formulation, applicable objects, and scope.Source: §1 · p. 6, Table 1 · p. 7

Applications

• The survey organizes applications into Improve Alignment, Improve Capability, and Improve Efficiency across eight scenarios, summarizing representative paradigms rather than merely listing papers.This organization is intended to clarify usage patterns and support transferring methods to new problems.Source: §1 · p. 6

Resources and Future Directions

• The work discusses challenges and future directions while curating over 200 papers tagged by localizing and steering methods in Table 2.The resource is intended to lower the barrier to entry and provide a practical, navigable reference for the community.Source: §1 · p. 6

2.1. Token Embedding

• Token embeddings map vocabulary items into residual-stream vectors, while modern LLMs typically apply positional information through RoPE within attention’s query and key vectors.The embedding matrix has shape W_E ∈ R^{|V|×d_model}; RoPE replaces input-layer absolute positional embeddings in modern architectures.Source: §2.1 · pp. 7–8

2.2. Transformer Block and Residual Stream

• The residual stream is a shared memory highway updated additively by MHA and FFN blocks, enabling decomposition of final predictions into component contributions for Logit Lens and causal mediation analysis.The additive form makes residual-stream features linear combinations of outputs from preceding components (Elhage et al., 2021; Meng et al., 2022, 2023).Source: §2.2 · p. 8

2.3. Multi-Head Attention (MHA)

• MHA contextualizes tokens through multiple heads that read via QK projections and write via OV projections, separating where information is attended from what information is transmitted.This decomposition supports functional roles such as Induction Heads and Previous Token Heads (Olsson et al., 2022; Singh et al., 2024; Wang et al., 2024b).Source: §2.3 · pp. 8–9

2.4. Feed-Forward Network (FFN)

• FFNs independently transform each token position and can be interpreted as Key-Value memories whose neurons detect patterns and write retrieved knowledge back to the residual stream.Each neuron consists of a key weight and value weight, linking activation detection to information writing (Geva et al., 2021, 2022; Dai et al., 2022).Source: §2.4 · p. 10

2.5. Sparse Autoencoder (SAE) Feature

• SAEs address polysemanticity from superposition by expanding dense activations into sparse semantic features, with expansion factors typically 16× to 128× the model dimension.SAE features provide an actionable vocabulary for steering model behavior by targeting individual feature activations (Templeton et al., 2024; Lieberum et al., 2024; He et al., 2025; Xu et al., 2025b; Cho and Hockenmaier, 2025; Li et al., 2025d).Source: §2.5 · pp. 10–11
• SAE quality is limited by dead latents, feature absorption, scaling costs, and imperfect faithfulness when reconstructed activations diverge from the original model dynamics.Proposed variants include Gated, Top-K, BatchTopK, JumpReLU, and Binary SAEs, while layer-wise training and overcomplete spaces increase time, memory, and storage requirements.Source: §2.5 · pp. 11–12

3. Localizing Methods

• Localizing Methods identify behavior-relevant objects such as neurons, attention heads, and SAE features, providing a diagnostic foundation for mechanism analysis and targeted steering.The methods narrow the search space to manageable functional units before detailed analysis or intervention.Source: §3 · p. 13

3.1. Magnitude Analysis

• Magnitude Analysis scores internal objects by numerical magnitude, then selects salient candidates through thresholding or top-k ranking across parameters, activations, attention heads, and layer representations.Common scores include L2 norms, L∞ norms, and frequency-based metrics; the method is training-free but data-dependent.Source: §3.1 · pp. 13, 15
• Magnitude Analysis is a scalable first-pass heuristic, but high activation does not establish causal necessity and rankings depend on data that elicits the target behavior.Its low computational cost avoids auxiliary training and expensive backward passes, whereas dormant behavior-relevant components may be missed.Source: §3.1 · p. 15

3.2. Causal Attribution

• Causal Attribution identifies functionally responsible objects by measuring output changes under controlled patching or ablation interventions, with larger effects indicating critical mediators.Patching replaces an object with a counterfactual activation, whereas ablation sets it to zero or a mean vector; Meng et al. (2022) used causal tracing to localize factual recall to early subject-token FFN retrieval and later final-token MHA transport.Source: §3.2 · pp. 15–17
• Causal Attribution provides definitive functional evidence but scales expensively with individually intervened objects, motivating gradient-based screening before fine-grained validation.Ablation has been used to isolate heads, neurons, and FFN value-vector roles across indirect-object identification, in-context learning, arithmetic reasoning, language-specific regions, and factual recall.Source: §3.2 · p. 17

3.3. Gradient Detection

• Gradient Detection ranks inputs, activations, heads, and parameters using target sensitivity, including gradient norms, gradient–input scores, and Integrated Gradients, as fast proxies for intervention effects.Scores can be aggregated over datasets and objectives, but rankings shift with the chosen target F(x).Source: §3.3 · pp. 17–19
• Gradient-ranked objects remain local, non-causal proxies, so Gradient Detection is typically paired with Causal Attribution despite requiring fewer backward passes than exhaustive intervention search.It applies broadly without additional training, but downstream cancellation and nonlinear finite interventions can invalidate first-order salience.Source: §3.3 · p. 19

3.4. Probing

• Probing trains restricted auxiliary predictors on frozen internal vectors to compare where labeled properties become decodable across layers and submodules.Typical workflows construct labeled factual or counterfactual evidence, log candidate states, and compare probe performance across objects; high probe accuracy alone does not show that the model uses the information.Source: §3.4 · pp. 19–20
• Probing trains matched supervised classifiers on residual, block-output, or SAE-feature activations to rank where labeled properties are most decodable across layers, heads, and FFNs.It is lightweight and standardized for representational localization, but decodability does not establish causal use, necessity, or sufficiency; rankings remain sensitive to labels, datasets, and token positions and typically require causal follow-up.Source: §3.4 · p. 21

3.5. Vocabulary Projection

• Vocabulary Projection uses the final unembedding matrix to decode intermediate residual states, attention heads, neuron value weights, and SAE features directly into semantic vocabulary predictions.It revealed an English-centric semantic bottleneck in multilingual models, where middle layers process abstract English-centered concepts before final layers return to the target language (Wendler et al., 2024), and identified IOI Name Mover and Negative Name Mover Heads (Wang et al., 2023b). Its zero-shot interpretation is efficient but relies on basis alignment and is approximate for sublayer components.Source: §3.5 · pp. 21–22

3.6. Circuit Discovery

• Circuit Discovery identifies sparse directed cross-layer pathways whose edges jointly mediate target behaviors, using patching, attribution scores, or feature-space replacement models followed by intervention validation.ACDC reduces exhaustive edge testing, while EAP scores edges from sender activation differences and receiver sensitivity; attribution rankings can miss nonlinear interactions and depend on the behavior, metric, and clean–corrupted contrast.Source: §3.6 · pp. 23, 25

4.1. Amplitude Manipulation

• Amplitude Manipulation modifies activation representations by zeroing, mean-centering, or patching, and scales specific features continuously, across residual states, attention-head outputs, neuron activations, and SAE features.Patching injects information from a different context, while scaling amplifies activation when α > 1 or attenuates it when 0 < α < 1.Source: §4.1 · p. 27
• Amplitude Manipulation is optimization-free, reversible, and suitable for real-time control, but depends on accurate localization and can cause side effects when features are polysemantic or scaling requires tuning.Unlike causal attribution, these operations actively control behavior rather than merely diagnosing causality.Source: §4.1 · pp. 27–29

4.2. Targeted Optimization

• Targeted Optimization constrains parameter updates with a mask over layers, modules, neurons, heads, or other subsets to achieve durable behavioral changes while preserving unrelated capabilities.Applications include localized knowledge editing, compact specialization, multilingual adaptation, and safety-preserving partial tuning, but success depends on correct localization and well-specified target and preservation data.Source: §4.2 · pp. 29–30

4.3. Vector Arithmetic

• Vector Arithmetic is lightweight and reversible, but its reliability depends on linear, sufficiently disentangled representations; orthogonal concept directions support compositionality, whereas entanglement causes interference.Weak intervention fidelity can improve surface behavior while reducing robustness or specificity and increasing jailbreak susceptibility (Goyal and Daumé III, 2026; Raedler et al., 2025).Source: §4.3 · pp. 32–33
• Targeted Optimization provides the strongest and most persistent control by updating localized parameters, whereas Amplitude Manipulation and Vector Arithmetic are cheaper, transient inference-time alternatives.Targeted Optimization requires weight and gradient access and higher computational cost, while activation-level methods are more sensitive to prompt variation and entangled features (Meng et al., 2022, 2023; Zhang et al., 2024b; Rimsky et al., 2024; Tan et al., 2024a).Source: §4.3 · p. 33, Table 3 · p. 33

• Localization identifies internal objects associated with specific model behaviors.Source: §5 · p. 34
• Steering intervenes on these internal objects to modulate model outputs.Source: §5 · p. 34
• The applications are organized around three objectives: alignment, capability, and efficiency.Source: §5 · p. 34

5.1.1. Safety and Reliability

• Safety alignment uses two complementary paradigms: direct manipulation of localized unsafe components and vector steering of approximately linear safety and reliability representations.Amplitude Manipulation suppresses unsafe activations, Targeted Optimization rewrites safety-relevant parameters, and Vector Arithmetic steers refusal, truthfulness, hallucination, and instruction adherence while largely preserving general capabilities.Source: §5.1.1 · p. 34
• Safety-relevant heads, neurons, circuits, and SAE features can be localized and selectively edited to weaken harmful behavior, reduce hallucinations, detoxify outputs, or improve refusal with limited collateral effects.Examples include safety neurons in early self-attention layers, toxicity- and hallucination-associated neurons, and harm- or refusal-related SAE features; persistent edits can update only localized circuits or neuron-associated weights.Source: §5.1.1 · pp. 34–35
• Latent-space steering diagnoses and controls refusal, factuality, hallucination, and instruction-following signals, including refusal suppression in jailbreaks and refusal cliffs caused by late-stage attention heads.Vector Arithmetic can remove false refusal while preserving true refusal, amplify factual signals, enhance truthfulness, and improve instruction adherence without full retraining.Source: §5.1.1 · p. 35

5.1.2. Fairness and Bias

• Fairness interventions follow three patterns: causal localization and selective debiasing for gender bias, distributed searches for demographic and cultural signals, and inference-time correction of evaluation biases.Methods include Amplitude Manipulation, Targeted Optimization, Gradient Detection, Magnitude Analysis, Probing, attention or position reassignment, and targeted scaling.Source: §5.1.2 · p. 36
• Gender bias can be reduced by editing localized late-layer heads, FFN-attention divisions, or gender-specific neurons, while broader racial, social, cultural, positional, and moral biases require distributed localization and targeted representation or state corrections.Racial signals were reported as more scattered across early and middle FFN layers than gender, while positional and moral-bias interventions targeted hidden-state channels, attention-derived positions, residual activations, or alignment-induced value shifts.Source: §5.1.2 · pp. 36–37

5.1.3. Persona and Role

• Persona alignment combines global vector modulation, component editing, and probing-based psychological diagnosis to control traits, roles, values, emotions, and truthfulness without necessarily changing weights.Vector Arithmetic supports sycophancy reduction, role and trait composition, and domain steering; localized neuron or head edits enable fine-grained control, while probing identifies where traits emerge and predicts behavior.Source: §5.1.3 · pp. 37–38
• Persona steering is effective but conditional: it can cause unintended safety side effects and works better for latent dispositions than knowledge-heavy identities, where coherence may degrade.Reported risks include reduced privacy awareness after joy steering and increased toxicity after anger steering, motivating rigorous safety evaluation.Source: §5.1.3 · pp. 38–39

5.2.1. Multilingualism

• Language-specific neurons and SAE features support targeted multilingual control, while residual-stream steering aligns cross-lingual representations and mitigates language bias or mixing.MWork found English-centric intermediate representations, and later interventions restored language-faithful generation; language-specific components also enabled switching, adaptation, and fine-grained linguistic control.Source: §5.2.1 · pp. 40–41

5.2.2. Knowledge Management

• MI manages knowledge through localized persistent edits, reversible activation interventions, stability constraints, and vector-based consolidation of specialized capabilities.These methods address factual updating, unlearning, conflict suppression, catastrophic forgetting, and multi-model composition while controlling collateral effects and interference.Source: §5.2.2 · pp. 41–43
• Knowledge-editing methods localize factual carriers, often in mid-layer FFN pathways, and apply targeted optimization or activation control for specific updates, unlearning, and toxicity reduction.Query-conditioned localization and language-agnostic factual neurons broaden editing beyond fixed knowledge-neuron assumptions, while SAE guardrails support precision unlearning and controlled fluency impact.Source: §5.2.2 · pp. 41–42

5.2.3. Logic and Reasoning

• Reasoning capability improves through refinement of numerical and logical circuits, steering vectors or features that control cognitive strategies, and monitors that trigger selective self-correction.These interventions target arithmetic components, backtracking, verification, reasoning intensity, uncertainty, and latent reasoning paths, while internal probes distinguish correct from hallucinatory steps.Source: §5.2.3 · pp. 43–45

5.3.1. Efficient Training

• Sparse fine-tuning updates task-critical neurons, heads, or circuits instead of external modules, often matching full fine-tuning while reducing overhead and sometimes updating less than 1% of parameters.Gradient Detection, Magnitude Analysis, and Circuit Discovery support methods including LANDeRMT, SIFT, NeFT (Xu et al., 2025a), and circuit tuning (Li et al., 2025h).Source: §5.3.1 · pp. 45–46
• Internal metrics expose training phase transitions that validation loss misses, enabling informed early stopping for grokking, In-Context Learning, and In-Context Meta-Learning.Restricted Loss, FFD, router similarity and consistency, LLC plateaus, and label-attention scores track circuit formation, grokking onset, structural transformations, and Non-Context → Semi-Context → Full-Context progression.Source: §5.3.1 · pp. 45–47

5.3.2. Efficient Inference

• Selective computation prunes low-importance tokens, KV states, layers, experts, or neurons while retaining critical retrieval heads and specialized sub-circuits for task-specific inference.TokenSkip (Xia et al., 2025), FitPrune (Ye et al., 2025b), ZipCache (He et al., 2024a), and model-level skipping methods use Magnitude Analysis, Gradient Detection, or Circuit Discovery; pruning Super Experts causes catastrophic performance degradation (Su et al., 2025c).Source: §5.3.2 · pp. 47–48
• Layer-specific adaptive quantization assigns higher precision to functionally sensitive layers and aggressively compresses robust layers, improving memory-accuracy trade-offs across hardware constraints.Sensitivity proxies include representational shifts, activation sensitivity, weight kurtosis, energy concentration, relevance propagation, and vocabulary-distribution similarity in methods such as SensiBoost, KurtBoost, LieQ, Mix-QViT, and LSAQ.Source: §5.3.2 · pp. 47–48

Scalability Constraints and Granularity

• MI remains difficult to scale beyond low-level components, while higher-level structure discovery depends on manual inspection and causal localization becomes infeasible beyond 100B-parameter models.Automation remains limited in robustness across prompts, tasks, and models; SAEs and exhaustive interventions add costs that grow with model size and feature dimensionality, forcing coarser or heuristic analyses (Kharlapenko et al., 2025; Nikankin et al., 2025a; Duan et al., 2025; Bricken et al., 2023; He et al., 2024b; Marks et al., 2025; Yao et al., 2024a; Lindsey et al., 2025; Nguyen et al., 2025d; Conmy et al., 2023; Hanna et al., 2024, 2025; Prakash et al., 2024; Li et al., 2025k; Zhang and Nanda, 2023).Source: §6 · p. 49

Distributed Mechanisms vs. Localized Sparsity

• Sparsity improves interpretability but can obscure distributed or inactive components of genuine mechanisms, leaving the balance between localized sparsity and mechanistic completeness unresolved.Developing metrics that balance these objectives remains an open challenge (Gao et al., 2025b; Pach et al., 2025).Source: §6 · p. 49

Intervention Robustness and Side Effects

• MI-informed editing and steering often lack robustness and predictability, because targeted behavioral changes can cause collateral degradation across tasks or domains.Yu and Ananiadou (2025) show that modifying very few neurons can substantially degrade overall language performance, making accurate localization and side-effect avoidance central challenges (Yin et al., 2024; Wang et al., 2025b; Jiang et al., 2024b; Zhang et al., 2024b, 2025d; Hsueh et al., 2024; Xu et al., 2025c; Da Silva et al., 2025; Braun et al., 2025; Zhang et al., 2025j).Source: §6 · p. 49

Evaluation Protocols for Actionable MI

• Table 4 proposes a minimal actionable-MI benchmark covering math reasoning, safety, and knowledge editing through localized features, primary metrics, side-effect checks, and representative datasets.The framework supports causal-faithfulness tests such as targeted ablation and standardized comparisons of amplitude manipulation, vector arithmetic, and targeted optimization on identical features.Source: Table 4 · p. 50, §6 · p. 50
• No consensus metrics or mechanism-level ground truth currently supports reliable validation and comparison of localization and explanation methods, despite emerging benchmarks.This evaluation gap makes faithfulness assessment inherently challenging (Miller et al., 2024; Mueller et al., 2025; Parrack et al., 2025; Nguyen et al., 2025c; Wu et al., 2025b; Karvonen et al., 2025).Source: §6 · p. 50
• Side-effect checks must test preserved capabilities in math reasoning, over-refusal in safety, and locality plus fluency after knowledge editing.Math interventions should preserve general knowledge and safety performance; safety interventions should balance harmful-prompt refusal measured by SORRY-Bench (Xie et al., 2025) against benign-prompt acceptance measured by OR-Bench (Cui et al., 2024); knowledge edits should avoid unrelated factual changes and degraded generation (Zhang et al., 2024c).Source: §6 · pp. 50–51

Broadening the Architectural Scope

• The “Locate, Steer, and Improve” framework should extend beyond decoder-only Transformers to MoE and multimodal architectures using architecture-specific interpretable objects and interventions.MoE models introduce router states, routing activations, routing decisions, and expert activations, while MLLMs add visual representations, image-patch attention, and cross-modal pathways; direct adaptation may be insufficient for sparse routing and entangled multimodal representations.Source: §7 · p. 51

Integration with Cognitive Science

• MI should connect localized mechanisms to higher-order cognitive organizations, including System 1/System 2 reasoning and subsystems for attention, memory, language, and executive control.Such comparisons could test whether LLM circuits resemble cognitive functions such as working memory or attention control, while enabling bidirectional exchange between MI and cognitive theories.Source: §7 · pp. 51–52

Theoretical Foundations

• Stronger theoretical foundations from cognitive science and information theory could unify disparate MI findings, reduce ad-hoc interpretation, and clarify which internal structures large-scale models should contain and why.The cited directions include connecting internal representations to cognitive-science principles (Davies and Khakzar, 2024; Wulff and Mata, 2025; Zhang et al., 2025b; Ren et al., 2025a) and information theory (Conklin and Smith, 2024).Source: §7 · p. 52

From Interpretation to Interpretable Design

• Interpretable design should move beyond post-hoc analysis toward backbone architectures that achieve transparency by construction while matching state-of-the-art black-box performance.Concept Bottleneck Models and Weight-sparse transformers impose architectural transparency but typically underperform black-box models on large-scale, complex tasks, leaving performance parity as the central challenge (Srivastava et al., 2024).Source: §7 · p. 52

• The “Locate, Steer, and Improve” pipeline organizes actionable MI by localizing interpretable objects, causally manipulating them, and leveraging them to enhance LLM alignment, capability, and efficiency.Source: §8 · p. 52
• Recent advances in safety and persona alignment, knowledge editing, and sparse fine-tuning are most effective when grounded in explicit mechanistic intervention.Source: §8 · p. 52
• The survey identifies challenges and future directions aimed at integrating interpretability, intervention, and model design.Source: §8 · p. 52

Architectural scope

• The survey does not systematically cover MoE, vision–language, or vision-only architectures, although many methods may generalize through adapted expert-level or modality-specific representations.Comprehensive treatment of these architectures is left to future work.Source: Limitation · p. 53

Evaluation limitations

• Unified benchmarks and standardized evaluation protocols are lacking, hindering rigorous comparison of localization methods and assessment of whether identified components are causally optimal.Downstream interventions often rely on a single localization method without formal guarantees; combining methods can partially mitigate this issue, but principled reproducible frameworks remain open.Source: Limitation · p. 53

• Table 5 categorizes surveyed papers by primary interpretable object, localizing method, and steering method, based on searches across major academic platforms.A dash denotes studies that apply localized mechanistic insights for analysis or monitoring without active intervention.Source: Table 5 · p. 54

Persona and Role (Improve Alignment)

• Alignment studies examine persona and role using objects including MHA, FFN, neurons, and the residual stream, with methods such as causal attribution, probing, magnitude analysis, and amplitude manipulation.Source: A. Summary of Surveyed Papers · p. 55

Improve Capability

• Capability studies cover multilingualism, knowledge management, and logic and reasoning through interventions on neurons, SAE features, FFNs, MHAs, and the residual stream.Reported steering methods include amplitude manipulation, vector arithmetic, and targeted optimization, alongside non-intervention analysis or monitoring.Source: A. Summary of Surveyed Papers · pp. 55–57

Efficient Training (Improve Efficiency)

• Efficient-training studies primarily use neuron, MHA, FFN, and joint FFN–MHA analyses with targeted optimization, while some works omit steering interventions.Source: A. Summary of Surveyed Papers · pp. 57–58

Efficient Inference (Improve Efficiency)

• Efficient-inference studies apply amplitude manipulation to token embeddings, MHAs, FFNs, neurons, and residual streams, while other papers use circuit discovery, magnitude analysis, or no intervention.Source: A. Summary of Surveyed Papers · p. 58