PaperlayerView exact revision on arXiv

Source-linked AI summary

MIB: A Mechanistic Interpretability Benchmark

Aaron Mueller, Atticus Geiger, Sarah Wiegreffe, Dana Arad, Iván Arcuschin, Adam Belfki, Yik Siu Chan, Jaden Fiotto-Kaufman, Tal Haklay, Michael Hanna, Jing Huang, Rohan Gupta, Yaniv Nikankin, Hadas Orgad, Nikhil Prakash, Anja Reusch, Aruna Sankaranarayanan, Shun Shao, Alessandro Stolfo, Martin Tutek, Amir Zur, David Bau, Yonatan Belinkov

arXiv:2504.13151v2cs.LGcs.AIcs.CL

TL;DR

Existing benchmarks often cover a specific method class or particular tasks and models, limiting comparisons across mechanistic interpretability methods. MIB evaluates methods across standardized tasks and models, finding distinct performance patterns across circuit and causal-variable localization.

  • Problem

    Existing benchmarks compare within particular method classes or on specific tasks and models, while causal localization is identified as a key goal of mechanistic interpretability.

    Source: §1 · p. 2

  • Method

    MIB provides a standardized benchmark for localization and featurization methods, using a common model suite, fixed counterfactual inputs, and principled metrics.

    Source: §1 · p. 2

  • Results

    Attribution and mask optimization outperform other circuit-localization approaches, supervised DAS performs best for causal-variable localization, and SAE features generally do not improve on neurons.

    Source: §1 · p. 2, §4.7 · p. 11

  • Takeaways & Limitations

    MIB reveals differentiation among methods and supports the conclusion that mechanistic interpretability has made real progress.

    Source: §1 · p. 2

    The baselines leave room for improvement: hyperparameter searches were not exhaustive, and some task and model settings remain underexplored.

    Source: §4.7 · p. 12

Abstract

from arXiv · show

How can we know whether new mechanistic interpretability methods achieve real improvements? In pursuit of lasting evaluation standards, we propose MIB, a Mechanistic Interpretability Benchmark, with two tracks spanning four tasks and five models. MIB favors methods that precisely and concisely recover relevant causal pathways or causal variables in neural language models. The circuit localization track compares methods that locate the model components - and connections between them - most important for performing a task (e.g., attribution patching or information flow routes). The causal variable localization track compares methods that featurize a hidden vector, e.g., sparse autoencoders (SAEs) or distributed alignment search (DAS), and align those features to a task-relevant causal variable. Using MIB, we find that attribution and mask optimization methods perform best on circuit localization. For causal variable localization, we find that the supervised DAS method performs best, while SAE features are not better than neurons, i.e., non-featurized hidden vectors. These findings illustrate that MIB enables meaningful comparisons, and increases our confidence that there has been real progress in the field.

1. Introduction

MIB proposes standardized comparisons of mechanistic interpretability methods, centered on localizing task-relevant causal mechanisms. Its results distinguish methods across circuit and causal-variable localization.

1. Introduction

  • The benchmark targets precise localization and causal validation of task mechanisms or concepts, which the authors identify as a goal of many mechanistic interpretability pipelines.They note existing benchmarks often focus on a method class or a narrower set of tasks and models.Source: §1 · p. 2
  • MIB aims to enable principled, stable comparisons of localization and featurization methods using standardized models, datasets, counterfactual inputs, metrics, and public leaderboards.Submissions are evaluated on a private test set.Source: §1 · p. 2
  • MIB has two tracks: circuit localization finds important model components, while causal-variable localization selects features aligned with task-relevant causal variables.The latter includes methods that featurize hidden vectors, such as mapping them to another vector space.Source: §1 · p. 2
  • Attribution and mask optimization methods outperform other circuit-localization approaches, while supervised DAS performs best for causal-variable localization and SAE features do not improve on standard hidden dimensions.The authors argue these differences show that MIB can distinguish methods and provide evidence of real progress.Source: §1 · p. 2

2. Materials

MIB evaluates four tasks with fixed counterfactual inputs, a selected model suite, and public and private test splits. Its leaderboards support method development and private-test evaluation.

2.1. Tasks

  • The task suite spans four tasks selected to vary in reasoning type, difficulty, and answer format, including IOI, arithmetic, MCQA, and ARC.The authors chose IOI and arithmetic as extensively studied tasks, and MCQA and ARC because they had received less study; ARC is split into Easy and Challenge subsets.Source: §2.1 · pp. 2–3

2.2. Counterfactual Inputs

  • Counterfactual interventions set model components to values they would take on alternative inputs, and evaluations use fixed instance-to-counterfactual mappings.For circuit localization, success means components outside the circuit minimally change predictions; for causal-variable localization, interventions on a variable should have the same effect as interventions on the component.Source: §2.2 · p. 3

2.3. Models

  • The benchmark uses four open-weight models spanning different sizes, families, capabilities, and prominence in mechanistic interpretability.Gemma 2 and Llama are generally capable across tasks, while GPT-2 Small is smaller and less capable; Qwen-2.5 performs well relative to its size but has been less studied.Source: §2.3 · pp. 3–4

2.4. Leaderboard

  • Two public leaderboards accept submissions, with a portal that runs evaluations on a private test set.The public test set is intended for faster method prototyping, while the private test set is intended to measure benchmark state of the art.Source: §2.4 · p. 4

3. Circuit Localization Track

The circuit localization track evaluates methods for finding causal subgraphs that explain task behavior. It introduces circuit metrics and compares localization methods, with EAP-IG-inputs achieving the highest average performance on CMD and CPR.

3.1. Circuit Metrics

  • Faithfulness is split into CPR, which rewards positive effects on task performance, and CMD, which captures strong effects including negative ones.The authors describe the two aims as complementary: understanding components that encourage behavior versus explaining the full algorithm behind it.Source: §3.1 · p. 4
  • CPR and CMD aggregate faithfulness across circuit sizes to capture minimality and faithfulness together, approximating the integrals with a Riemann sum over representative sizes.The authors present these metrics as relative comparison points rather than absolute measures.Source: §3.1 · p. 5
  • The weighted edge count treats a node as all its outgoing edges and weights each submodule’s edges by the fraction of its neurons included in the circuit.The count is normalized by the number of possible edges to yield a percentage.Source: §3.1 · p. 5

3.3. Results

  • EAP-IG-inputs with counterfactual ablations performs best on average on both CMD and CPR, while EAP-IG-activations, EAP, and UGS remain competitive.Table 2 reports CMD scores across methods and ablation types; CPR scores are reported in Appendix D.Source: Table 2 · p. 6, §3.3 · pp. 6–7
  • EAP-IG-activations and UGS are competitive with EAP-IG-inputs on CMD but have higher runtime and memory use, respectively.They are less competitive on CPR; UGS optimizes for maintaining model behavior rather than finding only positively impactful components.Source: §3.3 · p. 7
  • EActP dominates for IOI on GPT-2, but not Qwen-2.5 or InterpBench, despite computing exact indirect effects for each edge.The authors suggest fewer examples due to runtime may contribute; independent edge effects may also imperfectly predict joint ablation effects.Source: §3.3 · p. 7
  • CF-ablation circuits outperform mean- and optimal-ablation circuits, which score similarly.The authors attribute this to CF circuits more closely matching the evaluation setting and localizing distinctions in CF input pairs.Source: §3.3 · p. 7
  • Node-level circuits perform poorly, while IFR tends to underperform attribution methods but usually exceeds random circuits.The authors suggest node circuits pay for many edges per node, limiting their sparsity.Source: §3.3 · p. 7

4. Causal Variable Localization Track

The causal variable localization track tests whether hidden-vector features align with task-relevant concepts along causally active paths. It uses aligned interventions to compare model outputs with a high-level causal model, and finds DAS consistently leads while SAE and PCA features generally do not outperform standard hidden dimensions.

4.1. Causal Abstraction

  • The causal abstraction framework aligns high-level concepts with hidden-vector features and measures faithfulness by whether aligned interchange interventions produce the same outputs.The evaluation filters out examples where the model predicts incorrectly on the base input or a counterfactual; the authors caution that LM concepts may differ from human concepts.Source: §4.1 · pp. 7, 9

4.2. Causal Variable Localization Baselines

  • Baselines include Full Vector, PCA, SAE, DBM feature selection, and supervised DAS, which learns features to maximize faithfulness to the causal model.Alignments can target dynamically selected hidden vectors, such as the residual stream at the correct answer token.Source: §4.1 · p. 7, §4.2 · p. 9

4.3. MCQA and ARC (Easy)

  • On MCQA and ARC, DAS disentangles answer ordering from the output token in many layers, while the full-vector baseline performs poorly on average.The causal model represents answer position with XOrder and retrieves the answer token through OAnswer.Source: §4.3 · p. 9

4.4. Two-Digit Addition

  • Two-digit addition results are poor overall: Gemma-2 is at chance and Llama-3.1 shows only faint signs of success on the carry variable.The authors suggest tokenization may explain the difference and note that a carry variable may be absent or represented nonlinearly.Source: §4.4 · pp. 10–11

4.5. RAVEL

  • RAVEL attributes are generally localizable and disentangled, though results are weaker on Llama-3 and Full Vector fails under the balanced evaluation.Source: Table 3 · p. 10, §4.5 · p. 11

4.6. Indirect Object Identification.

  • For IOI, token and position variables can be disentangled at the head level, with better results when aligned to head features.The evaluation builds on experiments with four GPT-2 attention heads and tests counterfactual changes to subject tokens and positions.Source: §4.6 · p. 11

4.7. General Discussion

  • DAS consistently achieves the best results; DBM on standard dimensions is strong, but SAE and PCA features generally do not improve on those dimensions.The authors also report that DAS outperforming DBM favors non-basis-aligned directions, and that their baselines leave room for improvement because hyperparameter searches were not exhaustive.Source: Table 3 · p. 10, §4.7 · pp. 11–12

5. Related Work

Prior evaluations cover circuit discovery and causal-variable localization, but often constrain the tasks, models, or methods compared. Other benchmarks address model editing and automated or behavior-focused interpretability.

5. Related Work

  • Circuit-discovery evaluations include targeted tests and ground-truth circuit tasks, but the latter use relatively simple hand-crafted circuits, while many methods papers cover only a few tasks and models.Source: §5 · p. 12
  • RAVEL and CausalGym compare featurization methods in narrower domains, while SAEBench is limited to SAEs; MIB spans tasks, models, and methods.Source: §5 · p. 12
  • Other evaluation areas include targeted model editing, agents that describe latent functions, and benchmarks of explanations for language-model behavior.Source: §5 · p. 12

6. Conclusions

The paper presents MIB as a benchmark for directly comparing mechanistic interpretability methods. Its results corroborate the value of attribution methods and challenge the utility of SAEs as featurizers for known causal variables.

6. Conclusions

  • MIB provides a benchmark for directly comparing mechanistic interpretability methods.Source: §6 · p. 12
  • The findings corroborate recent results on attribution methods while challenging the utility of SAEs for featurizing known causal variables.Source: §6 · p. 12
  • The authors describe MIB as a living benchmark intended to scale with advances and support future state-of-the-art methods.Source: §6 · p. 12

Impact Statement

The impact statement frames MIB as a tool for standardizing interpretability evaluation, while noting that localization can be part of AI safety and robustness pipelines. It also says MIB is intended to support stronger conclusions about localization methods across downstream applications.

Impact Statement

  • The work aims to standardize evaluation of mechanistic interpretability methods, some of which contribute to AI safety and robustness pipelines through localization.Source: Impact Statement · p. 12
  • The authors say MIB is not expected to directly contribute to harms and is intended to support conclusions about localization methods regardless of downstream application.Source: Impact Statement · p. 12

A. Table of Notation

B. Limitations

MIB’s limitations concern how its tracks divide related problems, the availability and assumptions of causal models, and its focus on language models. The accompanying materials describe task construction, scoring, leaderboards, and baseline procedures.

B. Limitations

  • Separating featurization from causal-dependency localization simplifies evaluation, but the problems can influence each other and may need joint treatment.Feature-level units could improve circuit localization, while DAS outputs are not directly applicable to finding circuits.Source: B. Limitations · p. 19
  • Completeness cannot be tractably measured without ground-truth relevant components; MIB includes this metric for InterpBench but not its other models.The authors caution that circuit-complement faithfulness may not indicate recovery of the full set of dependencies.Source: B. Limitations · p. 19
  • Causal-variable faithfulness assumes the hypothesized high-level model exists in the computation graph, and evaluated models may not contain the shown graphs in exactly that form.The metric assesses whether the causal variable, rather than the entire high-level model, aligns with the representation.Source: B. Limitations · p. 19
  • MIB evaluates only large language models; the authors suggest extending future evaluations to other modalities.They describe the current model scope as a broad-coverage sample of models commonly studied in the literature.Source: B. Limitations · p. 19

C. Further Details on Materials

  • IOI uses eight counterfactuals, while arithmetic counterfactuals alter operands to change the answer and potentially the operation process.IOI’s splits use disjoint attributes; its circuit track uses ABC, while its causal-variable track uses three IO/S flips.Source: §C.1 · p. 19, §C.2 · p. 22
  • MCQA varies semantic content and answer formats, whereas ARC uses format-based counterfactuals and single-token answer choices.The MCQA materials also provide versions with 2–10 choices; ARC omits semantic counterfactuals because prompts lack a shared token-level template.Source: §C.3 · p. 23, §C.4 · p. 23

C.5. Model Performance

  • Public test sets support iteration, but private test sets require submission through an API; model scoring also distinguishes greedy from ranked-choice accuracy.Ranked-choice accuracy is an upper bound on greedy performance; the leaderboards recompute averages after filtering, and the causal-variable board aggregates layers, positions, and counterfactual types.Source: §C.5 · p. 25, §C.6 · pp. 25, 27

D.1. Methods

  • Circuit construction usually selects top-scoring edges, while greedy construction is used when needed for information-flow routes; baseline methods differ in scoring and computational demands.CPR uses high scores and CMD high-magnitude scores; UGS can exceed 80G GPU memory on larger models, and edge activation patching and optimal-ablation methods scale poorly with model size.Source: §D.1 · pp. 27–28, §D.2 · pp. 28–29

D.3. Further Circuit Localization Results

The CPR results broadly resemble CMD trends, with EAP and UGS less competitive against EAP-IG-inputs. Some methods also have substantial memory or runtime constraints.

D.3. Further Circuit Localization Results

  • EAP and UGS are less competitive with EAP-IG-inputs on CPR, while the broader trends largely resemble CMD results.Table 14 reports CPR scores across circuit-localization methods and ablation types; Table 15 reports private-test CMD scores, where lower is better.Source: §D.3 · p. 29, Table 15 · p. 30
  • UGS can exceed 80G GPU memory on larger models, and edge activation patching and optimal-ablation methods scale poorly with model size.The authors omit methods that take over one week to run.Source: §D.3 · pp. 29–30

E. Details on InterpBench Model Training

The authors train a simplified IOI model to make circuit evaluation more precise, since faithfulness alone is fuzzy and unbounded. They also report arithmetic-task circuit scores by operator and note an arbitrary runtime threshold.

E. Details on InterpBench Model Training

  • Circuit faithfulness is fuzzy and unbounded, motivating a trained transformer that predicts IOI indirect objects and implements a simplified IOI circuit.The model follows InterpBench methods and Gupta et al. (2024).Source: E. Details on InterpBench Model Training · p. 30
  • The InterpBench model has 6 layers and 4 heads per layer, with dmodel = 64 and dhead = 16, and was trained for 70 hours on one H100 GPU.Training used varying-length mini-batches with left padding; the authors swept hyperparameters for SIIT and used its three-losses variant.Source: E. Details on InterpBench Model Training · p. 30
  • The runtime threshold is arbitrary and does not prevent submissions of methods that take longer.Source: E. Details on InterpBench Model Training · p. 30
  • Arithmetic CPR and CMD scores on Llama-3.1 are generally similar across operators, with similar method rankings except for EAP-IG-activations CPR.The table reports scores separated by operator.Source: Table 17 · p. 31

F.1. Causal Abstraction Analysis

Causal abstraction analysis tests whether interventions on aligned LM features reproduce the outputs of a hypothesized high-level causal model. The authors evaluate unsupervised features and supervised DAS alignments using interchange interventions.

F.1. Causal Abstraction Analysis

  • A high-level model is an abstraction of an LM under an alignment when corresponding high-level and low-level interchange interventions produce the same output.The framework treats the LM as a low-level causal model and the hypothesized structure as a high-level causal model.Source: §F.1 · p. 31

F.2. Aligning Unsupervised Features to Causal Variables

  • PCA, SAEs, and hidden-vector dimensions are evaluated as unsupervised features, with differential binary masking selecting features aligned to each causal variable.SAE feature interventions include the base input's reconstruction error to preserve featurizer invertibility.Source: §F.2 · p. 32

F.3. Distributed Alignment Search

  • DAS is a supervised featurizer that learns an orthogonal transformation and selects the first k dimensions as a causal-variable subspace.The implementation learns only the k orthogonal vectors needed for the feature subset and trains on base-counterfactual pairs with interchange interventions.Source: §F.3 · p. 32

F.4. Hyperparameters

  • DAS uses dimensionalities of 16 for XOrder and XCarry, 32 for STok and SPos, half the residual stream for OAnswer, and an eighth for RAVEL.Learning rates are 0.01 except 1.0 for IOI, no regularization losses are used, and masking temperature decreases from 1.0 toward 0.01; training epochs and batch sizes vary by task and model.Source: §F.4 · p. 33

F.5. High-level Causal Models and Experimental Details for Each Task

For multiple-choice tasks, DAS aligns representations with a two-stage answer process, isolating answer position and answer content. In arithmetic, apparent carry alignment disappears with random counterfactuals, indicating the initial signal tracked tens-digit output instead.

F.5.1. MULTIPLE-CHOICE QUESTION ANSWERING

  • The multiple-choice causal model first maps the prompt to an XOrder pointer for the correct option, then uses OAnswer to retrieve that option's letter.Interchange experiments vary the correct-option position and label; reported results average these with counterfactuals changing only position or only label.Source: §F.5.1 · pp. 33–34, Figure 7 · p. 33
  • DAS isolates XOrder answer-position information in Gemma's middle layers and disentangles OAnswer from ordering and positional information across nearly every layer.Full-vector patching succeeds at the final layer and last token for OAnswer, where the model predicts its next token, but fails at other locations.Source: §F.5.1 · p. 34

F.5.2. ARITHMETIC

  • The arithmetic causal model parses addend digits, uses units-digit addition to determine the result's units digit and carry, then uses tens digits and carry to determine tens and hundreds digits.For 57 + 66, the model yields units digit 3, tens digit 3, and hundreds digit 1.Source: §F.5.2 · pp. 35–36, Figure 9 · p. 35
  • Random counterfactuals make XCarry alignment fail, showing that the earlier apparent carry signal likely represented tens-digit output correlated with carry.The targeted counterfactuals introduce or remove a carry, whereas random counterfactual outputs are independent of the carry value.Source: §F.5.2 · pp. 35–36

F.5.3. RAVEL

RAVEL models attribute queries by parsing an entity and requested attribute, then selecting among the entity's attribute values. DAS isolates ACountry information in Gemma's mid layers, while full-vector patching fails to separate it from other attributes.

F.5.3. RAVEL

  • RAVEL parses a city entity and queried attribute, retrieves continent, country, and language values, then selects the value matching the query.For a prompt asking for Paris's continent, the model outputs Europe.Source: §F.5.3 · p. 36
  • Llama achieves 66.5% accuracy and Gemma achieves 70.5% accuracy on RAVEL after failure cases are filtered out.Source: §F.5.3 · p. 36
  • DAS achieves high ACountry interchange accuracy in Gemma-2 2B layers 18 and 19, while full-vector patching performs poorly in later layers at the last token.The attribute-counterfactual dataset balances ACountry queries with ACont and ALang queries, so success requires changing only ACountry while preserving the others.Source: §F.5.3 · p. 37, Figure 12 · p. 37
  • With the more challenging Wikipedia counterfactual, DAS shows meaningful ACountry signal in mid layers, while all five methods have lower accuracy on ACountry than on the other attributes.Full Vector IIA remains around or below 50 across both token positions; DAS patterns are generally lower than with attribute counterfactuals.Source: §F.5.3 · pp. 37–38, Figure 13 · p. 37

F.5.4. INDIRECT OBJECT INDENTIFICATION

IOI tests whether a model completes a sentence with the indirect object rather than the subject. The section separates token-identity and position signals in a linear account of the task and searches for S-inhibition-head subsets that align with those variables.

F.5.4. INDIRECT OBJECT INDENTIFICATION

  • The IOI task asks a model to predict the indirect object as the sentence’s final word, rather than the subject.The paper models the logit difference between the subject and indirect-object names.Source: §F.5.4 · p. 38
  • Wang et al. (2023) identify S-inhibition heads that carry token and position signals to help the model avoid the subject.Their interventions separately invert subject position or token identity, or both.Source: §F.5.4 · p. 38
  • On this dataset, 0.048 + 2.005PositionSignal + 0.768TokenSignal best predicts the logit difference; Wang et al. (2023) report 2.31PositionSignal + 0.99TokenSignal.The authors note that Wang et al. did not attempt to disentangle the two signals, which this work does.Source: §F.5.4 · p. 39
  • The causal model distinguishes STok and SPos, which track the subject token and position, from binary TokenSignal and PositionSignal variables.Its OLogDiff variable compares the subject token and position with the original input to determine whether each was inverted.Source: §F.5.4 · p. 39
  • The predicted logit difference changes with interventions on STok and SPos, with the unmodified prediction given by 0.048 + 2.005 + 0.768.Intervening on either variable changes the sign of its corresponding coefficient; intervening on both changes both signs.Source: §F.5.4 · p. 39
  • A brute-force search aligns SPos and STok with every possible subset of four S-inhibition heads, reporting mean-squared error in Table 18.The main text reports SPos aligned to heads 7.3, 7.9, and 8.6, and STok to head 8.10.Source: §F.5.4 · p. 39, Table 18 · p. 39

Related papers

Loading 2504.13151v2…