Source-linked AI summary
MURANO: Design, Run, and Reproduce Mechanistic Interpretability Experiments as Composable Pipelines
Alireza Bayat Makou, Emirhan Böge, Phu Gia Hoang, Federico Tiblias, Jingcheng Niu, Subhabrata Dutta, Richard Eckart de Castilho, Iryna Gurevych
TL;DR
Mechanistic interpretability studies often span operations and libraries whose outputs, addresses, and execution order must be coordinated. Murano provides composable ordered steps with shared result contracts and canonical component addresses, and its case studies recover selected findings while illustrating SAE feature steering. The framework’s evaluations remain bounded by fixed settings and untested usability, performance, and sensitivity factors.
Problem
Existing libraries cover different parts of mechanistic interpretability workflows, requiring researchers to adapt outputs, align component names, and coordinate dependent operations.
Method
Murano orchestrates loading, recording, attribution, intervention, and evaluation as ordered steps exchanging named artifacts through declared contracts and canonical Node addresses.
Results
Murano recovers selected interpretability findings, with reproduced truth-direction values differing from originals by at most 0.03, and demonstrates an SAE feature-steering workflow.
Takeaways & Limitations
Murano provides a reusable compositional framework for expressing selected mechanistic interpretability workflows across multiple operations and underlying libraries.
Takeaways & Limitations
The case studies use fixed settings and do not systematically test alternative samples or hyperparameters, usability, runtime, memory, model coverage, or reproducibility.
Abstract
from arXiv · showhide
This paper presents Murano, an open source framework for designing, running, and reproducing mechanistic interpretability studies of large language models, intended for researchers across disciplines. These studies often combine loading, recording, attribution, intervention, and evaluation, while existing libraries tend to focus on different parts of this workflow. As a result, researchers using several libraries may need to adapt outputs from one for use by another. To bridge this gap, Murano represents operations from these five areas as composable steps. Steps exchange named result artifacts and declare the inputs they require and the outputs they produce. A pipeline executes its steps in the order supplied, and Murano uses canonical addresses when component identities pass between operations. Murano builds on existing interpretability and machine learning libraries. We demonstrate Murano through two reproductions of established interpretability studies and one illustrative sparse autoencoder case study.
1 Introduction
Murano addresses the coordination burden of mechanistic interpretability studies by composing diverse operations as ordered steps that exchange named artifacts through shared contracts. Its framework supports exploratory and reproducible workflows while allowing researchers to extend the step library.
- Existing interpretability libraries provide complementary capabilities, so multi-library studies may require aligning component names, output formats, operation order, and intervention code.
- Murano organizes loading, recording, attribution, intervention, and evaluation as composable operations exchanged through declared result inputs and outputs.
- Pipelines execute compatible steps in supplied order, while shared Results artifacts and canonical Node addresses connect dependent operations.
- The framework supports activation recording, attribution, intervention, and evaluation in exploratory analyses, new experiments, and reproducible case studies.
2 Background and Related Work
Mechanistic interpretability studies analyze internal computations through recording, attribution, and intervention, building on a growing ecosystem of libraries with different abstraction layers and coverage. Murano positions itself as a compositional layer across these methods and systems.
- Mechanistic interpretability explains model behavior in terms of internal components and computations represented through the transformer residual stream.
- Activation patching transfers an activation between inputs, while path patching restricts the intervention to information transmitted from a chosen sender to receiver.
- Logit lens, direct logit attribution, sparse autoencoders, and activation steering provide complementary ways to inspect or alter model representations.
- Existing systems specialize in particular abstraction layers or workflow subsets, making alignment of model addresses, data formats, and intervention code necessary when combined.
- Murano coordinates selected implementations through shared Step and Results contracts and canonical Node addresses.
3 System Overview
Murano implements experiments as ordered steps that mutate a shared Results container, use declared artifact contracts, and address model internals through canonical Node objects. Its operations cover recording, attribution, intervention, evaluation, persistence, and selected SAE workflows within documented support boundaries.
- A Pipeline passes one mutable Results container through ordered steps, and phases may be omitted or repeated rather than being mandatory.
- Steps declare result keys and optional input types; Pipeline.run() checks required inputs immediately before execution but not produced output types when written.
- Pipeline.validate() checks declared dependencies without execution, while later errors can still appear only after earlier steps have run.
- The backend loads Hugging Face models through nnterp’s StandardizedTransformer built on nnsight, with MuranoModel as the only ModelBackend implementation.
- Canonical Node objects encode layers, modules, heads, projections, and token positions, but supported fields vary by operation and architecture.
- Murano includes recording, attention capture, logit-lens projection, steering, ablation, attribution, path patching, evaluation, and persistence operations.
4 Usage
Murano usage consists of configuring steps in execution order, running them, and consuming named artifacts through helper functions or later steps. The framework also supports multi-step SAE feature-steering workflows and documents broad operation-category coverage.
- A logit-lens pipeline loads prompts, stores a PromptBatch, projects selected residual outputs, writes a LogitLensResult, and plots that artifact afterward.
- Users construct a Pipeline from configured steps, call run(), and can validate declared key and type dependencies beforehand when no existing results are required.
- MuranoModel convenience methods support recording and baseline-versus-intervened generation without defining additional workflow phases.
- Murano is the only compared framework whose documented public interface directly implements all seven listed operation categories.
- An SAE steering workflow loads a pretrained SAE, encodes activations, ranks candidate features on selected tokens, and steers generation.
5 Evaluation and Case Studies
Murano expresses two established interpretability reproductions and an SAE feature-steering case study as workflows of composable operations. The reproductions recover selected reported patterns, while the SAE intervention demonstrates that large fixed additions can overwhelm generation.
- 5.1 Indirect Object Identification: The three reported name mover heads show the three strongest negative effects, while the two reported negative mover heads show the two strongest positive effects.The measurements characterize attention patterns and copying tendency, not copying in the original forward pass.
- 5.1 Indirect Object Identification: Query patches lower the clean-to-corrupt logit gap by 28.9% for L9H9 and 22.8% for L10H0, but raise it by 4.9% for L9H6.The S-inhibition result therefore holds for two of the three tested name mover heads; value patches produce no measurable change at displayed precision.
- 5.1 Indirect Object Identification: The IOI workflow patches attention-head outputs and computes logit differences, with MLP outputs left unfrozen in the reported head sweep.PathPatch captures clean and corrupt head outputs, inserts selected corrupt outputs, and LogitDiffStep computes the score.
- 5.2 Truth Direction: The truth-direction reproduction obtains normalized indirect effects of 0.88 and 0.98, compared with 0.85 and 0.97 in the original study.These mass mean interventions use cities+neg_cities training in LLaMA-2-13B; the largest difference is 0.03.
- 5.3 Sparse Autoencoder Feature Steering: The SAE case study selects feature 1466 from eight candidates and adds its normalized vector during Gemma 2 2B Instruct generation.At strength 420, two displayed generations contain broken or repeated fragments; at strength 2000, all four collapse into repetitions of California or Sacramento.
- 5.3 Sparse Autoencoder Feature Steering: The shown core SAE workflow contains 13 logical statements with Murano and 24 in the direct sae-lens snippet.The workflow encodes residual states, ranks candidate features, labels promoted output tokens, and constructs the additive intervention; it does not cover feature clamping.
- 5.2 Truth Direction: The truth-direction probe-accuracy comparison spans two probe types, two training compositions, and six datasets, with a largest unrounded difference of 10.53 percentage points.Figure 3 uses the same 75–100 percent scale in both panels and rounds cell values to one decimal place.
6 Conclusion
Murano coordinates selected mechanistic interpretability operations as ordered steps exchanging named results and consistent component addresses. Its case studies express selected workflows and reproduce selected measurements, while leaving readability, reproducibility, and extension effort unevaluated.
- 6 Conclusion: Murano coordinates selected interpretability operations as ordered steps that exchange named results and use consistent addresses for model components.The framework encapsulates recurring implementation patterns as reusable operations.
- 6 Conclusion: The IOI head sweep recovers the three reported name mover heads and two reported negative mover heads, while the S-inhibition query result holds for only two of three tested name mover heads.The truth-direction intervention values differ from the original by at most 0.03, whereas the probe-accuracy grid differs by at most 10.53 percentage points.
- 6 Conclusion: The SAE case study shows that a large fixed feature addition can overwhelm generation.The case studies show how selected workflows can be expressed, but they do not directly evaluate readability, reproducibility, or extension effort.
Limitations
Murano has system and scope limitations, including sequential execution, backend and architecture dependencies, memory demands, and restricted experiment settings. Its case studies do not systematically test sensitivity to alternative samples or hyperparameters.
- Pipelines execute sequentially and currently depend on nnterp and nnsight, with model and Node support varying by operation and architecture.
- Token positions depend on tokenization and padding, while analyses retaining many activations can require substantial accelerator memory.
- Murano currently targets forward passes and bounded text generation with one model, leaving language agents with tools, persistent memory, and multi-turn state for future work.
- The case studies use fixed prompt sets, data splits, corruption schemes, and intervention settings without systematic sensitivity tests for alternative samples or hyperparameters.
- The framework comparison measures documented interface coverage rather than implementation quality, usability, or performance.
Ethics Statement
Murano is intended to support research on understanding and controlling language-model behavior, but its intervention capabilities could also be misused. Users are advised to follow institutional policies, model licenses, and responsible disclosure practices.
- Murano’s intervention capabilities could be misused to elicit harmful outputs or weaken safeguards.
- Users should follow applicable institutional policies, model licenses, and responsible disclosure practices.
A Framework Capability Comparison
Table 2 compares documented capabilities across libraries, a model-access backend, and a hosted platform, while Figures 4–6 compare Murano with direct implementations for representative workflows. The comparisons use documented interface coverage and show shorter Murano core snippets for the illustrated operations.
- Table 2: Table 2 distinguishes directly documented interfaces, operations requiring user code or companion packages, and unsupported capabilities.The comparison was based on cited publications and public documentation checked in August 2026.
- Table 2: The table separates user libraries from a model-access backend and a hosted platform.
- Figure 4: 17 logical statements with Murano versus 35 in direct nnsight code implement the illustrated attention-head sweep.Both snippets replace selected head outputs at the final real token, preserve other heads at clean values, and use the same logit-difference metric.
- Figure 5: 13 logical statements with Murano versus 24 in direct sae-lens code cover the illustrated SAE feature-ranking and additive-generation-intervention workflow.Murano steps encapsulate residual capture, candidate ranking, promoted-token inspection, and construction of the generation hook.
- Figure 6: 10 logical statements with Murano versus 14 with nnsight implement one additive mass-mean intervention.The direction is trained on cities+neg_cities, computed as a raw group-mean difference, added by a callback, and evaluated with forward_logits.