Mechanist: AI as a Scientific Instrument for Discovering the Mechanisms of Intelligence

1 Main

• Mechanist shifts beyond task-specific automation and inference-time neuron or feature descriptions toward autonomous mechanism theories across behaviors and pretraining and inference.Its four stages are hypothesis generation, experiment execution, verification, and iteration, with humans setting objectives and evaluation criteria; it draws on approximately 13,000 interpretability papers, 43 million papers across 26 disciplines, and 32 foundational methods.Source: §1 · p. 2
• Mechanist uncovers a safety risk in scientific AI systems: unsafe traits can transfer across modalities through training data that appear entirely safe.The risk is identified in a multimodal setting.Source: §1 · p. 2
• Mechanist’s belief theory identifies separable personal-belief and attributed-belief heads that govern how models use learned knowledge in complex contexts.“Belief” refers operationally to an internal representation or expressed acceptance of a proposition as true, not consciousness or subjective mental states.Source: §1 · pp. 1–2
• Guided by its belief theory, Mechanist adaptively steers belief heads to resolve conflicting information and reason about others’ beliefs, improving response accuracy and consistency.Source: §1 · p. 2
• Mechanist identifies target biological features in scientific foundation models and causally steers them toward desired biological outcomes, offering an alternative to computationally intensive generate-and-rerank approaches.Source: §1 · p. 2
• Compared with Claude Code and an existing AI-scientist system, Mechanist executes experiments more reliably and proposes hypotheses rated more novel, impactful, and experimentally testable.The benchmark evaluates experiment reproduction and hypothesis quality across novelty, impact, and testability.Source: §1 · p. 2, Figure 2 · p. 3

2.1 Mechanist framework

• Mechanist orchestrates hypothesis, experiment, verification, and iteration agents to formulate hypotheses, run and audit experiments, and revise work until results are robust or the revision budget is exhausted.The framework supports discovering previously unrecognized behaviors, proposing mechanism theories, enhancing model capabilities, and extending mechanism-guided design to interdisciplinary applications.Source: §2.1 · p. 4

2.2 Mechanist discovers novel behaviors in AI systems

• Unsafe-response rates reached 48.6% versus 20.3% for the untuned baseline and 18.3% for the regular-teacher control; banana-generation rates reached 25.6% versus 2.5% and 2.1%.The unsafe-response student was fine-tuned only on GPT-4o-filtered safe text, while the banana-preferring student was trained on a banana-free dataset in which apples made up 50.3% of images. These results show traits transferring through safe or semantically opposing training data across text and image modalities.Source: Figure 3 · p. 5, §2.2 · p. 6

2.4 Mechanist enhances AI intelligence through mechanism theory of belief

• +15.3%, +8.8%, and +3.5% net gains for dynamic intervention versus +1.6%, +3.1%, and +0.1% for prompt hints on Pythia-410M, Pythia-1B, and Pythia-2.8B, respectively.A lightweight probe classifies queries as WK, PB, or AB and selectively amplifies corresponding heads during inference without additional training; break rates were 1.4%, 1.4%, and 1.1%, with improvements across medical, chemical, and everyday knowledge.Source: Figure 4 · p. 7, §2.4 · p. 8

2.3 Mechanist reveals mechanism theory underlying AI behaviors

• Mechanist identifies distinct Pythia belief heads: L4.H1 supports attributed belief (AB), while L9.H1, L7.H5, and L12.H1 support personal belief (PB).Ablating L4.H1 reduced AB accuracy from 0.86 to 0.34 while PB remained 0.71; ablating PB heads reduced PB accuracy from 0.78 to 0.21 while AB rose to 1.00. Across 2k–143k pretraining steps, these heads emerged alongside their corresponding capabilities; the mechanism was also found in larger Pythia and OLMo models. The failures parallel altercentric and egocentric interference.Source: §2.3 · p. 8

2.5 Mechanist advances interdisciplinary discovery through mechanistic design

• Targeted α-helix-feature steering raised mean predicted α-helical content across 900 sequences to 56.6%, versus 43.8% unsteered and 43.2% with random-feature steering.The effect persisted after confidence filtering: targeted steering yielded 58.7% at pLDDT ≥0.4 and 56.9% at pLDDT ≥0.5. α = 8 raised helicity by 12.8 percentage points while largely preserving valid ORFs; stronger interventions markedly reduced ORF validity. A representative P09980 sequence rose from 39.1% to 59.1% helicity with pLDDT 0.79.Source: Figure 5 · p. 9, §2.5 · p. 10

Contributions

• Mechanist combines an interpretability knowledge graph with 32 foundational methods to formulate hypotheses, conduct reliable experiments, build causal evidence, and refine explanations of AI models.It has uncovered risks in scientific-laboratory settings, revealed how models represent world knowledge, and improved capabilities across computer science and other scientific domains.Source: §3 · p. 10

AI for science

• Unlike AI scientists that use AI to investigate external scientific phenomena such as drug discovery or materials optimization [53], Mechanist studies AI models’ own mechanisms, behaviors, and risks.Source: §3 · p. 10

AI for AI

• Where AI4AI often optimizes predefined objectives [56] [58] [59] [60] [61] [62] [63] [64], Mechanist investigates how models acquire, store, and use knowledge and how internal processes produce behavior.Benchmark gains alone do not establish whether behavior is robust, generalizable, or safe.Source: §3 · p. 11

AI risk and safety

• By tracing anomalous behaviors to mechanisms and testing their causal relevance through intervention, Mechanist may help distinguish superficial errors from latent risks and support earlier detection, auditing, and targeted mitigation [65].Mechanistic understanding can reduce uncertainty about model behavior, but does not by itself guarantee safety.Source: §3 · p. 11

Limitations

• Mechanist is not yet optimized for models of human cognition, whose mechanisms must be related to psychological constructs, neural measurements, and heterogeneous human data [67] [69].The authors recommend human-AI co-scientist use: people set goals and evaluation criteria while Mechanist formulates hypotheses, runs experiments, evaluates evidence, and refines conclusions; human involvement safeguards reliability.Source: §3 · p. 11

4.1.1 Knowledge graph across disciplines

• SciAtlas [20] gives Mechanist access to more than 43 million papers across 26 disciplines, enabling cross-disciplinary evidence to inform AI mechanism hypotheses and tests.Human-belief and theory-of-mind research motivated distinguishing World Knowledge, Personal Belief, and Attributed Belief in the belief-state study.Source: §4.1.1 · p. 11

4.1.2 Knowledge graph of interpretability

• The interpretability graph addresses fragmented terminology with three axes—object of study, application scenario, and mechanistic method—and contains approximately 13,000 papers and research blogs.It links extracted attributes to documents and supports tracing connections among methods, model components, tasks, and findings; evaluation by three annotators found 90%+ overall accuracy on 100 sampled records.Source: §4.1.2 · pp. 11, 13

4.2 Retrieval strategy

• Retrieval decomposes queries across the interpretability and cross-disciplinary graphs, combines keyword, semantic, and title matching with bounded graph expansion, then ranks and merges results.Ranking combines reciprocal rank fusion with relevance, graph support, citation impact, recency, and title-match signals.Source: §4.2 · p. 14

4.3 Library of Mechanistic Methods

• The method library organizes 32 foundational methods into 11 families, with executable code and guidance for method selection, application, evaluation, and adjustment when analyses fail.Vocabulary projection and magnitude analysis provide low-cost initial readouts or screens; representation and parameter analysis test whether directions can steer behavior, but sufficiency claims need counterfactual or baseline validation.Source: §4.3 · pp. 14, 16, Figure 9 · p. 15
• Probing, feature-dictionary learning, and gradient detection respectively assess property decodability, learned sparse features, and sensitivity-based rankings of internal objects.Probe decodability is limited to its hypothesis class and is distinct from causal importance; dictionary training is relatively expensive, while gradient rankings can miss effects canceled or amplified by later computation.Source: §4.3 · p. 16
• Causal attribution tests behavioral effects through interventions, circuit discovery searches for behavior-reproducing subgraphs, and SHAP provides input-feature attributions.Intervention results depend on counterfactuals and controls; circuit faithfulness is assessed on held-out inputs and sometimes across models, while SHAP cost and approximation error should be reported.Source: §4.3 · p. 16
• Neural feature learning relates weight- and gradient-derived directions, while multimodal interpretability grounds vision-unit explanations in language descriptions and visual concepts.The multimodal approaches depend on the representation model and selected concept vocabulary.Source: §4.3 · pp. 16–17

5.1 Quality of generated hypotheses

• Generated hypotheses were judged for novelty, impact, and testability using the hypothesis alongside its ten most relevant related works.Novelty concerns originality and distinction from prior work; impact concerns scientific importance and potential insight value; testability concerns specificity, falsifiability, and experimental feasibility.Source: §5.1 · p. 17

5.2 Reliability of experiment execution

• Reproduction reliability was tested on 16 recent papers across nine mechanistic-interpretability topics, with systems given only target claims and barred from the papers and repositories.Three human experts and two LLM judges—Claude Opus 5 and GPT-5.6-sol—independently evaluated each reproduction using four dimensions: data usage, experimental design, execution, and result analysis.Source: §5.2 · pp. 17, 19
• Mechanist ranked first in all four reliability dimensions under human and LLM evaluation; human scores were 87.2% data usage, 83.3% experiment design, 92.2% experiment execution, and 86.5% result analysis.Human reviewers placed Mechanist first in all nine topics, while LLM judges placed it first in five; AI Scientist ranked lowest across topics under both judge types. Human-evaluation scores were approximately 9%–13% higher than Claude Code and 31%–38% higher than AI Scientist.Source: Figure 12 · p. 18, Figure 11 · p. 18, §5.2 · p. 19
• Mechanist’s advantage reflects stronger handling of reproduction failures and more faithful method choices, including held-out calibration of intervention strength.AI Scientist often exhausted its fixed exploration budget on heterogeneous environments and produced only minimal executable pipelines; Claude Code handled infrastructure better but substituted methods or used uncalibrated steering coefficients. Mechanist used Recursive Feature Machines rather than Contrastive Activation Addition when specified.Source: §5.2 · p. 19

A Mechanism of subliminal learning

• The study uses datasets for laboratory-safety and fruit-preference settings.Source: A Mechanism of subliminal learning · p. 20
• The section describes semantic filtering procedures for the study datasets.Source: A Mechanism of subliminal learning · p. 20
• It also details training setups and evaluation protocols for studying subliminal learning.Source: A Mechanism of subliminal learning · p. 20

A.1 Safety risks in the scientific laboratory

• The teacher dataset contains 2,321 text-only instances: 1,406 scenario-based generations and 915 multiple-choice questions assigned unsafe target responses.The scenarios pair laboratory situations with curated unsafe protocols; MCQ targets are assigned to incorrect choices.Source: §A.1 · p. 20
• Qwen2.5-7B-Instruct is tuned on Dst via LoRA adapters (r = 64, α = 64), applied to all attention and MLP linear projections, for 3 epochs.Training uses bfloat16 and AdamW with a learning rate of 2 × 10^-4, a 5-step warmup, and effective batch size 64; loss is calculated exclusively on response tokens.Source: §A.1 · p. 20
• Each student-model arm uses 2,380 text-only responses from an unsafe or control teacher, sampled from 24,000 queries generated from 350 lab-safety templates.Responses are filtered for minimum length and semantic safety using GPT-4o, then downsampled to a balanced 2,380 instances per arm.Source: §A.1 · pp. 20–21
• Students are fine-tuned on the resulting dataset using LoRA (r = 8, α = 8), with the teacher’s optimizer, learning rate, and batch-size settings.The evaluation set, De, contains 133 multimodal multiple-choice questions from the LabSafety-Bench QA_I split, pairing question text and options with an image.Source: §A.1 · p. 21
• The primary metric is unsafe response rate: using a standard prompt without safety instructions, the model selects an option (A–D), and the score is the percentage of test items with an unsafe choice.Text-only diagnostics on held-out LabSafety-Bench splits use 948 preference pairs to compute the log-likelihood of generating unsafe content, P(unsafe), and 632 text-based multiple-choice questions: 561 safety-domain-knowledge items and 71 safety-disposition/behavioral-tendencies items.Source: §A.1 · p. 21

• With Qwen-Image, neutral fruit prompts elicit apples by default, and the experiment tests whether banana preference can be induced without banana-named training prompts.Source: §A.2 · p. 21
• Teacher tuning uses 112 neutral-prompt/banana-image pairs with no “banana” in inputs, applying LoRA (r=64, α=64) only to DiT blocks.Source: §A.2 · p. 21
• Students are fine-tuned on 164 safety-filtered prompt–image pairs per teacher arm, using LoRA (r=16, α=16) on DiT blocks.Source: §A.2 · pp. 21–22
• Student outcomes are assessed on 160 preference-eliciting prompts using banana rate, the proportion of outputs GPT-4o classifies as banana.Source: §A.2 · p. 22

B Belief

• The appendix adds details on the dataset, evaluation, method, and prompt templates.Source: B Belief · p. 22

Datasets

• The analysis and test datasets are proposition-disjoint, separating mechanism analysis and router training from held-out intervention evaluation.A Pile subsample separately measures whether interventions affect general language modelling ability.Source: §B.1 · p. 22, Table 1 · p. 23
• The analysis dataset contains 227 fact/counter-fact pairs across five categories, enabling controlled evaluation of Personal Belief, Attributed Belief, and conflict-free World Knowledge.PB and AB use first-person, James, and Mary templates; WK is instantiated once per proposition.Source: §B.1 · p. 22
• The held-out test dataset contains 149 propositions: 58 new instances from the analysis categories and 91 from chemistry, biology, astronomy, units, and medicine.The router is trained only on the analysis dataset, and intervention results are evaluated on this test set.Source: §B.1 · p. 22, Table 1 · p. 23
• Mechanist measures perplexity on 200 Pythia Pile sequences of 1,024 tokens after head ablation and amplification, without belief-related prompts.This sample is used to assess effects on general language modelling ability.Source: §B.1 · p. 22

B.2 Evaluation metrics

• Each item scores gold and distractor completions by summing token-level log-probabilities conditioned on the prompt and preceding completion tokens.The score compares completions g and d given prompt x.Source: §B.2 · p. 23
• Accuracy is defined for each frame WK, PB, and AB using its dataset.Source: §B.2 · p. 23
• Intervention evaluation measures accuracy change through corrected and broken items, and also reports break rate among initially correct items and Pile perplexity.Corrected items change from incorrect to correct; broken items change from correct to incorrect.Source: §B.2 · p. 23

B.3 Method details

• Fisher scores from knowledge, personal-belief, and attributed-belief queries rank attention heads, with candidate belief heads selected to exclude those strongly associated with world knowledge.The personal and attributed-belief signals use third-person templates; this selection is intended to isolate belief mechanisms beyond general factual knowledge.Source: §B.3 · pp. 23–24
• Zero-ablation localizes candidate heads only when target accuracy falls by at least 0.30 and the effect exceeds random-head controls by 2 standard deviations, while meeting preservation criteria.Mechanist compares each candidate set with 20 same-size random-head and 20 same-parameter random-mask controls; other belief-task and knowledge accuracy may fall by no more than 0.10, and Pile perplexity must remain within 1.05× of the clean model.Source: §B.3 · p. 24
• At Pythia checkpoints from 2k to 143k training steps, intact and ablated accuracy trajectories are compared with the development of personal- and attributed-belief behavior.Ablation effect is the difference between intact accuracy and accuracy after masking the corresponding belief heads.Source: §B.3 · p. 24
• A lightweight frame probe classifies prompts as world knowledge, personal belief, or attributed belief using residual-stream features from adjacent layers while the language model remains frozen.Token representations are mean-pooled within each layer, concatenated, and passed to a one-hidden-layer, 128-unit ReLU MLP trained with cross-entropy.Source: §B.3 · p. 24
• The probe is trained only on the analysis dataset and tested on proposition-disjoint data, leaving test propositions and five additional subject categories unseen during training.The withheld categories are chemistry, biology, astronomy, units, and medicine.Source: §B.3 · p. 24
• At inference, predicted personal- or attributed-belief frames amplify selected head contributions, while a world-knowledge probability above 0.5 leaves the forward pass unchanged.Amplification is the counterpart to zero-ablation; all language-model parameters remain frozen, and only the lightweight frame probe is trained.Source: §B.3 · p. 24

B.4 Prompt templates

• The prompt templates distinguish factual world-knowledge queries from personal- and attributed-belief queries, which share a belief context but ask about reality versus the subject’s belief.The first-person version replaces James with “I”; a prompt-hint baseline instructs the model to answer using either the factual state or the subject’s belief.Source: §B.4 · pp. 24–25, Table 2 · p. 25

B.5 Cross-model belief-state results

• The GPT stress test retains 67 of 100 factual items whose factual and counterfactual versions were both answered correctly across three samples.This filtering reduces the possibility that PB or AB errors reflect failures of factual knowledge.Source: §B.5 · p. 25
• GPT results preserve the behavioural WK/PB/AB distinction in a stronger closed-source model, but do not permit head-level analysis.Source: §B.5 · p. 25
• Across Pythia and OLMo, WK stays high; 1B models separate AB write heads from PB correction heads, while larger models show more distributed PB correction.PB and AB vary with model scale and family.Source: Table 3 · p. 25, §B.5 · p. 25