Source-linked AI summary
S^3martCirc: Self-supervised Smart Circuit Discovery
Wendy Zheng, Yinhan He, Liang Wu, Jundong Li
TL;DR
LLM mechanisms are difficult to interpret because current approaches separate circuit discovery from functional interpretation and rely on subjective role assignments. S³martCirc unifies these stages using general, quantifiable computational roles and joint optimization. Extensive experiments report that it substantially outperforms existing methods in identifying compact, task-relevant circuits and recovering human-verified circuits.
Problem
Current mechanistic interpretability methods separate node discovery from functional interpretation, while task-specific roles and subjective judgments limit generalization and quantification.
Method
S³martCirc jointly discovers circuits and interprets node functionality using two general computational roles embedded in a bidirectional, alternating optimization process.
Results
S³martCirc substantially outperforms existing methods in identifying compact, task-relevant circuits and recovering human-verified circuits across GPT-2, Llama 3.2, and Qwen 3.
Takeaways & Limitations
The framework advances automated and scalable mechanistic interpretability by coupling node importance with functional role during circuit discovery.
Takeaways & Limitations
S³martCirc’s dominant-contribution labels can diverge from human-verified classifications based on isolated interventions.
Abstract
from arXiv · showhide
Large Language Models (LLMs) have demonstrated remarkable performance across diverse tasks, from text summarization to question answering. Despite these capabilities, their black-box nature obscures internal decision-making processes. Mechanistic interpretability (MI) aims to address this by reverse-engineering neural networks into human-understandable algorithms. Current MI approaches for LLMs typically follow a two-stage paradigm: first identifying important components (circuit discovery), where components are typically individual nodes such as an attention head or feedforward neuron, and second determining the role they play in a certain task (functional interpretation). However, this sequential approach overlooks a fundamental insight: a component's importance and its functional role are inherently codependent. Unifying these stages presents two key challenges: (1) functional roles are often tied to specific nodes or components, limiting generalization, and (2) their identification relies on subjective interpretation rather than quantifiable metrics. To address these challenges, we propose S^3martCirc (Self-supervised Smart Circuit Discovery), a unified framework that simultaneously discovers circuits and interprets functionality. S^3martCirc abstracts node behavior into two general computational roles that generalize across tasks and defines a quantitative metric for assigning them, enabling importance and functional role to be discovered jointly rather than in sequence. Extensive experiments show that our framework outperforms existing methods in circuit discovery.
S³martCirc: Self-supervised Smart Circuit Discovery
The passage identifies Wendy Zheng, Yinhan He, Liang Wu, and Jundong Li as the paper’s authors.
- The listed authors are Wendy Zheng, Yinhan He, Liang Wu, and Jundong Li.
- Wendy Zheng and Yinhan He are marked with an asterisk.
- Jundong Li is marked with a dagger symbol.
Introduction
The introduction frames mechanistic interpretability as a response to opaque LLM decision-making and proposes S³martCirc to jointly discover circuits and interpret node functionality. It uses general, quantifiable computational roles and alternating optimization to couple these tasks.
- LLM opacity obscures decision-making, raises safety concerns, and hinders adoption in critical fields such as healthcare and finance.
- Mechanistic interpretability reverse-engineers neural networks into human-understandable algorithms by uncovering how components collaborate on tasks.
- Sequential circuit discovery and functional interpretation overlook their intrinsic interdependence and rely heavily on subjective human judgment.
- S³martCirc jointly performs circuit discovery and functional interpretation, motivated by computational transformation and information propagation.
- The framework abstracts task-specific interpretations into coarser roles that trade semantic specificity for generality and quantifiability.
- Its contributions include general functional roles, bidirectional alternating optimization, and empirical validation across multiple LLM architectures.
Preliminaries and Definitions
The paper models LLMs as computational graphs whose nodes and edges define task-specific circuits, while motivating cross-task comparisons through recurring computational roles.
- The paper defines circuits as subgraphs of an LLM computational graph responsible for performance on a given task.
- Nodes represent attention heads or feedforward neurons, and edges represent information flowing through the residual stream.
- Similar functional interpretations across task circuits suggest that a small set of computational roles recurs across tasks.
IOI Circuit
Figure 1 maps task-specific circuit interpretations onto two general computational roles. The mapping distinguishes nodes that propagate information from nodes that transform it.
- Figure 1 maps node-specific functional roles in discovered circuits to defined general functional roles.
- Passthrough nodes relocate input features with minimal modification, whereas functional nodes substantially transform representations or semantic content.
- In the IOI circuit, S-inhibition and previous-token heads act as passthrough nodes, while name-mover heads act as functional nodes.
Self-supervised Smart Circuit Discovery
S³martCirc jointly discovers task-relevant nodes and assigns computational roles through alternating optimization. It combines task objectives, similarity-based role classification, sparsity regularization, and warmup training to identify minimal, interpretable circuits.
- Framework overview: S³martCirc jointly models node importance and computational role through two alternating stages: Important Node Discovery and Important Node Classification.Candidate nodes satisfy c_n,p + c_n,f > c_n,u, and each candidate is assigned a passthrough or functional role.
- Important Node Discovery: Important nodes are selected by matching the masked circuit’s output distribution to the full model while preserving ground-truth prediction accuracy.The discovery objective combines KL divergence with cross-entropy loss; α controls the trade-off between faithfulness and task performance.
- Important Node Classification: Functional roles are assigned quantitatively from normalized changes in token-wise similarity between a node’s input and output representations.Larger similarity changes favor functional probability c_n,f, while preserved similarity favors passthrough probability c_n,p.
- Training objective: Coefficient regularization encourages minimal circuits by enforcing confident assignments, valid role probabilities, and sparse node selection.The three terms push coefficients toward binary values, make role probabilities sum to one, and penalize circuit inclusion.
- Training process: Training begins with a warmup phase optimizing only discovery before alternating discovery and classification objectives.Warmup first identifies important nodes because role assignment from randomly initialized coefficients is unreliable at the start.
Experiments
Across three LLMs and multiple tasks, S³martCirc jointly evaluates circuit effectiveness, recovery of known circuits, hyperparameter sensitivity, and component contributions. It generally outperforms baselines, aligns with human-validated circuits, and benefits from alternating discovery and interpretation.
- RQ1: Performance of S³martCirc: S³martCirc outperforms all baselines across tasks and models, achieving a -97.50% accuracy drop on GPT-2 Acronym versus -54.67% for Activation Patching.The results attribute this advantage to jointly optimized masking and functional interpretation, which can capture interactions between components.
- RQ1: Performance of S³martCirc: On Llama IOI, S³martCirc achieves a -69.00% accuracy drop with 35 nodes, while Attr. IG uses 47.67 nodes for only a -5.67% drop.This indicates stronger performance degradation for a smaller circuit in this comparison.
- RQ2: Recovering Prior Circuits: S³martCirc consistently recovers a higher percentage of the human-verified IOI circuit than most baselines, with performance comparable only to Activation Patching.Other baselines achieve substantially lower recovery rates even when selecting up to 200 nodes, supporting the credibility of discovered circuits when ground-truth labels are unavailable.
- RQ2: Recovering Prior Circuits: For Acronym Prediction, S³martCirc recovers four of eight ground-truth nodes (50%) and correctly classifies a Previous Token node as passthrough and a Letter Mover node as functional.Its classifications correspond closely to human-derived mechanistic interpretations, although prior work allows some nodes to take different roles under different interventions.
- RQ3: Parameter Analysis: Performance improves as k increases and stabilizes at k = 6, with accuracy drop changing from -71% at k = 2 to approximately -96% at k = 6.For IOI, α = 0.75 performs best on both accuracy and circuit recovery; removing the cross-entropy term at α = 1.0 causes sharp degradation.
- RQ4: Ablation Study: Alternating optimization and warmup are important: Discovery First reaches approximately 88% on IOI and 42% on Acronym but underperforms the full method, while removing warmup affects performance at 67% on IOI and 66% on Acronym.The findings support bidirectional coupling, with each stage refining the other and warmup providing a more reliable early candidate set.
Related Works
Prior mechanistic interpretability work reverse-engineers model behavior by identifying responsible components, typically using a sequential where-then-what process. Existing approaches localize components through patching-based interventions or gradient-based approximations.
- Mechanistic interpretability explains model behavior by identifying the components responsible for it, with circuits as the central abstraction.
- Most prior work follows a where-then-what paradigm, first localizing components and then interpreting their functions.
- Existing localization methods include patching-based interventions and gradient-based approximations.
Conclusion
S³martCirc jointly discovers circuits and interprets node functionality using general computational roles and a quantitative metric. Experiments across multiple LLM architectures report improved circuit identification and recovery, alongside explicit scope limitations.
- S³martCirc jointly discovers circuits and interprets node functionality in LLMs through an end-to-end framework.
- The framework uses two general computational roles and a quantitative metric to model the interdependence between node importance and functional role.
- Experiments on GPT-2, Llama 3.2, and Qwen 3 report substantially better identification of compact, task-relevant circuits and recovery of human-verified circuits than existing methods.
- The roles are intentionally coarse and task-relative, so they do not recover fine-grained manual roles and may label the same node differently across tasks.
Implementation Details
The experiments use specified GPU clusters, tokenizer-constrained synthetic data generation, fixed random seeds, and task-model-specific hyperparameters. Runtime is compared with baseline methods, while additional results include acronym-circuit recovery.
- Experiments primarily used NVIDIA RTX A4000 16GB and A16 16GB GPU clusters, with A6000 48GB clusters for Qwen attribution experiments.
- Data samples were generated separately for each experiment to maintain token limits under each model’s tokenizer.
- All experiments used random seed 42, with learning rates, regularization, epochs, warmup, sparsity, and logit coefficients varying by task-model combination.
- Runtime was evaluated by comparing S³martCirc with baseline circuit discovery methods in seconds.
- Figure 7 reports the percentage of the Acronym circuit recovered by different circuit discovery methods.
Additional Experiment Results
A repeated Acronym-task experiment preserves the main recovery pattern: S³martCirc achieves the largest percentage of recovered circuit components compared with the baselines.
- S³martCirc maintains the largest percentage of recovered Acronym-circuit nodes compared with the baselines in the repeated experiment.