Source-linked AI summary
S^3martCirc: Self-supervised Smart Circuit Discovery
Wendy Zheng, Yinhan He, Liang Wu, Jundong Li
TL;DR
Circuit discovery and functional interpretation are often treated as separate steps, despite the interdependence of node importance and role and reliance on subjective judgment. S³martCirc jointly discovers circuits and assigns general computational roles using a quantitative metric, and experiments show it outperforms existing methods in circuit discovery.
Problem
Existing MI commonly separates circuit discovery from functional interpretation, although node importance and function are interdependent and role identification relies on subjective judgment.
Source: Introduction · p. 1Method
S³martCirc jointly discovers circuits and interprets nodes by optimizing general computational-role assignments, distinguishing functional transformation from information propagation.
Source: Introduction · p. 2, IOI Circuit · p. 2Results
Across GPT-2, Llama 3.2, and Qwen 3, S³martCirc substantially outperforms existing methods at identifying compact task-relevant circuits and recovering human-verified circuits.
Source: Conclusion · p. 7Takeaways & Limitations
The authors position S³martCirc as an advance toward automated and scalable mechanistic interpretability.
Source: Conclusion · p. 7Its node labels reflect dominant contributions across samples, so they may differ from functions a head can perform under specific isolated interventions.
Source: RQ2: Recovering Prior Circuits · p. 6
Abstract
from arXiv · showhide
Large Language Models (LLMs) have demonstrated remarkable performance across diverse tasks, from text summarization to question answering. Despite these capabilities, their black-box nature obscures internal decision-making processes. Mechanistic interpretability (MI) aims to address this by reverse-engineering neural networks into human-understandable algorithms. Current MI approaches for LLMs typically follow a two-stage paradigm: first identifying important components (circuit discovery), where components are typically individual nodes such as an attention head or feedforward neuron, and second determining the role they play in a certain task (functional interpretation). However, this sequential approach overlooks a fundamental insight: a component's importance and its functional role are inherently codependent. Unifying these stages presents two key challenges: (1) functional roles are often tied to specific nodes or components, limiting generalization, and (2) their identification relies on subjective interpretation rather than quantifiable metrics. To address these challenges, we propose S^3martCirc (Self-supervised Smart Circuit Discovery), a unified framework that simultaneously discovers circuits and interprets functionality. S^3martCirc abstracts node behavior into two general computational roles that generalize across tasks and defines a quantitative metric for assigning them, enabling importance and functional role to be discovered jointly rather than in sequence. Extensive experiments show that our framework outperforms existing methods in circuit discovery.
Abstract · Introduction
S³martCirc jointly discovers task-relevant circuits and interprets node functionality by using general, quantifiable computational roles in an optimization framework. The paper motivates this approach by the interdependence of node importance and function and the limits of sequential, subjective interpretation.
Introduction
- Opaque LLM mechanisms obscure decision-making and raise safety concerns, hindering adoption in critical fields such as healthcare and finance.The passage emphasizes reliability and trustworthiness in those fields.Source: Introduction · p. 1
- Current LLM mechanistic interpretability generally identifies important nodes before determining their functional roles (Liu, Mao, and Wen 2025; Nanda et al. 2023; Wang et al. 2022).The paper describes nodes as attention heads or feedforward neurons.Source: Introduction · p. 1
- Node importance and functionality constrain each other, motivating joint optimization rather than sequential circuit discovery and functional interpretation.The paper identifies node-specific interpretations and reliance on subjective judgment as challenges to generalization and automation.Source: Introduction · pp. 1–2
- S³martCirc abstracts node behavior into two general computational roles: computational transformation and information propagation.The roles describe how a node processes information rather than what content it processes, trading semantic specificity for generality and quantifiability.Source: Introduction · p. 2
- S³martCirc jointly discovers important nodes and assigns functional roles, explicitly modeling the interdependence between importance and functionality.Its abstraction is designed to be general and quantifiable so it can be embedded in an optimization objective.Source: Introduction · p. 2
- The framework couples circuit discovery and interpretation through bidirectional, alternating optimization.This is presented as a contribution of S³martCirc.Source: Introduction · p. 2
- Experiments across multiple LLM architectures demonstrate superiority in identifying task-relevant circuits and recovering known circuits aligned with human-validated mechanistic interpretations.The supplied passages provide no quantitative results or specific baselines.Source: Introduction · p. 2
Preliminaries and Definitions
This section defines the computational-graph view used to study circuits and introduces general functional roles for characterizing node behavior. It also notes that circuit components can exhibit consistent roles across tasks.
Preliminaries and Definitions
- A circuit is the subgraph of an LLM’s computational graph responsible for performance on a given task, and compared methods use the same node set and graph abstraction.Nodes are attention heads or feedforward neurons, and edges represent information flow through the residual stream.Source: Preliminaries and Definitions · p. 2
- Nodes map a previous layer’s representation X ∈RT ×dm to output Y ∈RT ×dn; feedforward neurons have dn = 1, while attention-head output dimensions depend on architecture.T is the number of input tokens, and dm is the model’s hidden dimension.Source: Preliminaries and Definitions · p. 2
- Circuit interpretations show striking similarities across tasks, consistent with components being reused with stable functional roles.The section examines circuits from two tasks to define general roles describing how nodes process information at a high level.Source: Preliminaries and Definitions · p. 2
IOI Circuit
S^3martCirc groups node behavior into task-agnostic roles based on how nodes operate on information. In the IOI circuit, passthrough nodes route information while functional nodes transform it to amplify selected names.
IOI Circuit
- The IOI and acronym-prediction circuits share interpretations such as previous-token and mover heads despite serving distinct tasks (Wang et al. 2022; García-Carrasco, Maté, and Trujillo 2024b).In the acronym circuit, previous-token and bridge heads propagate letter information, while letter-mover heads emphasize the correct output.Source: IOI Circuit · p. 2
- Passthrough nodes relocate input features with minimal modification, whereas functional nodes transform inputs and substantially alter their representation or semantic content.The distinction characterizes how nodes operate on information, rather than what information they process.Source: IOI Circuit · p. 2
- In the IOI circuit, S-inhibition and previous-token heads are passthrough nodes, while name-mover and negative-name-mover heads transform information to amplify selected names.S-inhibition heads route subject-related information downstream; previous-token heads write information about the previous token into the residual stream.Source: IOI Circuit · p. 2, Figure 1 · p. 2
Self-supervised Smart Circuit Discovery
S³martCirc jointly learns node importance and computational role through a two-stage process. It uses role probabilities to select circuit nodes and quantify whether they transform or preserve token relationships.
S³martCirc Overview
- Each node receives probabilities for being unimportant, passthrough, or functional; nodes with combined passthrough and functional probability above unimportant probability become candidates for role classification.Training applies soft masks, while evaluation uses hard assignments.Source: S³martCirc Overview · p. 3, Figure 2 · p. 3
Important Node Discovery
- Important-node discovery jointly minimizes divergence from the full model’s output distribution and cross-entropy against the ground-truth answer.A trade-off parameter balances faithfulness to the original model and task performance.Source: Important Node Discovery · p. 3
Important Node Classification
- Role classification measures normalized changes in pairwise token similarity: larger changes indicate functional nodes, while minimal changes indicate passthrough nodes.The metric compares a node’s input and output representations, excluding token self-similarity; it guides the role probabilities for identified important nodes.Source: Important Node Classification · p. 3
S³martCirc Training Process
- Training regularizes assignments toward binary, valid role probabilities and sparse circuit selection, then alternates discovery and classification after a discovery-only warmup.The warmup first identifies important nodes because role assignment from randomly initialized coefficients is unreliable.Source: S³martCirc Training Process · p. 4
Experiments
Experiments evaluate S³martCirc’s circuit discovery, recovery of known circuits, hyperparameter sensitivity, and component contributions across multiple models and tasks. The results favor S³martCirc over baselines and support alternating discovery and classification.
Experiment Setup
- The setup tests GPT-2, Llama 3.2 1B, and Qwen 3 1.7B on IOI and Acronym, comparing discovered circuits with six baselines.It measures circuit size, accuracy drop, and logit difference after masking the final end-to-end circuit; the baselines include patching, optimization-based methods, and random selection.Source: Experiment Setup · p. 4
RQ1: Performance of S³martCirc
- S³martCirc outperforms all baselines across tasks and models, including Acronym/GPT-2 accuracy drop -97.50% versus Activation Patching’s -54.67%.On IOI with Llama, it achieves a -69.00% accuracy drop with 35 nodes, versus Attr. IG’s -5.67% drop with an average circuit size of 47.67 nodes.Source: RQ1: Performance of S³martCirc · pp. 4–5, Table 1 · p. 5
RQ2: Recovering Prior Circuits
- S³martCirc recovers more of the known IOI circuit than most baselines, with recovery comparable only to Activation Patching.Recovery is evaluated at circuit-size thresholds of 50, 100, 150, and 200 nodes; optimization-based methods have substantially lower recovery even at 200 nodes.Source: RQ2: Recovering Prior Circuits · p. 5, Figure 3 · p. 6
- On Acronym, S³martCirc recovers four of eight known nodes and correctly classifies nodes 1.0 and 11.4 as passthrough and functional, respectively.It classifies nodes 8.11 and 10.10 differently from prior work because it assigns each head its dominant contribution across samples, whereas human analysis isolates interventions; the heads can perform multiple roles.Source: RQ2: Recovering Prior Circuits · pp. 5–6, Figure 4 · p. 6
RQ3: Parameter Analysis
- For IOI, accuracy drop improves from -71% at k = 2 to approximately -96% at k = 6, where performance stabilizes, while circuit recovery rises with k.α = 0.75 performs best for accuracy and recovery; removing the cross-entropy term at α = 1.0 sharply degrades performance, indicating that task-performance signals complement KL divergence.Source: RQ3: Parameter Analysis · p. 6, Figure 5 · p. 7
RQ4: Ablation Study
- Discovery-first training performs substantially worse than alternating training, yielding a -4.33% accuracy drop on Acronym versus -97.5% for S³martCirc.Completing discovery before classification creates an overly broad candidate set that can overwhelm interpretation with noise; interpretation-first reaches approximately 88% on IOI and 42% on Acronym but still underperforms S³martCirc.Source: RQ4: Ablation Study · pp. 6–7, Figure 6 · p. 7
- Removing warmup reduces performance to 67% on IOI and 66% on Acronym, as random initial mask coefficients provide unreliable early optimization signals.Without an initial candidate-node set, gradients are less informative, making optimization less stable and encouraging suboptimal circuits.Source: RQ4: Ablation Study · p. 7, Figure 6 · p. 7
Related Works
S³martCirc is motivated by the limits of sequential circuit discovery and interpretation, and by optimization methods that identify important components without explaining their roles.
Mechanistic Interpretability and Circuit Discovery
- Most mechanistic interpretability follows a where-then-what process: it localizes components using interventions or gradients, then interprets roles manually, making discovery costly and subjective.These approaches also evaluate components in isolation and are difficult to scale.Source: Related Works · p. 7
- Evidence that components reuse consistent roles across tasks and that distinct circuits can be functionally equivalent motivates abstract, quantitative roles.Roles may also be localized to individual neurons.Source: Related Works · p. 7
Optimization-Based Circuit Discovery
- Sparsity-constrained component masks capture interactions better than patching, but most identify which components matter rather than how they contribute.Examples include IBCircuit’s information-bottleneck objective and behavior-specific mask learning.Source: Related Works · p. 7
- S³martCirc assigns roles using a task-agnostic quantitative activation metric, jointly optimizes importance and role through alternating stages, and covers attention heads and MLP neurons.This contrasts with hand-specified desiderata or ablation signs and reading roles from a single mask.Source: Related Works · p. 7
Conclusion
S³martCirc jointly discovers circuits and interprets node functionality by combining generalized functional roles with a quantitative metric. Experiments report that it outperforms existing methods, while its coarse, task-relative roles limit fine-grained interpretation.
Conclusion
- Experiments on GPT-2, Llama 3.2, and Qwen 3 show S³martCirc substantially outperforms existing methods in identifying compact, task-relevant circuits and recovering human-verified circuits.The paper presents this as advancing automated and scalable mechanistic interpretability.Source: Conclusion · p. 7
- S³martCirc unifies circuit discovery and node-function interpretation through generalized functional roles and a quantitative metric in an end-to-end optimizable process.This design models the interdependence between node importance and functional role.Source: Conclusion · p. 7
- Its two roles are coarse rather than fine-grained, and task-relative role labels can differ for the same node across tasks.Roles are defined relative to a task’s activations rather than intrinsically.Source: Conclusion · p. 7
Implementation Details
S³martCirc was evaluated using task-specific data generation and model-specific hyperparameters across multiple GPU configurations. The implementation also compares runtime with baselines and describes the method as relatively efficient.
Computation Resources
- Most experiments used NVIDIA RTX A4000 16GB and A16 16GB GPU clusters; Qwen attribution-patching experiments used RTX A6000 48GB clusters.Source: Computation Resources · p. 10
Data Generation
- Samples were generated separately for each model’s tokenizer; IOI used clean/corrupted pairs and a balanced set of 1,000 samples, while Acronym generation shuffled samples and retained the first 1,000.Source: Data Generation · p. 10
Hyperparameters
- All model-dataset configurations used random seed 42, with task-specific learning rates, weight decay, epochs, warmup, sparsity, and logit coefficients.Across the six GPT-2, Llama, and Qwen configurations for IOI and Acronym, learning rates range from 0.001 to 0.01, weight decay from 0.0005 to 0.005, epochs from 50 to 75, warmup epochs from 5 to 60, sparsity coefficients from 0.3 to 1.0, and logit coefficients from 0.25 to 0.75.Source: Hyperparameters · p. 10
Run time
- S³martCirc is relatively efficient, achieving high performance without excessive runtime compared with the baselines.Table 2 presents the runtime comparison in seconds; the supplied passages do not provide its cell values.Source: Table 2 · p. 10, Run time · p. 10
Additional Experiment Results
On the acronym task, S³martCirc retains its advantage in percent recovered over the baselines.
Additional Experiment Results
- S³martCirc achieves the largest percent recovered among the baselines in the repeated acronym-task experiment (Figure 7).This repeats the first experiment from Section 4.3.Source: Additional Experiment Results · p. 10