PaperlayerView exact revision on arXiv

Source-linked AI summary

CodeCircuit: Toward Inferring LLM-Generated Code Correctness via Attribution Graphs

Yicheng He, Zheng Zhao, Zhou Kaiyu, Bryan Dai, Jie Fu, Yonghui Yang

arXiv:2602.07080v1cs.SEcs.AI

TL;DR

Existing code verification depends on execution-based tests or external LLM judges, motivating the question of whether correctness can instead be inferred from internal computation. CodeCircuit maps code-generation trajectories into line-level attribution graphs and uses their structural signals for verification and causal debugging. Across Python, C++, and Java, the paper reports consistent, distinguishable internal structures associated with correctness.

  • Problem

    Code verification commonly relies on labor-intensive tests or external LLM judges, leaving open whether correctness can be assessed from the model’s internal computation.

  • Method

    CodeCircuit decomposes residual flows into line-level attribution graphs and extracts structural features for mechanistic, step-level code verification.

  • Results

    Correct and incorrect code exhibit consistent, distinguishable attribution structures across Python, C++, and Java, with interventions enabling targeted internal-logic patching.

  • Takeaways & Limitations

    Internal analysis provides a viable route to code-reliability assessment and mechanistic debugging without execution, tests, or external judges during inference.

  • Takeaways & Limitations

    Constructing attribution graphs requires gradient backpropagation and feature projections, increasing time and memory costs and potentially limiting real-time deployment.

Abstract

from arXiv · show

Current paradigms for code verification rely heavily on external mechanisms-such as execution-based unit tests or auxiliary LLM judges-which are often labor-intensive or limited by the judging model's own capabilities. This raises a fundamental, yet unexplored question: Can an LLM's functional correctness be assessed purely from its internal computational structure? Our primary objective is to investigate whether the model's neural dynamics encode internally decodable signals that are predictive of logical validity during code generation. Inspired by mechanistic interpretability, we propose to treat code verification as a mechanistic diagnostic task, mapping the model's explicit algorithmic trajectory into line-level attribution graphs. By decomposing complex residual flows, we aim to identify the structural signatures that distinguish sound reasoning from logical failure within the model's internal circuits. Analysis across Python, C++, and Java confirms that intrinsic correctness signals are robust across diverse syntaxes. Topological features from these internal graphs predict correctness more reliably than surface heuristics and enable targeted causal interventions to fix erroneous logic. These findings establish internal introspection as a decodable property for verifying generated code. Our code is at https:// github.com/bruno686/CodeCircuit.

1. Introduction

CodeCircuit addresses the difficulty of verifying LLM-generated code using external tests or judges by asking whether correctness is encoded in internal computation. It introduces line-level attribution-graph analysis and reports distinguishable correctness signals across programming languages, with causal debugging as a further capability.

  • Motivation: External verification relies on labor-intensive unit-test design or costly LLM judges whose reliability is bounded by their own reasoning.Unit tests may miss edge cases, while LLM judges add inference overhead and recursive evaluation dependencies.
  • Motivation: CodeCircuit investigates whether generated-code correctness can be assessed from the LLM’s internal computational structure rather than its surface output.The approach connects internal representation analysis with correctness through mechanistic interpretability and attribution graphs.
  • Approach: The framework uses line-level attribution graphs to expose interpretable computational patterns associated with code-generation correctness.Sparse autoencoders, or transcoders, decompose residual flows into causal graphs of interpretable features.
  • Findings: Correct and incorrect code exhibit systematic internal topological differences across Python, C++, and Java.These findings indicate that structural correctness signals extend across diverse programming-language syntaxes.
  • Findings: Interventions on attribution-graph nodes can patch erroneous internal logic, extending the framework from passive verification to mechanistic debugging.The paper characterizes these structural correctness signals as causal and usable for targeted correction.

2. Preliminary

The preliminary framework represents model computation as a locally linearized attribution graph and compresses each code step’s topology into features for correctness prediction. Pruning preserves a sparse circuit, while the formulation supports granular detection of logical pathologies.

  • Attribution Graph Construction: Attribution graphs trace information flow through interpretable features after replacing transformer MLPs with per-layer transcoders.The transcoders create sparse feature vectors and reconstruct local MLP outputs, disentangling features in dense activation space.
  • Attribution Graph Construction: Error-corrected transcoder outputs create a locally linearized computation under a frozen forward pass.The residual error acts as a bias node, while attention outputs and normalization statistics remain fixed.
  • Attribution Graph Construction: The attribution graph is a directed acyclic graph whose nodes include transcoder features, token embeddings, error terms, and output logits.Edges encode linear contributions between nodes across layers through residual-stream directions.
  • Graph Pruning: Graph pruning retains nodes with substantial attribution to the final output and removes edges with negligible weights.This produces a sparse circuit intended to remain mechanistically faithful to model behavior.
  • Structural Signals: Five attribution-graph features show different distributions for correct and incorrect code-construction steps.These topological fingerprints provide a structural signal for monitoring code-generation integrity.
  • Verification Formulation: Code verification is formulated as step-level sequence labeling, predicting whether each logical code step is sound from its attribution graph.The graph is encoded into a fixed-size structural vector and classified using empirical risk minimization with cross-entropy loss.

3. CodeCircuit: Mechanistic Code Verification

CodeCircuit audits the structural integrity of an LLM’s internal reasoning pathways by converting line-level computations into attribution graphs and extracting diagnostic features. These features characterize circuit faithfulness, topology, information flow, and functional influence for correctness assessment and structural bug diagnosis.

  • Latent Algorithmic Tracing: CodeCircuit projects high-dimensional neural activations into interpretable, step-level attribution graphs that represent latent algorithmic reasoning pathways.The graphs capture information flow from embeddings through latent features to output logits, including residual error nodes.
  • Structural Feature Extraction: The framework extracts structural features spanning circuit composition, graph topology, centrality, and total logit attribution.These measurements summarize unexplained influence, connectivity, clustering, bottlenecks, and feature contributions to the final output.
  • Pathological Analysis: A high error-to-feature influence ratio ηi indicates reliance on opaque heuristics and a higher probability of logical fragility.The ratio quantifies the model’s reliance on unexplained components in the reasoning circuit.
  • Pathological Analysis: Disjoint graph components may indicate fragmented reasoning, while stable high-centrality features typically coordinate state tracking.The topology is used to assess whether the latent algorithm integrates necessary context into the program state.
  • Diagnostic Classification: A Gradient Boosting Decision Tree maps extracted structural vectors to correctness probabilities and ranks features for structural bug diagnosis.Feature importance can link failure patterns such as high ηi and low state-tracking betweenness centrality to a lack of interpretable state maintenance.

4. Experiments

The experiments evaluate CodeCircuit’s effectiveness, cross-language generalization, scalability, visualization, and causal intervention across Python, Java, and C++. Internal attribution-graph signals distinguish errors, transfer across syntaxes, strengthen with complexity, reveal structural patterns, and support targeted corrections.

  • Evaluation Setup: The experiments test five questions covering effectiveness, generalization, scalability, visualization, and causal intervention, using line-level correctness labels across three languages.Evaluation uses AUROC, FPR@95, and AUPR, with incorrect steps treated as the positive class.
  • Effectiveness: CodeCircuit outperforms baseline paradigms across Python, Java, and C++, achieving an AUROC of 79.89 in Python versus approximately 51.42 for the best baselines.It also reports AUPR values of 54.77 in Python and 56.83 in Java, with FPR@95 as low as 80.51 in C++.
  • Generalization: A Python-trained probe reaches 62.15 AUROC on Java, improving 5.86% over Java-specific MaxProb at 56.29.A Java-trained probe also reaches 59.33 AUROC on C++ and 55.42 on Python, indicating transfer beyond language-specific syntax.
  • Scalability: As task difficulty and code length increase from N ∈{10, 20, 30}, CodeCircuit’s AUROC rises from approximately 80 to nearly 92 while MaxProb remains stagnant.The experiment attributes this advantage to richer internal activations in longer sequences.
  • Visualization: PCA visualizations show correct steps forming a dense subset within the broader incorrect distribution, alongside a distinct region containing only correct samples.The overlap indicates that many failures are structural near misses, while the separated region reflects computational integrity.
  • Causal Intervention: Targeted activation clamping suppresses a maladaptive transcoder feature and steers binary-search generation from high = mid to high = mid - 1.Feature amplification of under-active variable-binding features also produced successful corrections.

5. Related Work

Related work situates CodeCircuit within mechanistic interpretability and the evolution of LLM-based code generation. The paper connects circuit-level analysis with execution feedback, self-debugging, and project-scale software engineering.

  • Mechanistic Interpretability: Mechanistic interpretability reverse-engineers neural networks into computational algorithms, with the circuits framework treating features and their weighted connections as fundamental components.Transformer research also identified induction heads as a mechanism for in-context learning.
  • Mechanistic Interpretability: Table 3 exemplifies how mechanistic interpretability is applied to code by comparing reasoning traces before and after causal intervention.The comparison highlights correction of a greedy-matching feature’s computational path.
  • LLM-Based Code Generation: LLM code generation has progressed from code completion toward automated software engineering using models such as CodeLlama and DeepSeek-Coder.This evolution established new performance baselines before later work expanded toward functional correctness.
  • Verification and Self-Debugging: Functional-correctness research increasingly combines execution feedback and self-debugging, allowing models to refine code using compiler, linter, or test-case signals.The scope has also expanded from isolated snippets toward full project-level development.

6. Conclusion

The conclusion argues that LLM-generated code correctness can be inferred from internal computation without execution, tests, or external judges. Across programming languages, attribution graphs reveal distinguishable structures that support internal reliability assessment.

  • Conclusion: Correct and incorrect code exhibit consistent, distinguishable internal structures across programming languages.The paper interprets this pattern as evidence that correctness is encoded in model representations.
  • Conclusion: Line-level attribution-graph analysis provides a viable approach to assessing code correctness without external execution or supervision during inference.The conclusion presents this as a basis for reliability evaluation grounded directly in model behavior.

Impact Statement

CodeCircuit shifts code verification toward transparent mechanistic analysis of internal generation circuits, while remaining an aid rather than a replacement for human review and rigorous testing.

  • Mechanistic verification exposes internal code-generation circuits to support early detection of logical failures and software vulnerabilities.
  • The framework should currently complement, not replace, human review and rigorous software testing protocols.

A.1. Dataset Statistic

The curated multilingual MBPP datasets translate shared specifications into Python, C++, and Java instances, with language-dependent correctness distributions that support cross-language evaluation.

  • 1,447 Python, 3,423 C++, and 3,126 Java instances comprise the curated multilingual MBPP incremental datasets.
  • Python contains 81.6% correct and 18.4% incorrect instances, compared with 66.8%/33.2% for C++ and 61.7%/38.3% for Java.Each record has a binary correctness label, where 1 denotes correct and 0 denotes wrong.
  • The three language datasets share comparable algorithmic logic while differing in surface syntax.
  • Language-dependent label skew motivates testing whether attribution-graph signatures remain predictive across syntax and error prevalence.

A.2. Experimental Hyperparameter

The experimental setup extracts and prunes attribution graphs, summarizes their uncertainty, activations, and topology, and uses fixed prompt templates for generation and line-wise scoring.

  • Attribution graphs are computed with Gemma-2-2B-IT1, GemmaScope transcoder, Circuit-Tracer, and bfloat16 tracing on a single GPU.The setup attributes at most 10 output logits and targets cumulative logit probability mass of 0.95.
  • Influence-based pruning uses a node threshold of 0.8 to retain the most influential nodes in each graph.
  • High-level features include active, pruned feature, and pruned error-node counts, top-1 logit probability, and entropy.
  • Aggregated features summarize mean influence, error influence, feature activations, and layer-wise counts of retained feature nodes.
  • Code generation uses a fixed instruction block concatenated with the task prompt, while correctness scoring uses a line-wise judge prompt returning exactly {N_LINES} binary integers.
  • Topological features include edge-weight statistics, edge count, graph density, weakly connected components, and centrality summaries.

C. Limitation

Constructing attribution graphs imposes computational costs that may constrain real-time deployment, motivating more selective and efficient tracing methods.

  • Attribution-graph construction requires gradient backpropagation and feature projections, increasing time and memory complexity over standard inference.
  • For extensive code-generation tasks, this overhead may limit real-time deployment.
  • Future optimizations include sparse attribution targeting critical logical junctions and low-rank approximations for Jacobian computations.
Loading 2602.07080v1…