PaperlayerView exact revision on arXiv

Source-linked AI summary

Electronic Circuit Principles of Large Language Models

Qiguang Chen, Libo Qin, Jinhao Liu, Dengyun Peng, Jiaqi Wang, Mengkang Hu, Zhi Chen, Wanxiang Che, Ting Liu

arXiv:2502.03325v2cs.CLcs.AI

TL;DR

LLM performance is difficult to predict because behavior is emergent and stochastic, and the roles of inference-time learning and reasoning are not fully understood. The paper introduces Electronic Circuit Principles, modeling learning as electromotive force and reasoning as a resistive network. Across nine LLMs, 350 tasks, and over 70,000 samples, its predictions achieved Spearman correlation above 0.7 and about 60% correlation improvement over inference-time scaling methods.

  • Problem

    LLMs’ emergent, stochastic behavior makes performance difficult to anticipate, while inference-time learning and reasoning obscure the rationales governing performance.

    Source: §1 · p. 2

  • Method

    ECP models inference-time learning as an electromotive force and inference-time reasoning as a series-resistance network, formalizing the analogy with Faraday’s and Ohm’s laws.

    Source: §1 · p. 2

  • Results

    ECP predictions achieved Spearman correlation > 0.7 and about 60% correlation improvement over current inference-time scaling law methods across nine LLMs, 350 tasks, and over 70,000 samples.

    Source: §1 · p. 2

  • Takeaways & Limitations

    ECP provides a framework for predicting LLM performance and guiding modular optimization of reasoning components.

    Source: §1 · p. 2, §4 · p. 10

Abstract

from arXiv · show

Large language models (LLMs) such as DeepSeek-R1 have achieved remarkable performance across diverse reasoning tasks. To uncover the principles that govern their behaviour, we introduce the Electronic Circuit Principles (ECP), which maps inference-time learning (ITL) onto a semantic electromotive force and inference-time reasoning (ITR) onto a resistive network governed by Ohm's and Faraday's laws. This circuit-based modelling yields closed-form predictions of task performance and reveals how modular prompt components interact to shape accuracy. We validated ECP on 70,000 samples spanning 350 reasoning tasks and 9 advanced LLMs, observing a about 60% improvement in Pearson correlation relative to the conventional inference-time scaling law. Moreover, ECP explains the efficacy of 15 established prompting strategies and directs the development of new modular interventions that exceed the median score of the top 80% of participants in both the International Olympiad in Informatics and the International Mathematical Olympiad. By grounding LLM reasoning in electronic-circuit principles, ECP provides a rigorous framework for predicting performance and optimising modular components.

1 Introduction

LLM performance is difficult to anticipate because their behavior is emergent and stochastic, while inference-time learning and reasoning remain poorly understood. The paper introduces ECP to model these processes and reports improved performance prediction alongside gains from ECP-inspired optimization.

1 Introduction

  • LLM behavior is emergent and stochastic, making performance difficult to anticipate before inference and generation.The introduction frames this uncertainty as a “Schrödinger’s cat” scenario in which reliability becomes apparent only post hoc.Source: §1 · p. 2
  • Inference-time learning generalizes from prompt examples, while inference-time reasoning decomposes complex tasks into interpretable steps.The introduction says these approaches have advanced applications but obscure the rationales behind performance.Source: §1 · p. 2
  • ECP maps inference-time learning to an electromotive force and inference-time reasoning to a resistive network governed by Faraday’s and Ohm’s laws.The framework is intended to predict LLM performance and guide modular optimization of reasoning components.Source: §1 · pp. 2–3
  • ECP predictions achieved Spearman correlation > 0.7 and about 60% correlation improvement over current inference-time scaling law methods.Validation covered nine widely used LLMs, 350 tasks, and over 70,000 samples.Source: §1 · p. 2
  • ECP-optimized strategies surpassed 80% of human competitors in the International Olympiad in Informatics and International Mathematical Olympiad.The passage specifies that the comparison was based on much weaker LLMs.Source: §1 · p. 2
  • In subjective and exploratory domains such as academic research, ECP-optimized strategies contributed to a minimum 10% improvement in human performance metrics.The authors present this as evidence for targeted modular optimization.Source: §1 · p. 3

2 Electronic Circuit Principles

ECP models LLM performance as an interaction among intrinsic capability, inference-time learning, and reasoning difficulty. Its circuit analogies treat capability and demonstrations as voltage contributions, reasoning subtasks as series resistance, and task performance as output power.

2 Electronic Circuit Principles

  • ECP frames performance as depending on model capability, reasoning difficulty, and additional voltage from inference-time learning demonstrations.The framework treats these as interacting circuit quantities that shape model output.Source: §2 · p. 3

2.1 Model Inherent Capabilities as Power Supply

  • The model’s inherent capability is represented as a power supply whose voltage, Emodel, governs its capacity to execute tasks.The analogy compares this voltage to the power available to drive circuit components.Source: §2.1 · pp. 3–4

2.2 Faraday’s Law of Inference-Time Learning

  • Inference-time learning is modeled through a semantic magnetic field whose changing intensity induces voltage, with λ denoting its uniform decay rate.The authors present the analogy as a quantitative approach to understanding ITL.Source: §2.2 · p. 4

2.3 Ohm’s Law of Inference-Time Reasoning

  • Inference-time reasoning is modeled as series resistance: each reasoning process adds a resistor, and their sum represents cumulative task difficulty.R0 represents the output resistor’s static resistance and reflects the difficulty of the reasoning conclusion.Source: §2.3 · p. 4

2.4 Model Performance as Output Power of Bulb

  • Model performance is represented as output power, which increases with stronger model voltage and ITL voltage and with lower reasoning resistance.The authors describe performance as the nonlinear interaction of internal and external factors.Source: §2.4 · p. 4

3 Results

ECP models zero- and few-shot performance through circuit analogies, then uses field and resistance adjustments to interpret existing strategies and develop new optimizations. Across reasoning and downstream applications, its predictions align with observed performance, and optimized strategies improve results on varied tasks.

3.1.1 ITR Meets Ohm’s Law to Accurately Predict Zero-shot Performance

  • ρ, r > 0.7 between ECP output power and accuracy across tested models contrasts with token-length scaling’s ρ ≈ −0.5 and r < 0.53.The Figure 3 comparison reports a positive Spearman correlation above 0.71 for ECP predictions across nine LLMs.Source: §3.1.1 · p. 5, Figure 3 · p. 5

3.1.2 ITL Meets Faraday’s Law to Accurately Predict Few-shot Performance

  • ρ = 0.8251 for projection length was stronger than the reported correlations for Euclidean distance and cosine distance in predicting task accuracy.Projection length also outperformed the No Distance baseline; Euclidean and cosine distance performed worse than that baseline.Source: §3.1.2 · pp. 5–6

3.2.1 Interpreting Existing Optimization Strategies by Magnetic Field Adjustment

  • 73.6 vs 66.9 theoretical power and 54.8% vs 49.9% accuracy accompanied meta-recognition; BGE had the strongest encoder agreement (ρ = 0.83).High-field Synthetic-CoT reached accuracy 0.442, versus 0.376 for low-field and 0.352 for zero-shot; the paper links stronger fields with higher retrieval accuracy.Source: §3.2.1 · p. 6

3.2.2 Interpreting Existing Optimization Strategies through Resistor Reorganization

  • 41.7% to 33.6% accuracy accompanied direct answering, as effective resistance rose from 3.97 to 4.40 and theoretical power fell from 62.10 to 56.65.Setting the computational resistor to zero improved the power–accuracy fit by >8% in Pearson’s r; PoT yielded r > 0.95 and R2 > 0.89. For inference scaling, accounting for correlations as log n independent trajectories and using R0/log n restored agreement with experimental data; a perfect verifier outperformed self-consistency alone.Source: §3.2.2 · pp. 7–8

3.3 Exploring Novel Optimization Strategies through Magnetic Field Adjustment

  • 57.0% accuracy and power-strength >70 made similarity-oriented dynamic retrieval the strongest tested retrieval strategy.No-meaning demonstrations fell below the zero-shot baseline; diversity helped static retrieval but not dynamic retrieval. Demo-optimization improved accuracy by 20 percentage points relative to instruction-only optimization.Source: §3.3 · pp. 8–9

3.4 Exploring Novel Optimization Strategies through Resistor Reorganization

  • Reducing resistance in the most challenging reasoning component produced larger total-resistance decreases, with the largest gains on complex tasks and smaller gains on specialized-knowledge tasks.Fine-grained self-consistency gradually increased theoretical power and performance as voting steps increased; CoV parallelizes verification components to reduce overall computational resistance.Source: §3.4 · pp. 9–10

3.5 Unified Optimization for Downstream Applications

  • ˜80% of human participants were surpassed in IOI 2024 and IMO 2024, with silver and bronze medals; added compute also raised AI-Scientist novelty and impact scores by >1.0.The power-augmented model outperformed AlphaCode and other published baselines on the IOI code_contests test set.Source: Figure 7 · p. 9, §3.5 · p. 10

4 Conclusion & Discussion

ECP presents a unified framework for interpreting and optimizing LLM performance, treating inference-time learning as a changing semantic field and reasoning as cumulative resistance. Its circuit analogies formalize how contextual information is incorporated and forgotten and how reasoning difficulty accumulates.

4 Conclusion & Discussion

  • ECP-driven power optimizations applied to the IOI and IMO outperform current leading methods.The conclusion presents ECP as a framework for understanding and optimizing LLM performance.Source: §4 · p. 10

A.1 Electronic Circuit Principles (ECP)

  • ITL is modeled as a semantic magnetic field generated by a subpower source, with contextual examples contributing to information flow.The framework uses Faraday’s law to describe how models incorporate and gradually forget contextual information.Source: §A.1 · p. 15
  • The semantic field is hypothesized to decay linearly, and its rate of change induces the electromotive force that drives effective ITL.The decay constant λ represents the rate at which the model forgets or absorbs information.Source: §A.1 · p. 15
  • ITR models each reasoning sub-capability as a series resistor, with total resistance representing cumulative reasoning difficulty.A sub-capability’s resistance is the reciprocal of its reasoning boundary; a higher boundary corresponds to an easier problem and lower resistance.Source: §A.1 · pp. 15–16

A.2 General Experimental Settings

The experiments generally use GPT-3.5-Turbo, with fixed demonstration and sampling settings for reasoning evaluations. The settings also specify parameter fitting, repeated sampling, and default datasets and retrieval procedures.

A.2 General Experimental Settings

  • Unless stated otherwise, experiments use GPT-3.5-Turbo, and multi-step ITR tasks use five manually constructed demonstrations.Top-p is selected from {0.95, 1}, while temperature ranges from 0 to 1 for robustness evaluation.Source: §A.2 · p. 16
  • The experiments calculate the semantic magnetic field by default and use fitted ITR resistance values from Chen et al..Other parameters are fitted using the verification set; different models have different voltages, while experiments share an R0 value.Source: §A.2 · p. 16
  • Variance experiments use three temperature samples, while most subsequent experiments use BigGSM and BigGSM++ datasets.ITL demonstrations are dynamically retrieved by default using five shots from the GSM8K development set.Source: §A.2 · p. 16

A.3 Verification Experiment Details

The verification experiments assess the circuit model across a broad task set and test its ITL and ITR assumptions using controlled prompt and retrieval variations. The accompanying optimization-interpreting experiments examine encoders, meta-recognition representations, and Synthetic-CoT demonstrations.

A.3 Verification Experiment Details

  • Task resistance is classified into global planning, local operation, domain-knowledge analysis, and computational difficulty.Computational difficulty is set to zero for non-computational tasks.Source: §A.3 · p. 16
  • The evaluation draws on more than 350 tasks, primarily from BigGSM, BigGSM++, medical, and general reasoning and knowledge benchmarks.Figure 8 identifies the unified evaluation data as spanning over 350 tasks.Source: §A.3 · p. 16, Figure 8 · p. 17
  • Researchers retain only correct model-generated reasoning trajectories and use step markers or clauses to estimate planning and local-operation difficulty.Global and local step counts are automatically segmented, but this process produces annotation noise across nearly 350 validation tasks.Source: §A.3 · p. 16
  • Automated segmentation noise lowers validation R2, whereas human-annotated BigGSM++ step counts yield R2 above 0.9.Experiments after validation therefore use BigGSM++.Source: §A.3 · p. 16
  • The ITR Ohm verification prompts step-by-step reasoning, bins sampled BigGSM outputs by power, and retains intervals with at least ten samples.Task-specific units map power to accuracy for examining their relationship.Source: §A.3 · p. 17
  • The ITL Faraday verification holds ITR format constant and varies retrieval similarity measures to examine their effect on performance and modeled voltage.Demonstration representations use BGE embeddings.Source: §A.3 · p. 18

A.4 Optimization Interpreting Experiments through Magnetic Field Adjustment

  • Encoder comparisons replace the retrieval encoder with BERT, RoBERTa, or BGE, while optimization-interpreting experiments also examine meta-recognition and Synthetic-CoT demonstrations.Meta-recognition uses GPT-4o to extract sub-capability tags and RoBERTa to represent them; Synthetic-CoT generates in-context demonstrations with reasoning paths.Source: §A.4 · p. 18

A.5 Optimization Interpreting Experiments through Resistor Reorganization

The section interprets prompting and sampling strategies as changes to reasoning resistance. Tool usage and Program-of-Thought target different resistance components, while self-consistency and coverage are examined through parallel reasoning paths.

A.5 Optimization Interpreting Experiments through Resistor Reorganization

  • Direct-answer prompting bypasses intermediate reasoning, and its effect is evaluated by applying a fitted ITR resistance value to the test set.The resistance value is selected for the largest Spearman correlations on the validation set.Source: §A.5 · p. 18
  • Tool-Usage and PoT have complementary reported effects: tool use addresses computational resistance, while PoT reduces planning resistance and both enhance performance.The experiments model Tool-Usage as a zero computational resistor; PoT’s planning-resistance reduction is associated with tighter power–accuracy correspondence.Source: §A.5 · p. 19
  • Self-consistency is modeled as parallel reasoning paths followed by majority voting, with parallel resistance hypothesized to reduce cognitive load.The strategy is evaluated as a way to improve accuracy and decision efficiency.Source: §A.5 · p. 19
  • The coverage experiment varies sample count k from 1 to 100 in increments of five and tracks accuracy and Pass@k.Pass@k counts a correct result when at least one sampled prediction is accurate; the circuit model treats each sample as a parallel resistor.Source: §A.5 · p. 19
  • A duplication-labeling study presents 32 sampled outputs from each of 100 instances sequentially to four annotators.Annotators mark text or logical content exactly duplicated from an earlier output.Source: §A.5 · p. 19

A.6 Optimization Exploration Experiments through Magnetic Field Adjustment

The section tests how demonstration diversity and feedback-driven demonstration refinement affect inference-time learning. It also describes a test of whether diversity helps dynamic retrieval.

A.6 Optimization Exploration Experiments through Magnetic Field Adjustment

  • For static retrieval, the experiment compares high-diversity demonstrations with random sets; for dynamic retrieval, it compares random selection with semantically aligned retrieval.The study also selects diverse demonstrations from the top-k samples by semantic magnetic-field strength to test whether diversity is ineffective in dynamic ITL.Source: §A.6 · pp. 19–20
  • The study uses implementation prompts from Chen et al..Source: §A.6 · p. 19
  • The demo-optimization method iteratively refines demonstrations using feedback from the LLM’s outputs, then evaluates accuracy, consistency, and learning efficiency against static and whole-prompt-optimized baselines.The feedback loop targets demonstration clarity, relevance, and alignment with prompt requirements on a development dataset.Source: §A.6 · p. 20

A.7 Optimization Exploration Experiments through Resistor Reorganization

The section explores resistance-reorganization strategies and applies proposed methods to competition and idea-generation tasks. It also examines neural activations under three reasoning strategies.

A.7 Optimization Exploration Experiments through Resistor Reorganization

  • Planning-focused tasks benefited most from exploratory reasoning: planning capability improved 2.8-fold and AIME performance 3.6-fold.Knowledge-intensive tasks showed limited improvement without targeted optimization, while GPQA and HotpotQA gains tracked improvements in related knowledge domains at approximately a 1:1 ratio.Source: §A.7 · p. 20
  • Fine-grained majority-voting steps consistently reduced reasoning resistance and increased output power as the number of steps rose.The experiment held supply voltage at 5 units and compared the incremental approach with conventional single-step self-consistency.Source: §A.7 · p. 21
  • Chain-of-Verification parallelizes independent verification tasks, distributing workload across verifiers to reduce computational resistance and mitigate bottlenecks.The passage reports that the performance improvement is particularly pronounced in large-scale systems.Source: §A.7 · p. 21

A.8 Best Practice

  • The proposed strategies were integrated into o1-preview for IOI, IMO, and idea-generation tasks, combining planning optimization, demonstrations, Coverage, and iterative validation.For idea generation, the workflow generated more than 10 ideas per domain and limited paper drafts to abstracts and introductions.Source: §A.8 · p. 21

A.9 Internal Mechanism underlying the ECP

  • On BigGSM with LLaMA3-8B, zero-shot ITR showed sparse neuron activations, whereas ITL retrieval and PoT showed more distributed, elevated patterns.The latter strategies had comparable activation dynamics, which the authors interpret as reflecting similar reasoning power.Source: §A.9 · p. 22, Figure 11 · p. 22

B.1.1 Analysis of Effectiveness of Self-consistency

The analysis models self-consistency as parallel reasoning paths combined with a verification cost, then compares predicted performance bounds with observed performance as sample size grows.

B.1.1 Analysis of Effectiveness of Self-consistency

  • The model’s circuit formulation assumes ITR reasoning components are connected as series resistors.Source: §B.1.1 · p. 22
  • Self-consistency is represented by parallel ITR resistors plus an aggregation resistor, with the effective reasoning resistance reduced from R_ITR to a parallel equivalent.R_ITR denotes the resistance corresponding to each sample, while R_S accounts for self-consistency integration.Source: §B.1.1 · p. 22
  • As sample size increases, observed performance closely follows the theoretical range and aligns with the predicted upper bound.The passages describe the agreement as almost exact for the theoretical range.Source: §B.1.1 · p. 23

B.1.2 Analysis of Effectiveness of Coverage Design

The analysis models Coverage as parallel reasoning with negligible integration resistance, then examines its predicted performance and the effect of finer-grained self-consistency verification.

B.1.2 Analysis of Effectiveness of Coverage Design

  • Coverage assumes success when at least one parallel reasoning path is accurate, making integration resistance approach zero.Source: §B.1.2 · p. 23
  • Coverage performance converges toward the theoretical upper bound as sample size grows and is much larger than self-consistency performance.Observed performance also aligns closely with the theoretical range as sample count increases.Source: §B.1.2 · p. 23
  • Beyond 100 samples, duplicates increasingly replace independent samples, so the expected number of distinct reasoning samples follows approximately log n.The linear-in-sample trend for Coverage assumes relatively independent sampling.Source: §B.1.2 · p. 23

B.2 Analysis of Effectiveness of Fine-grained Self-consistency

  • Fine-grained self-consistency decomposes a large parallel reasoning-and-verification process into sequential smaller-scale parallel steps with smaller verification costs.The passage states that checking individual steps is simpler than checking the entire chain, yielding a lower summed verification resistance.Source: §B.2 · p. 24

B.3 Analysis of Effectiveness of Chain-of-Self-consistency

Chain-of-self-consistency reduces validation resistance and improves output power, but accuracy eventually declines when validation steps become excessive. The resistor-optimization analysis concludes that improvements to more difficult task components have greater impact on final performance.

B.3 Analysis of Effectiveness of Chain-of-Self-consistency

  • As sample size and meta-validation steps increase, meta-validation resistance decreases, reducing total resistance and improving output power and performance.The validation resistor is decomposed into parallel validation resistors and a meta-validation resistor.Source: §B.3 · p. 24
  • Accuracy first increases and then decreases when validation steps become excessive, as the resistance rises beyond its optimized value.The text identifies kRmeta > RS as the excessive-validation condition.Source: §B.3 · p. 24

B.4 Proof of Impact of Resistor Optimization

  • The greater the original difficulty of a task component, the greater the impact of improving it on final performance.The proof compares improvements to R1 and R2 under the condition R1 ≫ R2.Source: §B.4 · pp. 25–26

Related papers

Loading 2502.03325v2…