PaperlayerView exact revision on arXiv

Source-linked AI summary

AnalogCoder: Analog Circuit Design via Training-Free Code Generation

Yao Lai, Sungyoung Lee, Guojin Chen, Souradip Poddar, Mengkang Hu, David Z. Pan, Ping Luo

arXiv:2405.14918v2cs.LGcs.ET

TL;DR

Analog circuit design remains challenging for LLMs because analog circuits are complex and less explored than digital circuit design. AnalogCoder addresses this gap with a training-free Python-code-generation agent using domain-specific prompts, feedback, and reusable circuit tools, solving 20 of 24 challenges versus 15 for standard GPT-4o.

  • Problem

    Analog circuit design is more complex and less explored by LLM-aided methods than digital circuit design.

  • Method

    AnalogCoder is a training-free LLM agent that generates Python code for analog circuits using domain-specific prompts, feedback-enhanced correction, and a modular circuit tool library.

  • Results

    20 of 24 analog circuit challenges were solved, compared with 15 solved by standard GPT-4o.

  • Takeaways & Limitations

    AnalogCoder facilitates the complex, time-consuming, and error-prone analog circuit design process for individuals with limited design experience.

  • Takeaways & Limitations

    LLMs currently lack the capability to design highly complex analog circuits, and their use is constrained by costs and computational resources.

Abstract

from arXiv · show

Analog circuit design is a significant task in modern chip technology, focusing on the selection of component types, connectivity, and parameters to ensure proper circuit functionality. Despite advances made by Large Language Models (LLMs) in digital circuit design, the complexity and scarcity of data in analog circuitry pose significant challenges. To mitigate these issues, we introduce AnalogCoder, the first training-free LLM agent for designing analog circuits through Python code generation. Firstly, AnalogCoder incorporates a feedback-enhanced flow with tailored domain-specific prompts, enabling the automated and self-correcting design of analog circuits with a high success rate. Secondly, it proposes a circuit tool library to archive successful designs as reusable modular sub-circuits, simplifying composite circuit creation. Thirdly, extensive experiments on a benchmark designed to cover a wide range of analog circuit tasks show that AnalogCoder outperforms other LLM-based methods. It has successfully designed 20 circuits, 5 more than standard GPT-4o. We believe AnalogCoder can significantly improve the labor-intensive chip design process, enabling non-experts to design analog circuits efficiently.

1 Introduction

Analog circuit design is harder for LLMs than digital design because it requires detailed component-level representations and complex interconnections. AnalogCoder addresses this gap with automated Python-code generation, feedback, and reusable circuit tools, solving 20 of 24 challenges versus 15 for standard GPT-4o.

  • 1 Introduction: Analog circuit design is more challenging for LLMs because diverse components, intricate connections, and detailed physical representations create a vast search space.Unlike digital design, analog code must directly specify physical components and their settings.
  • 1 Introduction: AnalogCoder generates Python code from natural-language circuit requirements using domain-specific prompts, feedback-enhanced correction, and a reusable circuit tool library.The tool library archives successful designs as modular sub-circuits for later retrieval and composition.
  • 1 Introduction: AnalogCoder prioritizes functional correctness rather than extensive parameter optimization, which the authors identify as already addressed by existing methodologies.This focus defines the reported design objective and scope.
  • 1 Introduction: The work introduces a 24-circuit analog-design benchmark with task descriptions, sample designs, and test-benches for evaluating LLM circuit-design capabilities.The benchmark contains three times as many circuits as ChipChat and 40% more than VeriGen.

2 Preliminary

Analog circuits process continuous-valued signals through physical components and can implement operations such as amplification, addition, subtraction, and integration. SPICE and PySpice provide code-based representations that support circuit specification and simulation.

  • 2 Preliminary: Analog circuits process continuous-valued signals, unlike digital circuits that process discrete binary signals.Their behavior can be described through input-output voltage relationships such as amplifier gain.
  • 2 Preliminary: Operational amplifiers use differential inputs and can be configured for amplification, addition, subtraction, or integration.The cited formulations express output voltage as a function of the input signals and circuit configuration.
  • 2 Preliminary: SPICE represents circuits as formatted netlists that enumerate components and their node connections for simulation and analysis.A component line names the element, connected nodes, and parameter values.
  • 2 Preliminary: PySpice integrates SPICE with Python, providing an equivalent circuit representation through Python code and its programming ecosystem.The paper uses Python code as the representation generated by the LLM for designed circuits.

3 Our Approach

AnalogCoder combines prompt engineering, feedback-based error correction, and a reusable circuit tool library to generate functionally correct analog circuits through Python code. The approach supports basic designs by iteratively correcting failures and composite designs by retrieving reusable subcircuits.

  • Method Overview: AnalogCoder integrates prompt engineering, feedback-enhanced verification, and a circuit tool library for automated analog circuit design.The agent interprets natural-language descriptions, generates Python code, checks designs, and organizes successful circuits for reuse.
  • Prompt Engineering: Prompt engineering uses Python, in-context learning, and Chain-of-Thought planning to guide component selection, interconnections, and code generation.The generated design plan precedes the corresponding circuit code.
  • Composite Circuit Design: Composite circuit design queries the library for subcircuit invocation methods before integrating the retrieved modules into the design prompt.This extends the feedback-based basic-design process with retrieval and reuse of previously successful circuit implementations.
  • Feedback-Enhanced Design Flow: Feedback-enhanced design returns runtime and circuit-verification errors to the LLM so failed designs can be automatically corrected.The flow uses error information from checking to refine generated code without requiring human intervention.
  • Circuit Tool Library: Successfully designed basic circuits are stored as reusable subcircuit tools, with descriptions and specifications used as keys and code and invocation methods as values.The library supports querying relevant tools and incorporating retrieved subcircuits into composite-circuit prompts.

4 Experiments

Experiments evaluate LLMs on a 24-task analog-circuit benchmark using Pass@k, ablations, fine-tuning, visualizations, and attempt-budget analysis. AnalogCoder and GPT-4o achieve the strongest overall design performance, while feedback, prompting, and reusable circuit tools improve results.

  • Metrics: Pass@k measures the ratio of correct generations within k independent trials, with higher values indicating better performance.The experiments use Pass@1 and Pass@5, with trial counts varying by model category.
  • Main Results: GPT-4o remains the best LLM for analog circuit design, while its circuit tool library further improves challenging composite-circuit design.AnalogCoder combines GPT-4o with prompt engineering, design-flow feedback, and the circuit tool library; the library also benefits Llama-3.
  • Ablations: Removing prompting components or the proposed design flow consistently decreases GPT-3.5 design performance.The ablations remove Python generation, in-context information, Chain-of-Thought reasoning, or feedback-enhanced flow.
  • Fine-tuning: Fine-tuned GPT-3.5 generally improves design performance but struggles to design additional circuits when training data is limited.The evaluation uses 3-fold cross-validation, with two task subsets for fine-tuning and one for testing.
  • Visualization: Visualization results show that even slight circuit discrepancies can render an analog design non-functional.Figure 6 presents successful and failed circuit diagrams and identifies the source LLM for each design.
  • Attempt Times: Most designs are completed within three attempts, after which success becomes unlikely while token consumption continues to rise.The study used 50 trials across three tasks, with up to five attempts per trial, and set three as the default attempt limit.

5 Conclusion

The paper concludes that AnalogCoder uses training-free Python-code generation, prompts, feedback, and a circuit tool library to automate analog circuit design. It reports broader accessibility while acknowledging limits in circuit complexity and computational resources.

  • Attempt Budget: Success probability drops significantly after the third design attempt, supporting a default attempt limit of three.This conclusion is based on 50 trials per task with a maximum of five attempts.
  • Conclusion: AnalogCoder combines Python code generation, crafted prompts, feedback-enhanced design, and a circuit tool library for automatic analog circuit design.The paper also provides an open-source benchmark for future research.
  • Conclusion: The method is intended to help people with limited design experience create analog circuits more easily.The conclusion characterizes analog design as complex, time-consuming, and error-prone.
  • Limitations: LLMs currently lack the capability to design highly complex analog circuits, and their use is constrained by cost and computational-resource availability.The paper identifies these constraints as current limitations and notes possible improvement with future advances.

A.1 Benchmark Details

The benchmark covers basic and composite analog-circuit design tasks, with difficulty determined by component count and connection complexity. Its prompts require a detailed topology plan followed by complete runnable PySpice-compatible Python code.

  • Example Task: The example benchmark task specifies a two-stage amplifier with a common-source current-source-load first stage and a common-source resistor-load second stage.The example uses Vin and Vbias as inputs, Vout as the output, NMOS devices M1 and M3, PMOS device M2, and resistor R1.
  • Example Topology: The first amplifier stage connects NMOS M1 to the input and ground while PMOS M2 provides a current-source load from Vdd.M1 and M2 share a drain node, with M2 biased by Vbias.
  • Example Topology: The second stage uses NMOS M3 as a common-source amplifier and resistor R1 to convert its current into the output voltage.M3 is driven by the first-stage drain node, and R1 connects its drain to Vdd through the output node.
  • Implementation Example: The implementation example uses PySpice netlist imports, transistor models, power supplies, device dimensions, and a simulator call to encode the two-stage amplifier.The prompt explicitly requests runnable PySpice-compatible code without placeholders or redundant code after simulator = circuit.simulator().
  • Prompt Requirements: The design prompt asks the model to produce both a detailed device-and-interconnectivity plan and complete Python code for the circuit topology.The code must describe device properties and node connections rather than leaving placeholders.

Tips

The prompts require runnable circuit code, explicit topology planning, and careful handling of MOSFETs, biasing, nodes, and reusable subcircuits.

  • Device constraints: MOSFET prompts emphasize correct parameter ordering, source-connected bulks, threshold-voltage-based biasing, and nominal transistor sizing.They also instruct designers to avoid AC source voltages and consider operating points.
  • Subcircuit use: Subcircuit-aware prompts provide reusable circuit information, invocation code, input/output nodes, and instructions to select as few subcircuits as possible.Composite-circuit prompts ask the model to choose subcircuits and enumerate their IDs before generating the topology.
  • Circuit requirements: Design prompts require input and output node names to appear in the circuit and commonly assume Vdd = 5.0 V without adding subcircuit power supplies.Specific prompt placeholders such as [TASK], [INPUT], and [OUTPUT] are replaced with task information.
  • Ablation prompts: The ablation variants separately test SPICE generation, removal of contextual examples, and removal of Chain-of-Thought reasoning.The SPICE variant changes the target language, while the other variants remove context or intermediate component-and-connection enumeration.
  • Output format: Runnable outputs must include a detailed device-and-connectivity plan followed by complete Python/PySpice or NgSpice code without placeholders.Python outputs should end at simulator = circuit.simulator(), while NgSpice outputs should end at .end.

A.4 Examples of Generated Circuits

The examples show AnalogCoder generating both basic and composite PySpice circuits, including a telescopic-cascode op-amp and an RC phase-shift oscillator.

  • Generated examples: AnalogCoder examples cover a basic cascade op-amp and a composite RC-shift oscillator built from the SingleStageOpamp subcircuit.The composite example connects three RC stages between the op-amp output, midpoint bias, and inverting input.
  • Basic circuit: The op-amp example defines MOSFET models, a 5 V supply, input and bias voltages, and a SingleStageOpamp instance in PySpice.The example labels the circuit as a single-stage telescopic-cascode op-amp.
  • Verification: The feedback-enhanced flow performs four main checks, each inspecting a specific part of the language model’s circuit design.The supplied passage refers to Table 8 for the individual checks but does not enumerate them here.

A.6 Experimental Settings

The experiments use simplified MOSFET modeling, fixed software and inference settings, and a fine-tuning split based on grouped successful designs.

  • Device modeling: The study uses Level-1 MOS SPICE models to reduce design complexity, with kp as transconductance parameter and vto as threshold voltage.All circuits include MOSFETs, but the LLM is not required to use a specified MOS SPICE model.
  • Software environment: All Python circuit code was tested with Python 3.10 and PySpice 1.5.
  • Inference settings: LLM inference used temperature = 0.5 and top_p = 1.0 to generate varied results across trials.
  • Fine-tuning setup: Successful designs were grouped into three categories, with samples from two groups used for fine-tuning and the remaining group reserved for testing.Testing on excluded Task IDs 12 and 15 found that GPT-3.5, GPT-4, and Llama-3 could not correctly complete the designs.
  • Data caveat: Fine-tuning data preprocessing removed infrequent u_V and circuit.I content, so post-fine-tuning outcomes may be unstable.

A.7 Supplementary Experiments

Supplementary experiments compare additional language models and GPT-4 variants, while visualizations include circuit outputs and an oscillator waveform.

  • Additional models: Additional representative models, including Mixtral, CodeLlama, Llama, QwenCode, DeepSeek-Coder, and Codestral, were evaluated in Table 10.
  • GPT-4 comparison: GPT-4o with the circuit tool library designed one more circuit than GPT-4, while their overall capabilities were otherwise similar.The authors therefore implemented AnalogCoder with GPT-4o based on overall cost considerations.
  • Additional-model results: The evaluated smaller or less-trained models successfully designed no more than four analog circuits.Reported examples include Mistral-7B with 2 solved, Llama 3-8B with 1, Qwen-1.5-110B with 2, and Llama 2-7B/13B with 0.
  • Visual analysis: The supplementary material presents additional visualizations and a simulated waveform for an AnalogCoder-generated RC-shift oscillator.

A.8 Error Bar Analysis

The analysis estimates confidence intervals for circuit-design success rates using independent trials and examines how sampling affects interval width and bias. It also reports that trial counts reflect financial and computational constraints.

  • Theoretical design success is modeled with a binomial distribution, and Wilson score intervals estimate its confidence range.The number of successful designs c is treated as c ∼ Bin(n, p), with confidence level CI = 90%.
  • The experiments use independent trials, with sample sizes of n = 15 or 30 selected partly according to financial and computational resources.
  • Increasing sampling iterations reduces the confidence-interval width to a certain extent.The intervals estimate the theoretical success probability from experimental Pass@p values.
  • Smaller Pass@1 values tend to underestimate the theoretical success probability, whereas larger values tend to overestimate it.

B Related Work

Related work applies LLMs to code generation and electronic design automation, while highlighting challenges from uneven training-data coverage across programming languages and domains.

  • Code-generation models can perform poorly on underrepresented domains because training data is imbalanced across programming languages and application areas.Prior work addresses this issue by adding domain-specific code samples or adapting models with internal design-code data.
  • LLM research in electronic design automation supports hardware chatbots, HDL and script generation, and code verification and analysis.
Loading 2405.14918v2…