PaperlayerView exact revision on arXiv

Source-linked AI summary

SPICEPilot: Navigating SPICE Code Generation and Simulation with AI Guidance

Deepak Vungarala, Sakila Alam, Arnob Ghosh, Shaahin Angizi

arXiv:2410.20553v1cs.ARcs.AI

TL;DR

LLMs’ use in hardware design remains limited by a lack of hardware-domain knowledge. SPICEPilot combines PySpice-based code generation, hardware-guided prompts, and dataset construction with benchmarking; its prompt-engineered approach reports gains on Pass@1 and pass@5 and generates all 24 benchmark circuits.

  • Problem

    LLM use in hardware design remains limited because models lack hardware-domain knowledge, despite their capabilities in Python code generation.

    Source: §I · p. 1

  • Method

    SPICEPilot uses PySpice and hardware-guided prompts to generate and validate Python-based SPICE code, building a dataset and benchmark without fine-tuning.

    Source: §I · p. 1, §IV · p. 3, A. Framework · p. 4

  • Results

    With prompt-engineered ACB descriptions, the framework reports a 52.90% improvement in Pass@1, a 1.91% improvement at pass@5, and generation of all 24 benchmark circuits.

    Source: §V · p. 5

  • Takeaways & Limitations

    The framework offers an open-source dataset approach to data scarcity and a foundation for further LLM optimization in analog circuit applications.

    Source: §VII · p. 5

Abstract

from arXiv · show

Large Language Models (LLMs) have shown great potential in automating code generation; however, their ability to generate accurate circuit-level SPICE code remains limited due to a lack of hardware-specific knowledge. In this paper, we analyze and identify the typical limitations of existing LLMs in SPICE code generation. To address these limitations, we present SPICEPilot a novel Python-based dataset generated using PySpice, along with its accompanying framework. This marks a significant step forward in automating SPICE code generation across various circuit configurations. Our framework automates the creation of SPICE simulation scripts, introduces standardized benchmarking metrics to evaluate LLM's ability for circuit generation, and outlines a roadmap for integrating LLMs into the hardware design process. SPICEPilot is open-sourced under the permissive MIT license at https://github.com/ACADLab/SPICEPilot.git.

I. INTRODUCTION

LLMs’ application to hardware design remains limited by a lack of hardware-domain knowledge, while analog-circuit generation has seen few works. SPICEPilot proposes a PySpice-based dataset and framework, evaluates LLMs on fundamental circuits, and introduces benchmarking criteria and a roadmap for future work.

I. INTRODUCTION

  • SPICEPilot uses PySpice to generate Python-based SPICE code across transistor models and circuit configurations, and introduces standardized criteria for benchmarking generated circuits.The framework is intended to support circuit creation and simulation.Source: §I · p. 1
  • The paper evaluates open-source and proprietary LLMs on their ability to generate fundamental circuits.The contribution statement names both model categories but does not provide evaluation outcomes here.Source: §I · pp. 1–2
  • SPICEPilot proposes using LLMs to generate open-source analog-circuit datasets and introduces a metric for assessing hardware-design performance.The paper frames dataset generation as a response to data scarcity.Source: §I · p. 2
  • The paper outlines a roadmap for future advances in LLMs for analog circuit design.Its stated research questions address reliability, specialized models, data scarcity, dataset generation, circuit reasoning, and benchmarking.Source: §I · pp. 1–2

II. BACKGROUND

The background surveys LLM-based hardware-design work, including HDL/HLS generation and analog-circuit efforts, and describes SPICE as a tool for simulating and modeling electronic circuits.

II. BACKGROUND

  • Prior hardware-design systems use LLMs for tasks such as Verilog generation, RTL-to-GDSII workflows, design optimization, and design-space exploration.The survey also notes gaps including prompt optimization, tailored datasets, fine-tuning, and hallucination.Source: §II · p. 2
  • AnalogCoder is described as an early analog-circuit generator, while AmpAgent targets multi-stage amplifier schematic design and process and performance porting.Source: §II · p. 2
  • SPICE predicts circuit behavior through mathematical analysis and models components connected through circuit nodes in a netlist.The described components include passive elements and semiconductor devices such as MOSFETs and BJTs.Source: §II · p. 2

III. HOW TALL DOES LLM STAND IN SPICE CODE GENERATION?

The paper examines whether LLMs can generate SPICE code, a specialized task for which their proficiency is uncertain because such tasks are less common in pretraining.

III. HOW TALL DOES LLM STAND IN SPICE CODE GENERATION?

  • LLMs’ proficiency in generating SPICE code remains uncertain because this specialized task is less frequently encountered during pretraining.Source: §III · p. 2
  • The paper evaluates state-of-the-art LLMs to characterize their capabilities and limitations in SPICE code generation.The authors say this assessment is intended to provide insight for future work in automated hardware design.Source: §III · p. 2
  • The authors identify common limitations in SPICE code generation as a basis for reassessing LLM potential in SPICE-based hardware-design automation.Source: §III · p. 2

A. Misconception of Gate Width and Length

The section identifies gate-width and gate-length misconceptions as a source of inaccurate SPICE circuit generation, highlighting failures to apply a standard transistor-width ratio.

A. Misconception of Gate Width and Length

B. Inability to Perform Circuit Analysis

LLMs often fail to choose and perform circuit analyses suited to a circuit’s operational requirements. Table II is identified as an analysis of LLM ability in SPICE generation.

B. Inability to Perform Circuit Analysis

  • LLMs fail to autonomously select the appropriate analysis—transient, DC, or AC—based on circuit requirements.The paper associates these analysis types with time-domain response, steady-state behavior, and frequency response, respectively.Source: B. Inability to Perform Circuit Analysis · p. 3
  • Table II is identified as an analysis of LLM ability in SPICE generation.Source: Table II · p. 3
  • Plain implementations of the tested LLMs fail to perform proper circuit analysis.The text says Table II highlights this failure.Source: B. Inability to Perform Circuit Analysis · p. 3

C. Incorrect Input Signal Assignment and Device Parameter Configuration

LLMs make errors in assigning circuit inputs and device parameters, while SPICEPilot uses a PySpice workflow with reference-guided generation and output validation to address common errors.

C. Incorrect Input Signal Assignment and Device Parameter Configuration

  • LLMs may misassign signal levels, frequencies, or waveform types and overlook operating temperature, undermining simulation reliability.Temperature affects carrier mobility and semiconductor-device behavior.Source: C. Incorrect Input Signal Assignment… · p. 3

IV. SPICEPILOT

  • SPICEPilot uses PySpice and a step-by-step reference to guide code generation and validate each LLM output without costly fine-tuning.The reference supplies circuit logic, reasoning, and error-identification guidance intended to reduce trivial errors and common mistakes.Source: §IV · p. 3
  • The paper relates its reference-guided approach to in-context learning, which Dai et al. [20] describe as producing meta-gradients through forward computation.The passage contrasts this with explicit fine-tuning, which updates parameters through backpropagation.Source: §IV · p. 3

A. Framework

SPICEPilot turns user circuit specifications into validated PySpice code through iterative LLM generation, human review, and error feedback, while extracting circuit metrics for a curated dataset.

A. Framework

  • The workflow starts with user specifications, which the Pilot Prompt augments with hardware knowledge and PySpice coding guidance.Source: A. Framework · p. 4
  • Generated PySpice code undergoes expert review for netlist correctness, with error comments returned to the LLM for revision when needed.The generate–validate–feedback cycle repeats until valid code is produced.Source: A. Framework · p. 4
  • Verified code yields a SPICE netlist from which circuit parameters, including transistor count, and metrics such as gain are extracted with metadata.Source: A. Framework · p. 4
  • Validated code enters a dataset for future LLM improvement through in-context learning or fine-tuning, and the framework outputs functional SPICE and PySpice code.Source: A. Framework · p. 4

B. Dataset Generation and Benchmarking

SPICEPilot stores paired PySpice and SPICE representations and establishes a 60-circuit benchmark classified by transistor count, while identifying node count as a possible refinement.

B. Dataset Generation and Benchmarking

V. EXPERIMENTAL ANALYSIS

The experiments compare SPICEPilot with the AnalogCoder Benchmark and evaluate models on an enhanced benchmark. SPICEPilot reports higher Pass@1 and Pass@5 scores than ACB and generates all 24 circuits in that benchmark.

V. EXPERIMENTAL ANALYSIS

  • The evaluation adopts Pass@k (k=1, 5) as its main standard, measuring the proportion of correct generations within k independent attempts across n trials (n ≥ k), with higher values indicating better performance.Evaluated models include CodeLlama-70B-Instruct [21], Wizardcoder-33B-V1.1 [22], Llama3-70B, GPT-3.5, and GPT-4o.Source: §V · p. 4
  • The ACB experiment uses Claude-3.5 Sonnet with a Pilot prompt and adds more detailed verbal circuit descriptions to the benchmark prompts.The Pilot prompt is supplied as an initial reference for learning.Source: §V · pp. 4–5
  • 52.90% improvement in Pass@1 and 1.91% improvement in Pass@5 are reported for SPICEPilot over ACB, with all 24 benchmark circuits generated.The reported comparison is presented across models in Table IV.Source: Table IV · p. 5, §V · p. 5
  • On the enhanced benchmark, two closed-source models were tested on 10 randomly selected circuits of varying complexity; hard-circuit pass rates were initially low.The authors attribute the low hard-circuit percentage to factors including Python module definitions unsupported by SPICE and circuit tuning; they note that repeated prompting or CoT may address these issues.Source: §V · p. 5, Table V · p. 5

VI. DISCUSSIONS AND FUTURE WORKS

SPICEPilot generated accurate netlists in the reported experiments, but functional efficiency and key parameters such as gain require more circuit-specific knowledge. The authors propose augmenting the framework and dataset with richer circuit knowledge and representations.

VI. DISCUSSIONS AND FUTURE WORKS

  • Accurate netlists did not by themselves ensure functional efficiency or key parameters such as gain, which the authors say require further knowledge instillation.They attribute this need to the LLM’s limited circuit-specific intelligence.Source: §VI · p. 5
  • The proposed extensions include circuit images as multimodal inputs and Python-generated circuit and waveform representations to support visual reasoning.The framework is described as addressing a data bottleneck through automated data augmentation.Source: §VI · p. 5
  • The dataset could add detailed descriptions, following the methodology proposed in, to support generation of more sophisticated and accurate SPICE models.Source: §VI · p. 5

VII. CONCLUSION

The conclusion presents SPICEPilot as a PySpice-based framework for SPICE-code generation and benchmarking across analog and digital circuits. Its evaluation highlights current LLM limitations and future potential, while its open-source datasets address data scarcity and provide a foundation for further research.

VII. CONCLUSION

  • SPICEPilot combines LLMs and PySpice to automate SPICE-code generation and provide a framework for benchmarking circuit performance.Source: §VII · p. 5
  • Evaluation of open-source and proprietary LLMs underscores current limitations and future potential for AI-driven code generation in hardware design.Source: §VII · p. 5
  • The open-source datasets address data scarcity, and the work lays a foundation for further research on LLMs for analog circuit applications.Source: §VII · p. 5

Related papers

Loading 2410.20553v1…