PaperlayerView exact revision on arXiv

Source-linked AI summary

Masala-CHAI: A Large-Scale SPICE Netlist Dataset for Analog Circuits by Harnessing AI

Jitendra Bhandari, Vineet Bhat, Yuheng He, Hamed Rahmani, Siddharth Garg, Ramesh Karri

arXiv:2411.14299v5cs.AR

TL;DR

Open SPICE datasets are limited, and turning schematic images into netlists requires painstaking manual work. Masala-CHAI automates schematic-to-netlist dataset creation and supports LLM fine-tuning; fine-tuned GPT-4o achieved performance across all evaluated design IDs. The resulting dataset and code are open-sourced and extensible.

  • Problem

    Open SPICE datasets are limited compared with Verilog, while analog schematics in textbooks and papers require painstaking manual conversion into netlists.

    Source: §I · p. 1

  • Method

    Masala-CHAI combines automated schematic extraction, component detection, net annotation, prompt tuning, and verification to create SPICE netlists and a dataset for fine-tuning LLMs.

    Source: §I · p. 1, A. Dataset Creation · p. 4

  • Results

    Fine-tuned GPT-4o achieved higher Pass@k scores on design IDs 1–13 and nonzero performance on IDs 14–20, while its “Best Result” covered all design IDs.

    Source: B. Finetuning · p. 5

  • Takeaways & Limitations

    The open-sourced dataset and codebase provide a SPICE netlist resource that can be extended with additional textbooks and documentation.

    Source: §VI · p. 6

Abstract

from arXiv · show

Masala-CHAI is a fully automated framework leveraging large language models (LLMs) to generate Simulation Programs with Integrated Circuit Emphasis (SPICE) netlists. It addresses a long-standing challenge in circuit design automation: automating netlist generation for analog circuits. Automating this workflow could accelerate the creation of fine-tuned LLMs for analog circuit design and verification. In this work, we identify key challenges in automated netlist generation and evaluate multimodal capabilities of state-of-the-art LLMs, particularly GPT-4, in addressing them. We propose a three-step workflow to overcome existing limitations: labeling analog circuits, prompt tuning, and netlist verification. This approach enables end-to-end SPICE netlist generation from circuit schematic images, tackling the persistent challenge of accurate netlist generation. We utilize Masala-CHAI to collect a corpus of 7,500 schematics that span varying complexities in 10 textbooks and benchmark various open source and proprietary LLMs. Models fine-tuned on Masala-CHAI when used in LLM-agentic frameworks such as AnalogCoder achieve a notable 46% improvement in Pass@1 scores. We open-source our dataset and code for community-driven development.

I. INTRODUCTION

Masala-CHAI addresses the limited availability of open SPICE datasets and the laborious conversion of schematic images into netlists. It combines automated extraction, labeling, prompt tuning, and verification to build a large dataset and support LLM training.

I. INTRODUCTION

  • Open SPICE datasets are limited relative to Verilog, while converting textbook and research-paper schematic images into netlists is painstaking and time-consuming.The passages contrast plentiful analog diagrams with the manual work needed to turn them into SPICE netlists.Source: §I · p. 1
  • The framework automates schematic-to-netlist extraction using component detectors, net identification, prompt tuning, and post-extraction verification.It also uses a schematic extractor and custom-trained deep network models.Source: §I · p. 1
  • 46% absolute improvement in Pass@1 scores followed incorporating GPT-4o fine-tuned on Masala-CHAI into the AnalogCoder agentic framework.The paper reports this as an improvement over the framework without that incorporation.Source: §I · p. 2
  • Masala-CHAI produced an open-source dataset of about 7,500 SPICE netlists with metadata from 10 textbooks.The contributions describe it as the largest curated and labeled SPICE dataset available to the authors' knowledge.Source: §I · p. 2

II. RELATED WORK

Prior LLM work in hardware has focused largely on digital tasks, while accurate SPICE extraction from schematics remains challenging. The paper examines GPT-4o’s errors and describes component-detection and prompt-tuning approaches to address them.

II. RELATED WORK

  • Prior LLM hardware work includes Verilog generation and other digital tasks, while earlier ML schematic-to-netlist solutions were not reproducible because open-source implementations were unavailable.The AMSNet comparison table distinguishes manual, partially automated, and fully automated approaches.Source: §II · p. 2, Table I · p. 2

III. KEY CHALLENGES IN SPICE NETLIST EXTRACTION

  • Multimodal LLMs such as GPT-4o do not automatically extract accurate SPICE netlists from schematics, motivating analysis of their failure modes and an automated framework without manual annotations.The paper frames this challenge against the limited availability of open SPICE datasets.Source: §III · p. 2

A. Can GPT-4o detect electrical components accurately?

  • GPT-4o often confuses NMOS and PMOS transistors or omits components such as PMOS transistors and current sources, although it identifies all components in some cases.The component-recognition evaluation used analog schematics containing passive and active devices.Source: A. Can GPT-4o detect electrical… · p. 2, Figure 2 · p. 2
  • A customized CNN-based object detector is proposed to detect and bound schematic components more accurately than relying on GPT-4o for recognition.The approach is based on a deep CNN object-detection model such as YOLO.Source: A. Can GPT-4o detect electrical… · p. 2
  • A targeted prompt enhancement emphasizing structural differences between NMOS and PMOS transistors achieved notable improvements in LLM component-differentiation accuracy, with the paper reporting empirical support for the approach in Section IV.Figure 3 shows prompt tuning guiding a GPT to differentiate between NMOS and PMOS.Source: A. Can GPT-4o detect electrical… · p. 2, Figure 3 · p. 2

B. Can GPT-4o accurately connect circuit components?

GPT-4o can map two-terminal devices but makes errors when interpreting net connectivity, MOSFET terminals, and differential pairs. The proposed Deep Hough transform net annotations reduce ambiguity and improve its schematic-to-netlist interpretation.

B. Can GPT-4o accurately connect circuit components?

  • GPT-4o incorrectly treats some intersecting nets as connected, confuses MOSFET drain, gate, and source terminals, and mishandles differential input and output connections.The paper notes that connectivity conventions vary across schematic notations and gives examples of incorrect transistor connections and voltage assignments.Source: B. Can GPT-4o accurately connect… · p. 3
  • Deep Hough transform line priors are used to annotate nets, and adding those annotations to the prompt reduces ambiguity and improves GPT-4o’s component-connection interpretation, particularly for complex schematics.Source: B. Can GPT-4o accurately connect… · p. 3

IV. METHODOLOGY

Masala-CHAI converts textbook schematics into SPICE netlists through a workflow that extracts schematics and context, labels circuits, tunes prompts, and verifies netlists. Its labeling workflow includes automated component detection using a fine-tuned YOLOv8 model.

IV. METHODOLOGY

  • Masala-CHAI’s workflow extracts textbook schematics and descriptive text, then converts the schematic images into netlists through labeling, prompt tuning, and verification.The descriptive text may come from a figure caption or other text on the page and supplies context for LLM fine-tuning.Source: §IV · p. 3

A. Labeling Analog Circuit

  • Automated labeling simplifies netlist generation by detecting circuit components and facilitating net annotation, replacing a manual process that hinders large-scale dataset creation.Source: A. Labeling Analog Circuit · p. 3

1) Detection of Circuit Components:

  • The component detector is a fine-tuned YOLOv8 model trained on 4,300 annotated circuit diagrams spanning 12 component classes.The model uses 640x640 images and was trained for 1,000 epochs with a learning rate of 0.01; it outputs component locations, bounding boxes, and class labels.Source: 1) Detection of Circuit Components · p. 3

2) Net Detection:

Masala-CHAI detects schematic nets by isolating line segments and clustering nearby endpoints, while prompt tuning guides GPT to generate SPICE netlists from schematics.

2) Net Detection:

  • Net detection removes components before applying deep Hough transform line priors to isolate schematic nets.This leaves net-representing lines for subsequent analysis and reduces ambiguity from lines inside components.Source: 2) Net Detection · p. 3
  • Line segments whose endpoints fall within 40 pixels are clustered as one net, with the radius tuned and clusters manually checked.The procedure is illustrated in the annotated-schematic workflow.Source: 2) Net Detection · p. 3, Figure 6 · p. 4

B. Prompt Tuning

  • Prompt tuning explores prompt designs to improve GPT's schematic understanding and produce domain-specific responses.The Figure 7 prompt is used at stage ➍ to query GPT.Source: B. Prompt Tuning · p. 4, Figure 7 · p. 4

C. SPICE netlist verification

Masala-CHAI adds LLM-based post-extraction checks to correct common netlist errors, and uses the automated workflow to assemble and characterize a dataset of textbook schematics.

C. SPICE netlist verification

  • Generated netlists can still contain floating nets, so a final LLM verification step identifies and corrects common errors.The open-source checks can be extended to address other error sources.Source: C. SPICE netlist verification · p. 4

A. Dataset Creation

  • The dataset contains approximately 7,500 schematic images drawn from 10 textbooks, with captions and descriptions also collected.Default component parameters are used because the goal is structurally correct netlists.Source: A. Dataset Creation · p. 4, Figure 8 · p. 5
  • Figure 9 characterizes variation in component counts, node counts, MOSFET counts, and SPICE netlist line counts.The distributions represent circuit complexity and connectivity and support benchmarking automated SPICE generation.Source: Figure 9 · p. 5, A. Dataset Creation · p. 5

B. Finetuning

The paper fine-tunes open-source and proprietary LLMs on Masala-CHAI and evaluates netlist generation on 20 circuit-design problems. Fine-tuned GPT-4o generates netlists for every benchmark design.

B. Finetuning

  • The evaluated models included CodeLlama-70B [39], DeepSeek-V2 [40], GPT-3.5, and GPT-4o-mini; all baseline LLMs were fine-tuned using the corpus collected through MASALA-CHAI.Open-source models were fine-tuned on an Nvidia A100 80GB GPU using LoRA, while GPT models used their respective fine-tuning playground.Source: B. Finetuning · p. 5
  • Fine-tuning improves Pass@k for open-source models and expands their coverage to design IDs 6–8; fine-tuned GPT-3.5 and GPT-4o reach IDs 14–20 with non-zero performance.The paper also illustrates successful and failed generated netlists, including topology and connection errors.Source: B. Finetuning · p. 5
  • The benchmark uses 20 design problems, and models are prompted to generate the corresponding SPICE netlists.The benchmark designs are identified by ID in Table II.Source: B. Finetuning · p. 5, Table II · p. 6
  • Fine-tuned GPT-4o successfully generates SPICE netlists for all design IDs in the evaluation benchmark.Table III summarizes Pass@k performance across the LLMs and their fine-tuned variants.Source: B. Finetuning · p. 5, Table III · p. 6

C. Integration into the AnalogCoder [14] Framework

Using Masala-CHAI-fine-tuned GPT-4o in AnalogCoder improves Pass@1 on many Tasks 12–20, with other reported tasks remaining at 100%. On Task 20, the fine-tuned model with AnalogCoder feedback reaches 60%, compared with 20% for base GPT-4o and 40% for the fine-tuned model alone.

C. Integration into the AnalogCoder [14] Framework

  • Task 20 Pass@1 rises from 20% with base GPT-4o to 60% with Masala-CHAI-fine-tuned GPT-4o in AnalogCoder.The fine-tuned GPT-4o model alone reaches 40% Pass@1 on Task 20.Source: C. Integration into the AnalogCoder… · p. 6
  • Many Tasks 12–20 improve with the fine-tuned model in AnalogCoder, while other reported results remain unchanged at 100%.Table IV reports the task-level Pass@1 and Pass@5 comparisons, excluding tasks where AnalogCoder already achieves 100% success.Source: Table IV · p. 6, C. Integration into the AnalogCoder… · p. 6

VI. CONCLUSION

Masala-CHAI automates the creation of large-scale SPICE netlist datasets from analog textbooks, and its dataset and codebase are open-source and extensible.

VI. CONCLUSION

  • Masala-CHAI produces a publicly available dataset of 7,500 SPICE netlists with corresponding captions.The framework is described as creating large-scale datasets from analog textbooks.Source: §VI · p. 6
  • Fine-tuning LLMs on Masala-CHAI demonstrates its use for generating SPICE netlists from natural-language prompts.Source: §VI · p. 6
  • The dataset and codebase are open-source and designed to be extended with additional textbooks and documentation.Source: §VI · p. 6

APPENDIX

The evaluation compares SPICE netlists through normalized graph edit distance and reports a 100% similarity score against AMSNet netlists. The comparison finds identical component counts and neighboring elements despite differences in naming conventions.

A. Evaluation

  • The evaluation parses netlists into component-and-connection graphs and uses normalized graph edit distance on a 0-to-100 scale.Graph nodes represent components, while edges represent interconnections; component types make the representation invariant to component labels.Source: A. Evaluation · p. 8

B. Verifying against AMSNet [13] netlists

  • Masala-CHAI netlists score 100% similarity against AMSNet netlists, with matching component counts and neighboring elements despite naming differences.Figure 11 shows the compared graph representations.Source: B. Verifying against AMSNet [13]… · p. 8

C. Role of high-quality dataset on Finetuning

The paper compares GPT-4o fine-tuned on AMSNet and Masala-CHAI using a Common Source Amplifier example. It notes that AMSNet lacks netlist descriptions, whereas Masala-CHAI provides descriptions alongside netlists for fine-tuning.

C. Role of high-quality dataset on Finetuning

Related papers

Loading 2411.14299v5…