Source-linked AI summary
AnalogPortBench: Benchmarking LLMs on Analog Circuit Porting
Saeid Rajabi, Aly Moussa, Vishal Saxena, Chengmo Yang, Satwik Patnaik
TL;DR
Analog circuit porting can require redesign when lower supply reduces voltage headroom, and existing LLM benchmarks largely omit cross-node porting and infeasible specifications. AnalogPortBench evaluates six LLMs on 54 porting tasks with varied difficulty and simulation-based scoring. Models port simple circuits more successfully than headroom-sensitive amplifiers, while simulation feedback helps more than additional samples or reasoning.
Problem
Existing analog LLM benchmarks mostly focus on one process node and feasible specifications, leaving cross-node porting and infeasibility recognition untested.
Source: §I · p. 1Method
AnalogPortBench evaluates six LLMs on 54 tasks porting ten circuits from 0.18 µm to SKY130, with tiers covering reduced supply, harder specifications, and infeasible cases.
Source: §I · p. 1, §III · p. 2Results
Models port simple circuits but rarely redesign headroom-sensitive amplifiers, only one model reliably recognizes infeasible specifications, and simulation feedback helps more than additional samples or reasoning.
Source: §VI · p. 4Takeaways & Limitations
Headroom-aware topology choice and infeasibility recognition remain open gaps in the evaluated LLMs' analog porting capabilities.
Source: §VI · p. 4
Abstract
from arXiv · showhide
Analog circuit design still relies heavily on expert knowledge, and porting proven designs to new process nodes is a recurring bottleneck. As a lower supply reduces available voltage headroom, a working topology may require redesign or fail to meet specifications. While large language models (LLMs) have begun to generate and optimize analog circuits, existing benchmarks focus on one process node and assume feasible specifications. To bridge this gap, we introduce AnalogPortBench, a benchmark of 54 tasks for porting ten circuits, from current mirrors to telescopic amplifiers, from a 0.18 $\mu$m process to the open-source SKY130 PDK. Tasks range from a direct port through a reduced supply and harder specifications to infeasible ones, and every feasible task has a reference design verified at five process corners. Evaluating six frontier LLMs at two reasoning levels, with and without simulation feedback, we find that models port simple circuits but rarely re-design headroom-sensitive amplifiers, seldom recognize infeasible specifications, and gain more from simulation feedback than from more samples or reasoning. Code and data are available at https://github.com/saeidrajabii/AnalogPortBench.
I. INTRODUCTION
Analog circuit porting is difficult because a lower supply can remove the voltage headroom needed by a working topology, while some specifications may be physically unattainable. AnalogPortBench addresses gaps in existing benchmarks with porting tasks that test topology changes and recognition of infeasible specifications.
I. INTRODUCTION
- When supply falls from 1.8 V to 1.2 V, a device stack may lose the headroom needed for saturation, forcing redesign or acceptance that a specification is out of reach.Source: §I · p. 1
- Existing analog benchmarks mostly evaluate design or sizing within one process node, while porting may require topology changes and some specifications may be unattainable.Analog porting therefore tests both adapting a design to a new environment and recognizing when physics rules out a valid port.Source: §I · p. 1
- AnalogPortBench contains 54 tasks across ten circuits, including four tasks with infeasible specifications and reference designs verified at five process corners.The circuits are organized into three difficulty grades and four task tiers.Source: §I · p. 1
- AnalogPortBench uses an open-source SKY130 and ngspice evaluation flow that checks generated designs from netlist validity through specification limits.Source: §I · p. 1
- The benchmark evaluates six LLMs at two reasoning levels, with and without simulation feedback, across five research questions about porting, headroom, infeasibility, and failure points.The questions also compare the effects of reasoning and simulation feedback.Source: §I · p. 1
II. BACKGROUND AND RELATED WORK
Prior work uses LLMs to generate and size analog circuits, but most benchmarks test designs within one process. AnalogPortBench evaluates cross-node porting with rule checks, simulation, and scoring that also covers infeasible-task answers.
II. BACKGROUND AND RELATED WORK
- Analog LLM research includes generating circuits, sizing devices, guiding numerical optimization, using symbolic solvers, and reading schematics.Earlier analog optimization also used Bayesian optimization and reinforcement learning.Source: §II · p. 1
- The benchmark flow takes a 0.18 µm netlist and specifications, checks and simulates the SKY130 netlist, and scores results stage by stage, including INFEASIBLE answers on T4 tasks.Simulation feedback permits up to three attempts.Source: Figure 1 · p. 2
- Most prior analog benchmarks evaluate designs within one process; the cited agents keep topology fixed and do not test whether specifications are feasible.EEsizer moves an op-amp across nodes, but also keeps its topology fixed.Source: §II · p. 2
III. ANALOGPORTBENCH BENCHMARK
AnalogPortBench tests whether LLMs can port working designs to SKY130, adapt topologies when headroom requires it, and identify infeasible specifications. Its task tiers separate process-node porting, reduced supply, harder feasible redesign, and deliberately infeasible cases.
III. ANALOGPORTBENCH BENCHMARK
- Each task asks an LLM to port a 0.18 µm design and specification to SKY130, then scores the returned netlist by checks and simulation or by its infeasibility declaration.Source: §III · p. 2
- The 54 tasks span ten circuits grouped into three grades by voltage headroom and design effort, from basic circuits to headroom-sensitive telescopic OTAs.G1 includes current mirrors, bias circuits, and a single-stage OTA; G2 includes two-stage Miller and folded-cascode OTAs.Source: §III · p. 2
- T1 tests porting at 1.8 V, T2 lowers supply to 1.2 V, and T3 tests redesign against harder specifications that reference designs meet.For telescopic OTAs, T2 instead uses three alternative specification points trading gain, speed, and swing.Source: §III · p. 2
- T4 sets specifications beyond a device physics bound to test whether the model declares infeasibility; the benchmark has 50 feasible tasks and four infeasible tasks.The stated bound assumes output feedback is excluded.Source: §III · p. 2
- Each feasible task has a simulation-tuned reference design that meets every limit with all transistors in saturation at five process corners.The checks use 27 °C and nominal supply voltage; benchmark rules also require current mirroring, prohibit direct input-output connections, and limit input current to 1 µA.Source: §III · p. 2
IV. EXPERIMENTAL SETUP
The evaluation varies model, reasoning effort, and simulation feedback while applying task-specific prompts and simulation-based scoring. Failed medium-effort T1–T3 attempts can receive up to two further rounds of feedback.
IV. EXPERIMENTAL SETUP
- Six LLMs are evaluated at low and medium reasoning effort, with each effort answering each task three times independently.The models include Claude Opus, Sonnet, and Haiku 5.5, plus GPT-6-sol, GPT-6.1-sol, and GPT-6-luna.Source: §IV · p. 2
- The setup defines metrics including current error, output resistance, compliance voltage, output-current matching over voltage, bandwidth, margins, settling, and swing.The passage also defines OTA, LV, Cc–Rz, UGB, and UGF.Source: §IV · p. 3
- Prompts provide the source netlist, specifications, test conditions, available SKY130 devices, and benchmark rules, but not testbench files.Models may change dimensions, biasing, and topology; an INFEASIBLE answer must state the smallest relaxation and include a best-effort netlist.Source: §IV · p. 3
- Answers are rule-checked and simulated at the typical corner and 27 °C, then classified as pass, fail, does not work, or invalid.The operating point is checked to confirm transistor saturation.Source: §IV · p. 3
- At medium effort, failed first samples on T1–T3 continue with simulation feedback for at most three total attempts.Feedback provides measured metrics, limits, pass status, and transistor operating points, but no diagnosis of the failure cause.Source: §IV · p. 3
V. RESULTS
LLMs port simple circuits more successfully than harder ones, while reduced headroom and infeasible specifications expose important weaknesses. Simulation-based evaluation also shows that a direct device swap is rarely sufficient.
Evaluation overview
- The naïve device swap passes only 2 of 54 tasks, while the reference designs confirm feasibility of all T1–T3 tasks.Table IV reports zero-shot pass rates by tier and grade, infeasibility flags, and three-try resampling or feedback outcomes.Source: §V · p. 3, Table IV · p. 4
RQ1. Can LLMs port a working design?
- 93% of Opus 5.5’s T1 answers pass on G1 circuits, but no model exceeds 22% on G2 or G3, even at the original supply.Most failing T1 answers are working circuits that miss a limit rather than broken netlists.Source: §V · p. 3
RQ2. What breaks when the headroom shrinks?
- Lowering supply from 1.8 V to 1.2 V cuts pass rates from 53% to 33% for Opus 5.5 and from 40% to 25% for GPT-6.1-sol.For the headroom-sensitive telescopic cascode, Opus 5.5 changed to a two-stage Miller OTA and met every limit; a correct port required topology change as well as re-sizing.Source: §V · p. 3
RQ3. Do LLMs recognize infeasible specifications?
- Opus 5.5 flags 10–11 of 12 infeasible G1 specifications with almost no false alarms, while GPT-6.1-sol rarely flags them despite porting well.Higher reasoning effort did not improve infeasibility detection.Source: §V · p. 3
that silently miss an unreachable limit.
Simulation feedback helps all models more than repeated sampling, especially on hard T3 tasks. The failure patterns also vary by circuit grade, with gain remaining a challenge even when hard-circuit transistors are saturated.
RQ4. More reasoning or simulation feedback?
- Feedback solves over half of feasible tasks for the two strongest models, versus about a third with resampling; on T3 it raises solved tasks from 2 to 13 for GPT-6.1-sol and from 2 to 12 for Opus 5.5.More reasoning helps only the strongest models, while others gain little or nothing despite using up to 4.6× more tokens.Source: that silently miss an unreachable limit · p. 4
RQ5. Where in the design flow do designs fail?
- On medium circuits, gain is met by at most 16% of answers versus up to 73% for stability, suggesting a gain–stability trade-off.On hard circuits, saturation separates models, and even designs with all transistors saturated lose gain.Source: that silently miss an unreachable limit · p. 4
VI. CONCLUSION
ANALOGPORTBENCH evaluates LLM porting of analog circuits to a new process node and lower supply. Across six frontier models, simple circuits are ported more successfully than headroom-sensitive amplifiers, and simulation feedback improves performance more than additional samples or reasoning.
Conclusion
- The benchmark evaluates LLMs porting analog circuits to a new process node and lower supply.It evaluates six frontier LLMs.Source: §VI · p. 4
- Models port simple circuits but rarely redesign headroom-sensitive amplifiers, and only one model reliably recognizes infeasible specifications.Source: §VI · p. 4
- Simulation feedback improves performance more than additional samples or reasoning; open gaps include headroom-aware topology choice, infeasibility recognition, LLM-guided numerical sizing, and corner-robust design.Source: §VI · p. 4