Source-linked AI summary
THEIA: A Multimodal Dataset and Benchmark for Vision-Language Analysis of Layout
Giuseppe Chiari, Michele Piccoli, Federico Viola, Davide Zoni
TL;DR
Extracting semantic information from analog-circuit GDSII layouts requires manual inspection or rigid, expensive scripts. THEIA pairs layout images with question-answer data and benchmarks task-specific VLM fine-tuning; across five tasks, Qwen2.5-VL-7B accuracy improves by 65-81 percentage points under a controlled protocol.
Problem
Extracting high-level semantic insights from GDSII layouts requires exhaustive manual inspection or rigid, computationally expensive scripts.
Source: §1 · p. 2Method
THEIA pairs analog-layout images with question-answer conversations and benchmarks task-specific fine-tuning of a pretrained VLM for GDSII layout analysis.
Source: §1 · pp. 2–3Results
65-81 percentage points: task-specific fine-tuning improves Qwen2.5-VL-7B accuracy across five tasks under a controlled protocol.
Source: §5 · p. 10Takeaways & Limitations
THEIA provides a realistic basis for evaluating analog-layout semantics rather than raw geometry alone.
Source: §5 · p. 10The evaluation uses only the SkyWater 130 nm technology node and does not assess cross-node generalization.
Source: §7 · p. 10
Abstract
from arXiv · showhide
The integration of artificial intelligence into computer-aided design frameworks has sparked a shift in the design of analog integrated circuits (ICs), transitioning the field from using manual and algorithmic-based solutions to adopting automated and intelligent paradigms. In this scenario, the GDSII file represents the industry-standard database containing the ultimate and most accurate source of information of the analog circuit, encapsulating the complex physical geometries and parasitic realities that define tape out performance. This paper proposes THEIA, a novel dataset containing thousands of layout images paired with question-answer conversations, along with a benchmark that employs a fine-tuned vision-language model (VLM) to analyze GDSII files of analog circuits, enabling designers to interact with and query physical layouts as intuitive, meaningful entities. Experimental results using thousands of analog designs across five realistic tasks demonstrate that the proposed fine-tuned VLM outperforms state-of-the-art general-purpose VLMs by a significant margin (up to 73%), highlighting a fundamental gap between general-purpose multimodal reasoning and domain-specific layout understanding.
1 Introduction
THEIA treats analog GDSII layouts as visual entities for natural-language querying and AI-assisted analysis, addressing the difficulty of extracting circuit-level meaning from complex physical geometries. Its dataset and domain-specific VLM benchmark target a gap between general-purpose multimodal models and analog-layout understanding.
Introduction
- Off-the-shelf VLMs struggle with abstract, multilayered GDSII geometries, with performance decreasing on medium and complex tasks.The five identification tasks cover single-device identification, simple and complex topology recognition, and device counting in simple and complex layouts.Source: §1 · p. 2, Figure 2 · p. 2
- GDSII captures physical parasitics that affect manufactured-circuit behavior, but extracting semantic insights from layouts has required exhaustive manual inspection or rigid, computationally expensive scripts.The introduction frames the analog design flow as a schematic-based front end followed by a GDSII-based back end for physical implementation, parasitic extraction, and final verification.Source: §1 · pp. 1–2
- Fine-tuned models outperform strong zero-shot baselines by up to 73% accuracy.The benchmark evaluates five realistic tasks spanning multiple circuit topologies and complexity levels.Source: §1 · pp. 2–3
- THEIA pairs analog layout images with question-answer conversations and provides an open-source training and evaluation pipeline for reproducible VLM fine-tuning and benchmarking.Its tasks range from single-device identification to base-topology recognition and more complex reasoning over full circuit layouts.Source: §1 · p. 3
- The work adapts a pretrained VLM specifically for analog-layout understanding and evaluates it on DRC-free, LVS-compliant GDSII layouts against state-of-the-art VLM baselines.The evaluation uses a large collection of representative analog circuits.Source: §1 · pp. 2–3
2 Related Work
Related work covers front-end AI for circuit design and back-end placement, verification, and analog-layout methods; the authors identify direct analysis of analog-circuit GDSII as an unaddressed target.
Front-end
Back-end
- Back-end work addresses placement prediction and verification, but not layout analysis and identification; examples include ACDC, SOLOMON, LLM-HD, and DRC-Coder [23].Source: §2 · p. 3
- Analog layout methods include parameterized generators BAG, algorithmic ALIGN and MAGICAL, and AI-driven DNN [26], Bayesian [28], reinforcement-learning [29] [30] [31] [32], and graph-neural-network [30] approaches.Source: §2 · p. 3
- The authors state that, to their knowledge, their work is the first to directly target analog circuits’ GDSII representation, enabling analog AI assistants.Source: §2 · p. 4
3 THEIA Dataset Creation and Benchmark Methodology
THEIA’s methodology converts analog-circuit GDSII layouts into task-specific Q&A datasets, fine-tunes VLMs, and evaluates their answers on held-out test data.
3.1 THEIA Dataset Creation
- Real GDSII layouts are converted into task-specific Q&As and train, validation, and test splits, with all layouts passing DRC and LVS checks.Metadata extraction records device types and counts, while each layout is rendered as a PNG; all Q&As from one layout stay in a single split to prevent image or layout overlap across splits.Source: §3.1 · pp. 4–5
3.2 VLM Fine-tuning
- Single-task fine-tuning retrieves layout images, adapts them to the vision encoder’s patch format, serializes Q&A exchanges, and predicts answer tokens with masked next-token cross-entropy loss.Chat serialization marks user and assistant roles and the end of sequence, conditioning responses jointly on the visual input and query.Source: §3.2 · pp. 5–6
3.3 VLM Deployment
- For each test Q&A, the fine-tuned VLM answers from the question and layout image, and an LLM judge assigns a binary Match/No Match based on semantic consistency with ground truth.The ground-truth answer is withheld from chat serialization and is used by the judge to compare against the model’s prediction.Source: §3.3 · p. 6
4 Results
Results describe a three-tier, five-task evaluation of analog-layout understanding and report performance, transfer, and robustness analyses. Task-specific fine-tuning substantially improves performance, while data scaling and alternative evaluation forms probe robustness.
4.1 Experimental Setup
- THEIA contains 77,354 Q&As across five tasks and three complexity tiers, spanning 20,000 single-component, 5,894 base-circuit, and 4,140 mixed-layout variants.Layouts use the SkyWater 130nm PDK and satisfy DRC and LVS; tasks progress from device identification through circuit counting and topology recognition.Source: §4.1 · pp. 7–8
4.2 Experimental Results
- On held-out ALIGN layouts from the same PDK, THEIA-fine-tuned InternLM-4KHD achieves 89.5% (34/38) topology identification and 73.7% (84/114) component counting.For closed-set Tasks A and B, Inspector CNN baselines achieve 97% and 91%; these are task-specific references rather than open-ended VLM comparisons.Source: Table 5 · p. 8, §4.2 · p. 9
4.2.1 Evaluation Robustness
- Exact match and normalized string match coincide with LLM-judge Pass@1; open-ended evaluation leaves Task A unchanged, reduces Task B by one percentage point, and improves counting-task accuracy.Task-specific count accuracy coincides for Tasks C and D, while Task E achieves 89% topology-set F1; Task B’s 16.7% majority-class baseline is below its 84% fine-tuned-VLM accuracy.Source: Table 6 · p. 9, §4.2.1 · pp. 9–10, Table 8 · p. 10
4.2 Experimental Results
- 100% on Task A, 84% on Task B, 83% on Task C, 63% on Task D, and 81% on Task E: task-specific fine-tuned InternLM-4KHD accuracy.Zero-shot InternLM-4KHD and GPT-5.2 perform substantially lower on Medium and Hard tasks, while GPT-5.2 remains competitive on Task A.Source: §4.2 · p. 9
- 65–81 percentage-point gains across all five tasks raise Qwen2.5-VL-7B accuracy from 19%–24% zero-shot to 85%–100% after fine-tuning.The before-and-after comparison uses identical splits, prompts, answer format, and deterministic decoding, isolating adaptation effects across backbones.Source: §4.2 · p. 9
4.2.1 Evaluation Robustness
- As training data rises from 25% to 100%, Task A saturates immediately, Tasks B and C improve monotonically, and Tasks D and E trend upward less smoothly.The evaluation protocol and held-out test set remain fixed across data fractions.Source: Figure 6 · p. 10, §4.2.1 · p. 10
5 Conclusions
THEIA pairs verified GDSII layouts with visual question-answering tasks to benchmark analog-layout understanding. Its results show that task-specific fine-tuning improves performance where unadapted general-purpose VLMs struggle, and its released resources support reproducible research.
5 Conclusions
- 65–81 percentage points: under a controlled protocol, fine-tuning Qwen2.5-VL-7B improves accuracy across five tasks and transfers across backbones.The passage reports the improvement as a range and does not specify individual task values.Source: §5 · p. 10
- THEIA pairs verified GDSII layouts with visual question-answering tasks covering component and topology identification, device counting, and mixed-topology reasoning.Layouts come from human-designed circuit templates and are validated through DRC and LVS checks, supporting evaluation of layout semantics rather than raw geometry alone.Source: §5 · p. 10
- Unadapted general-purpose VLMs struggle as task structure and ambiguity increase, while task-specific fine-tuning substantially improves performance.Source: §5 · p. 10
- Released data, a fine-tuning pipeline, and evaluation implementations support reproducible research into multimodal analog-layout reasoning and future interactive CAD tools.Source: §5 · p. 10
7 Limitations
THEIA covers representative analog components and topologies but not the full diversity of analog designs, and its evaluation is limited to one technology node without cross-node validation.
7 Limitations
- THEIA does not capture the full diversity and variability of analog designs despite covering representative components and topologies.Its coverage ranges from basic building blocks to more complex systems.Source: §7 · p. 10
- The evaluation focuses on the SkyWater 130 nm PDK and does not explicitly assess cross-node generalization.Source: §7 · p. 10
- Cross-node validation is left to future work because technology migration is costly and infrequent in common industry practice.Source: §7 · p. 10
A Appendix · A.1 Fine-tuning & Deployment Configuration
The appendix identifies the fine-tuning and inference configuration used for InternLM-XComposer2-4KHD.
A.1 Fine-tuning & Deployment Configuration
- Table 9 reports the fine-tuning and inference parameters used to configure and evaluate InternLM-XComposer2-4KHD.The supplied passages identify the table's scope but do not provide its parameter values.Source: §A.1 · p. 15, Table 9 · p. 15
A.2 Dataset Structure and Statistics
THEIA organizes layout files, rendered images, metadata, and task-specific Q&As across circuit categories and variants. Its 77 354 Q&As and geometric and device-level statistics capture dataset diversity and complexity.
A.2 Dataset Structure and Statistics
- Routing density is defined by routed-metal polygon area relative to layout bounding-box area, alongside metrics capturing structural and parametric diversity.The numerator aggregates polygon areas across SkyWater routed-metal layers; collectively, these measures support realistic and challenging reasoning tasks.Source: §A.2 · p. 16
- Four directories organize single-component layouts, base topologies, mixed layouts, and task-specific Q&As; circuit layouts are further organized by variant.Each variant directory contains a GDS file, rendered PNG, and associated metadata file.Source: §A.2 · p. 15, Figure 7 · p. 16
- 77 354 Q&As comprise 19 997 easy, 33 369 medium, and 23 988 hard samples, with balanced circuit-type distributions for simple tasks and exclusively mixed topologies for hard tasks.Table 10 reports the distribution across training, validation, and test splits by task and complexity; the supplied passage does not provide the split-specific counts.Source: §A.2 · p. 15, Table 10 · p. 16
- Dataset statistics cover layout area, polygon and layer counts, aspect ratio, transistor width and length, finger count, and capacitor and resistor dimensions.Capacitor area is reported as a dataset-wide range.Source: §A.2 · pp. 15–16, Table 11 · p. 17
A.3 Additional Evaluation Analyses
Section A.3 documents task records, evaluation implementation, answer-option construction, and Task-B class-balance statistics; it notes that validation and evaluation results are reported in Section 4.2.1.
A.3 Additional Evaluation Analyses
- The subsection documents released task records and the evaluation implementation.Source: §A.3 · p. 17
- It also details answer-option construction and Task-B class-balance statistics.Source: §A.3 · p. 17
- Deterministic validation, LLM-judge agreement, and open-ended evaluation are reported in Section 4.2.1.Source: §A.3 · p. 17
A.3.1 Released Data and Evaluation Implementation
The release packages task-specific conversations and fine-tuning and evaluation code, including deterministic choice scoring and LLM judging. Task B is class-balanced, and its majority-class baseline trails the fine-tuned VLM.
A.3.1 Released Data and Evaluation Implementation
- The repository provides five task-specific train/validation/test JSON files pairing relative PNG paths with two-turn user/assistant conversations.Source: §A.3.1 · p. 17
- The release includes InternLM LoRA fine-tuning and preprocessing, plus deterministic A–F scoring and model-agnostic LLM judging.The scorer removes the image placeholder and compares the final answer letter with ground truth; the judge logs prompts, predictions, verdicts, and disagreements, using temperature 0, top-p 1, and at most six generated tokens.Source: §A.3.1 · p. 17
- For counting tasks C/D, correct counts are retained while distractors are sampled nearby first, candidate order is shuffled, and the number and position of answer options vary.If nearby integers do not suffice, valid random counts are sampled.Source: §A.3.1 · p. 17
- Task B’s six topology families each occupy 16.3%–17.1% of every split, while the majority-class Gate Driver baseline scores 50/300 = 16.7% versus 84% for the fine-tuned VLM.Table 14 reports class counts and the majority-class baseline.Source: §A.3.1 · p. 17, Table 14 · p. 18
NeurIPS Paper Checklist … 16. Declaration of LLM usage
The checklist reports the authors’ declarations on the paper’s scope, reproducibility, ethical considerations, and use of LLMs. It states that THEIA’s dataset and baseline are open-source and that LLMs are used in the methodology, but not in developing the dataset concept or baseline methodology.
NeurIPS Paper Checklist; 1. Claims; 2. Limitations; 3. Theory assumptions and proofs
- The checklist is intended to support responsible research and requires Yes, No, or N/A answers with brief justifications; the authors say the abstract and introduction accurately reflect the paper’s claims.The authors say limitations are discussed in Section 7 and that the work presents no novel theoretical results.Source: NeurIPS Paper Checklist · p. 20, §1 · p. 20, §2 · p. 21
4. Experimental result reproducibility; 5. Open access to data and code
- The authors say the paper provides information to assess dataset creation and reproduce baseline results, and that the dataset and baseline code are released open-source.The reproducibility details are reported in Sections 3 and 4 and Appendix A.Source: §4 · p. 22, §5 · p. 22
6. Experimental setting/details; 7. Experiment statistical significance
- The authors say Sections 3 and 4 and Appendix A provide information needed to assess data creation and reproduce VLM fine-tuning baseline results.They also state that the dataset is fully characterized by statistical measures; the passage does not specify error bars or significance tests.Source: §6 · p. 23, §7 · p. 23
8. Experiments compute resources; 9. Code of ethics; 10. Broader impacts
- The authors report that Section 4 describes the computer resources used for dataset generation and experiments, and that the research conforms to the NeurIPS Code of Ethics.For broader impacts, they point to the abstract, introduction, related work, and conclusions as discussing how THEIA can foster research in analog circuit design automation.Source: §8 · p. 24, §9 · p. 24, §10 · p. 24
11. Safeguards; 12. Licenses for existing assets; 13. New assets
- The authors state that the dataset and baseline pose no significant threats of misuse, and report that people involved in creating, curating, and disseminating assets are credited.They answer N/A on safeguards and Yes on crediting creators and asset documentation.Source: §11 · p. 25, §12 · p. 25, §13 · p. 25
14. Crowdsourcing and research with human subjects
- The authors state that the work involves neither crowdsourcing nor research with human subjects.The same justification is given for the human-subjects and IRB checklist items.Source: §14 · p. 26
16. Declaration of LLM usage
- LLMs are used in the methodology and experimental validation, but not in developing the dataset concept or baseline methodology.The authors identify Sections 3 and 4 as describing the LLM roles, respectively.Source: §16 · p. 27