PaperlayerView exact revision on arXiv

Source-linked AI summary

LLM-Driven, Datasheet-Aware Automated Hardware Compatibility Verification for Early-Stage, Pre-Schematic Embedded System Design

Haotian Qiao, Robert P. Dick

arXiv:2608.25217v1cs.AIeess.SY

TL;DR

DEVICES addresses early-stage hardware compatibility verification from datasheets and high-level connectivity descriptions, before detailed schematic implementation. It uses traceable, modular verification stages with task-specific criteria and achieves 97.5% compatibility-verification accuracy across seven designs.

  • Problem

    Early-stage designers need to determine whether interconnected components satisfy electrical, communication-interface, and environmental constraints before schematic capture, simulation, and implementation.

  • Method

    DEVICES constructs connectivity-aware verification tasks, selects relevant engineering properties using formalized domain-oriented criteria, and preserves intermediate artifacts for traceable compatibility evaluation.

  • Results

    97.5% compatibility-verification accuracy was achieved across seven embedded-system designs, compared with 14.9% for one-shot prompting without criteria and 62.7% with criteria.

  • Takeaways & Limitations

    Task-specific context construction and explicit verification criteria improve transparency, reliability, interpretability, and scalable specification-based hardware compatibility verification.

  • Takeaways & Limitations

    One-shot LLM processing lacks independently verifiable interfaces between engineering evidence and generated verification operations, and LLMs may inconsistently follow specified procedures.

Abstract

from arXiv · show

We present an LLM-driven, datasheet-aware framework for early-stage hardware compatibility verification that identifies documentation-level interface incompatibilities based on hardware datasheets and high-level component connectivity descriptions. It does not require, and can therefore be used, before detailed schematic simulation and implementation. We view trustworthy LLM-assisted design automation not as directly generating answers from documents, but as transforming engineering information through traceable verification stages. Given hardware datasheets and high-level component connectivity descriptions, the framework constructs a design graph that captures device connectivity and shared interaction domains, retrieves only the engineering properties required by explicit, domain-oriented verification criteria , and generates deterministic scripts for compatibility evaluation. By decomposing compatibility analysis into modular stages and preserving intermediate results, the framework reduces context overhead, improves transparency and tractability, enables scaling, and avoids reliance on LLMs for numerical computation. Evaluated on seven embedded-system designs comprising 34 datasheets, our framework achieves 97.5% compatibility-verification accuracy and an 8.6 times reduction in input context size compared with ``upload-and-query'' workflows. These results demonstrate the feasibility of LLM-assisted, specification-based hardware compatibility verification at an early design stage, as well as the need for, and substantial benefits of, modular task decomposition, formalized verification criteria, and task-aware compact context construction.

I. INTRODUCTION

DEVICES targets early, documentation-level interface compatibility verification, addressing the omissions, context burden, and lack of traceability that limit straightforward LLM document prompting.

  • Motivation: Late-discovered component incompatibilities can cause difficult diagnosis, substantial rework, longer development, and higher costs, including PCB respins.Industry data reports an average of 2.9 respins per project, costing $44,000 per respin on average.
  • Framework scope: DEVICES evaluates electrical, communication, power, and environmental compatibility before detailed schematic implementation using datasheets and high-level connectivity descriptions.Its scope excludes software, packaging, and application-specific functional compatibility.
  • Motivation: Manual datasheet review is tedious and error-prone because designers may search hundreds of pages for relevant specifications for each tentative system design.The framework automates this review for systems such as environmental sensor nodes.
  • Limitations of upload-and-query: Long, sparse, heterogeneous contexts scatter compatibility specifications and can exceed effective context lengths, causing LLMs to omit required engineering properties.Five datasheets produced approximately 223k input tokens, yet GPT-5.4 omitted multiple required properties; omissions increased with additional datasheets.
  • Framework approach: DEVICES retrieves only properties required by target criteria, organizes them into compact task-specific contexts, and decomposes verification into controlled stages.Structured instructions operate on externally stored, traceable properties, while deterministic local processing applies the generated operations.
  • Verification criteria: Formalized domain-oriented criteria define required properties, procedures, and rules so users need not manually construct detailed compatibility prompts.New interaction or hardware types can be added by supplying corresponding criteria without changing the underlying framework.

Problem Definition

The paper defines early-stage interface compatibility verification as determining whether connected components satisfy documented electrical, power, communication, and environmental constraints.

  • Problem Definition: The objective is to determine automatically whether interconnected components satisfy engineering constraints required for intended integration from datasheets and natural-language connectivity descriptions.Evaluation occurs before schematic capture, simulation, implementation, or deployment.
  • Problem Definition: Compatibility analysis covers electrical and power, communication-interface, and environmental dimensions at interfaces specified by high-level connectivity descriptions.The framework identifies potential incompatibilities across these three dimensions.
  • Problem Definition: DEVICES represents connectivity and decomposes compatibility evaluation into explicit intermediate representations for controlled reasoning and procedural generation.Intermediate artifacts are preserved to support transparent and auditable verification.
  • Terminology: A modular task is a self-contained subtask focused on one verification domain and only its relevant engineering properties and procedures.This isolates domain-specific compatibility checks within the broader system-level analysis.
  • Terminology: Engineering properties include quantitative specifications, qualitative attributes, and associated semantics such as units, operating conditions, and device variants.Examples include supply voltage, maximum clock frequency, protocol, operating mode, and operating temperature.
  • Terminology: A verification criterion specifies the required properties, procedure, and deterministic compatibility rule for a particular interaction or compatibility domain.Power and communication-interface compatibility use different criteria because their required properties and evaluation procedures differ.

Contributions

DEVICES contributes a connectivity-aware, modular verification framework for early architecture-stage compatibility screening, while remaining limited by its current reliance on textual specifications.

  • Framework contribution: DEVICES transforms interface-level compatibility checking into explicit engineering procedures by using a design graph to segment interactions into domain-specific tasks.Only criteria relevant to each interaction are evaluated, so workload scales with component interactions rather than total specification volume.
  • Scope boundary: The framework currently focuses on textual hardware specifications and does not support hardware whose key specifications are primarily conveyed through figures.Switching regulators are an example because efficiency may depend on graphs involving output voltage, load current, and switching frequency.
  • Scope boundary: Technical diagram interpretation is outside the current implementation but could be integrated through figure-understanding and operating-point reasoning tools.The paper identifies diagram interpretation as an active research area.
  • Relation to prior work: DEVICES targets an earlier architecture-design stage than schematic-review systems that require concrete pin-level or EDA representations.It complements lower-level verification by screening documentation-level incompatibilities before detailed schematic design and simulation.

III. METHOD

DEVICES addresses scalability in LLM-guided hardware compatibility verification by decomposing analysis into connectivity-aware, domain-oriented tasks with compact, task-specific contexts. Its pipeline uses formalized criteria to guide retrieval and deterministic script generation.

  • DEVICES decomposes compatibility verification into connectivity-aware, domain-oriented modular tasks that evaluate only properties relevant to each task.This makes reasoning scale with relevant component interactions rather than the entire hardware knowledge space.
  • The framework constructs a design graph from high-level connectivity descriptions to segment component interactions into domain-specific verification tasks.The graph supports subsequent retrieval and verification for each netnode.
  • DEVICES hierarchically retrieves required engineering properties and generates scripts that execute the specified compatibility verifications.The pipeline assembles retrieved properties, evaluation procedures, and deterministic compatibility rules into task-aware contexts.
  • Formalized verification criteria specify what to verify, which properties to retrieve, and how to evaluate them through prompting templates.These criteria address the inconsistency and incompleteness of ad hoc prompt construction.
  • Explicit evaluation rules transform implicit engineering practices into reproducible procedures that can be executed through generated scripts.This separates engineering criteria from the LLM’s numerical computation.

2) Required Engineering Properties:

DEVICES represents system interactions as design-graph netnodes and retrieves engineering properties according to reusable, domain-specific compatibility requirements. This structure limits each verification task to relevant component interactions and properties.

  • Required Engineering Properties: Communication verification criteria can be defined at the interface level and reused across devices supporting SPI, UART, or I2C.Communication interfaces share common electrical and protocol-level compatibility requirements.
  • Required Engineering Properties: Power compatibility criteria depend on device type because power-conversion architectures impose different engineering constraints.LDO evaluation accounts for dropout voltage, whereas switching-regulator efficiency depends on multiple operating conditions.
  • Design Graph: The design graph decomposes system compatibility into connectivity-aware subtasks so each task considers only relevant component interactions.This reduces unnecessary context and mitigates LLM attention dilution.
  • Design Graph: Power, data, and environment netnodes represent shared supply domains, communication channels, and physical operating environments.Power nodes distinguish source and sink relationships, data nodes group peers by interface, and environment nodes group devices sharing conditions.
  • Design Graph Construction: Figure 2’s example graph represents power, communication, and shared operating-condition relationships among five devices.The graph is generated from the high-level connectivity description.
  • Design Graph Construction: A local deterministic script validates structured node and edge definitions before constructing and caching the design graph.The LLM does not directly generate the complete graph.

1) High-Level Component Connectivity Description:

DEVICES requires a high-level connectivity description that identifies the component relationships needed to instantiate power, data, and environment netnodes. Preparing these descriptions still requires some manual datasheet inspection.

  • High-Level Component Connectivity Description: Power connectivity descriptions specify device power pins and corresponding power domains, which may differ within one component.An EKG sensor may have separate digital-circuit and LED power domains with different voltages and sources.
  • High-Level Component Connectivity Description: High-level connectivity descriptions still require hardware-specific information that users may obtain by inspecting datasheet pin functions and configurations.This effort is lower than designing a detailed schematic but remains manual.
  • High-Level Component Connectivity Description: DEVICES associates retrieved engineering properties with the devices and interactions represented in the design graph.Retrieval is performed for each device participating in a target netnode using the applicable criteria’s required property descriptions.

1) Hierarchical and Task-Aware Retrieval:

DEVICES uses hierarchical, task-aware retrieval to narrow property searches before assigning candidate properties to verification requirements. It preserves property fidelity, model applicability, and efficiency through structured references, filtering, caching, and bounded queries.

  • Hierarchical and Task-Aware Retrieval: Hierarchical retrieval narrows the property search space before the LLM evaluates candidate relevance for a target compatibility analysis.This reduces unnecessary information processing, query overhead, and token costs.
  • Hierarchical and Task-Aware Retrieval: DEVICES references retrieved properties by unique knowledge-graph node IDs instead of regenerating their contents.This avoids unintended changes to numerical values, conditions, or descriptions.
  • Hierarchical and Task-Aware Retrieval: Model-aware filtering prevents combining specifications from different device variants when one datasheet covers multiple models.Only properties associated with the applicable model are retrieved.
  • Hierarchical and Task-Aware Retrieval: 20 candidate properties are provided per retrieval query to maintain compact contexts while preserving retrieval accuracy.The framework also uses caching to avoid repeated retrieval operations.
  • Hierarchical and Task-Aware Retrieval: Retrieval is independent of connectivity topology once the verification domain is determined, enabling reuse by device-domain context.The relevant properties depend on the device model and domain rather than the identities of connected devices.
  • Hierarchical and Task-Aware Retrieval: Cache hits bypass LLM retrieval for matching device-domain contexts and reduce their LLM token consumption to zero.

D. Verification Script Generation and Constraint Evaluation

DEVICES generates deterministic scripts from retrieved engineering properties and evaluates interface, environmental, and power compatibility using domain-specific constraints. Cascaded power demands are propagated across connected power netnodes while contexts remain compact and task-specific.

  • Script Generation: DEVICES supplies retrieved engineering properties to an LLM, which generates Python scripts for compatibility constraints while deterministic scripts perform numerical operations.Numerical operations include unit conversion, range comparison, and mathematical computation.
  • Data Compatibility: Data compatibility evaluates communication modes, logic voltage levels, clock-frequency overlap, and applicable interface-specific electrical requirements.I2C additionally requires sink-current capability verification.
  • Environment Compatibility: Environmental compatibility requires a non-empty intersection of all participating devices’ modeled operating ranges, including temperature, humidity, and pressure.Data and environment netnodes are evaluated independently because their constraints do not propagate across different netnodes.
  • Power Compatibility: Power compatibility evaluates voltage compatibility, power capacity, and power-delivery relationships across transitively connected power netnodes.Downstream demand must be propagated upstream through the power-delivery tree.
  • Power Model: DEVICES models regulators as both power sinks and sources, while sensors and MCUs consume power but do not supply downstream components.This unified representation permits input or output power to be zero; only regulators and motor drivers currently act as both.
  • Power Constraints: For each power netnode, the maximum peak demand across downstream sink modes must not exceed the supplying device’s maximum output capability.Power calculations conservatively account for conversion losses and may exceed some systems’ actual consumption.

A. Evaluation Setup

The evaluation covers seven embedded-system designs using 34 datasheets across representative IoT domains. DEVICES is assessed for strict end-to-end compatibility accuracy, context size, and property faithfulness against manually established ground truth.

  • Evaluation Designs: The evaluation uses seven embedded-system designs and 34 hardware datasheets totaling 1210 pages across environmental sensing, health monitoring, and location tracking.Multiple variants within one datasheet are evaluated independently but counted as a single datasheet.
  • Evaluated Compatibility Domains: The designs evaluate UART, SPI, and I2C communication, temperature, humidity, and pressure environments, plus several power modules.Power modules include solar panels, batteries, power adapters, and LDOs.
  • Scope Boundary: Switching regulators such as buck converters are unsupported because their efficiency is typically represented by figures, while DEVICES does not support figure understanding.This limits the current evaluation to supported textual specifications and numerical power descriptions.
  • Evaluation Metrics: DEVICES is evaluated from three perspectives: compatibility verification accuracy, context size, and property faithfulness.These measures respectively address evaluation correctness, LLM input overhead, and grounding in supplied engineering properties.
  • Evaluation Metrics: Compatibility verification accuracy requires correct properties, procedures, and conclusions; any error makes the constraint incorrect.The metric therefore measures end-to-end verification correctness rather than only the final compatibility decision.
  • Ground Truth: Ground truth is established manually from datasheets and connectivity descriptions by identifying required properties, procedures, and expected compatibility conclusions.The ground truth is independently evaluated for accuracy.

C. Baseline

The baselines test direct document prompting without criteria and one-shot prompting with criteria but without structured property retrieval. Results show that criteria improve accuracy, while compact task-aware context remains necessary for reliable verification.

  • Baseline Design: The first baseline directly provides all relevant datasheets and connectivity descriptions to an LLM without manually constructed verification criteria.It represents a common one-shot document-prompting workflow.
  • Baseline Design: The second baseline adds DEVICES’ verification procedures, required properties, and compatibility rules but omits structured property retrieval.This isolates the contribution of verification criteria from task-aware context refinement.
  • Results: 97.5% accuracy is achieved by DEVICES, compared with 14.9% without verification criteria and 62.7% with verification criteria.The comparison spans seven designs, each containing at least five distinct hardware components.
  • Results: The baseline without criteria frequently omits required criteria and engineering properties, with inconsistent evaluation behavior across runs.The results indicate that relying on an LLM to infer complete requirements from lengthy documentation is insufficient.
  • Results: Explicit criteria improve accuracy by providing unambiguous procedures and semantic anchors for identifying relevant properties.Generated scripts also follow the provided procedures more consistently after criteria are added.
  • Results: The criteria-enhanced baseline still combines specifications from incompatible operating conditions or device variants in long, sparse document contexts.This demonstrates that explicit criteria alone are insufficient without task-aware context construction and selective property exposure.

2) Context size:

DEVICES reduces per-query context by decomposing compatibility verification into task-specific interactions and properties, while preserving traceable, scalable reasoning. This structured approach remains valuable beyond current LLM context limitations because it supports transparent, reliable, criteria-driven evaluation.

  • Context size:: DEVICES restricts each verification query to task-relevant component interactions and engineering properties, reducing irrelevant information exposed to the LLM.The query context is determined primarily by the involved connectivity relationship and associated properties rather than the total hardware specification content.
  • Context size:: For DEVICES, the reported context size aggregates all netnode queries, while each individual query remains substantially smaller than the aggregate.The maximum context size is approximately 12.6k input tokens and the average context size per query is approximately 4.7k input tokens.
  • Context size:: DEVICES preserves property provenance: its generated scripts use only properties available in the provided contexts, as do both baselines.Observed accuracy differences therefore primarily reflect property selection, condition consistency, and evaluation procedures rather than external information.
  • Context size:: As designs grow, DEVICES adds modular verification queries instead of proportionally expanding the context of each query.This allows verification workload to scale with relevant component interactions rather than requiring one reasoning process over the entire hardware specification space.
  • Context size:: Context construction remains valuable even with larger future context windows because it provides task-specific evidence, traceability, and well-defined engineering criteria.Future LLM improvements can enhance localized extraction, property identification, and procedure-generation tasks without replacing the structured architecture.
  • Context size:: Across seven embedded-system designs, DEVICES achieved 97.5% compatibility-checking accuracy and reduced input context size by 8.6× versus one-shot full-PDF prompting.The results support early, specification-based compatibility verification before detailed schematic design and simulation.
Loading 2608.25217v1…