SyntheticHLS: Building Diverse Synthetic High-Level Synthesis Datasets using LLMs

I. INTRODUCTION

• Existing HLS datasets emphasize configuration diversity over coverage of distinct designs, leaving generalization to unseen designs a challenge.Zero-shot LLM designs are described as small, simple, and unrepresentative of parameterizable real-world designs.Source: §I · p. 1
• SyntheticHLS uses iterative LLM-guided mutation to optimize selected design metrics, producing designs beyond naive zero-shot generation.The framework also introduces quantitative metrics for structural complexity and design-space scalability as mutation objectives.Source: §I · p. 1
• Cross-validation shows iteratively mutated SyntheticHLS designs provide the most generalizable training corpus among benchmark and zero-shot designs.The study also characterizes metric-guided trajectories and resulting design-space diversity.Source: §I · p. 2
• SyntheticHLS releases an open-source Python package with a parallel execution backend for scalable dataset generation.Source: §I · p. 2

II. PRIOR WORK

• Common curated and aggregated HLS datasets draw from narrow-domain benchmarks or enumerate many points from few source designs, leaving generalization to unseen designs challenging.Source: §II · p. 2
• Iceberg explores zero-shot LLM-generated HLS source code but constructs design spaces separately through static analysis, limiting its systematic generation study.Its first generated output is used, and prompting only nudges application diversity, loop counts, or memory use.Source: §II · p. 2
• Systematic cross-validation across benchmark and synthetic training and testing datasets has been absent from prior HLS QoR work.Source: §II · p. 2

III. SYNTHETICHLS FRAMEWORK

• SyntheticHLS separates generation into seed design generation and iterative mutation, optimizing collected designs for diversity, code complexity, and Pareto-frontier scaling.Source: §III · p. 2

A. Seed Design Generation

• Seed designs pair an HLS kernel with a self-contained testbench and OptDSL design-space specification, then retain only compilable, functional, synthesizable designs.Prompts target application-oriented kernels across domains, with multiple interacting stages and explicit dataflow, memory, and control dependencies.Source: A. Seed Design Generation · pp. 2–3

B. Iterative Design Mutation

• Each mutation iteration generates candidates, checks correctness and synthesizability, and selects a passing design as the next seed.The target-metric and iteration schedule is user-specified; the resulting set is called Final designs.Source: B. Iterative Design Mutation · p. 3

1) Feedback-Guided Mutation:

• Mutation prompts combine current design files, the iteration's target metric, and evaluation reports; repeated LLM sampling produces candidates for selection.Candidate count and mutation schedule are user-definable; passing candidates are chosen by target metric, with average function lines of code breaking ties, while all-fail batches trigger repair or trajectory termination.Source: 1) Feedback-Guided Mutation · p. 3, Figure 2 · p. 3

2) Candidate Selection:

• Passing candidates are ranked by the target metric; ties go to the design with the highest average function LoC.A single passing candidate advances directly.Source: 2) Candidate Selection · p. 3
• When no candidate passes, the LLM repairs each failed design using its source, error messages, and fix-oriented instructions before reevaluation.If none of the repaired designs passes, that design’s iteration terminates early.Source: 2) Candidate Selection · p. 3
• Repairing only when every candidate fails may bias the advancing designs toward less diverse or complex candidates, while repairing every failure would roughly double LLM and synthesis invocations.The authors leave open whether repairing all candidates would increase corpus diversity.Source: 2) Candidate Selection · p. 3

C. Quantifying HLS Designs

• The framework uses structural-complexity and design-space metrics to guide targeted improvements during each mutation iteration.The metrics describe source-code organization and design scalability and parameterizability.Source: C. Quantifying HLS Designs · p. 3

1) Structural Complexity:

• Static analysis of C/C++ source extracts functions and a call graph to measure function count, maximum call-chain depth, and average lines of code per function.These metrics reflect modular decomposition, hierarchy, and the distribution of code across functions.Source: 1) Structural Complexity · p. 4

2) Pareto Score:

• A design space pairs HLS source with pragma options; its scalability depends on a Pareto frontier spanning many points across a wide latency–resource range.The paired OptDSL specification supports parameterizable and grouped array-partitioning, loop-unrolling, and pipelining pragmas.Source: 2) Pareto Score · p. 4
• The Pareto Score combines frontier extent with a penalty for its largest gap; lower scores indicate a smoother, denser, more continuous frontier.The score is minimized when selected as the current iteration’s optimization target.Source: 2) Pareto Score · p. 4
• For each candidate, the procedure samples 64 design points by default, runs HLS synthesis, and computes LUT–latency and FF–latency Pareto frontiers for scoring.The default sample count is chosen to keep synthesis runtime tractable; resulting summaries feed the mutation loop.Source: 2) Pareto Score · p. 4

D. Extensible API and Parallel Backend

• Users can target different domains and tune seed counts, mutation schedules, and candidates generated per iteration.SyntheticHLS is implemented as an open-source, extensible Python package.Source: D. Extensible API and Parallel Backend · p. 4
• At the design level, separate fixed-size thread pools manage LLM inference, C simulation, and HLS synthesis, bounding concurrency for each tool type.At the run level, another pool executes seed generation and mutation across LLM backends and application domains.Source: D. Extensible API and Parallel Backend · p. 4
• A user-specified job budget is divided across active runs, while Pareto-score computation dispatches additional HLS tool calls in parallel through HLSFactory.This forms the third level of the parallel execution strategy.Source: D. Extensible API and Parallel Backend · p. 4

IV. EVALUATIONS

• One evaluation cross-validates a deep-learning QoR proxy model trained on synthetic designs and common HLS benchmarks.The evaluation compares model behavior across those design sources.Source: §IV · p. 4
• A second evaluation measures how design metrics change over multiple iterations of a design trajectory.This complements the model cross-validation evaluation.Source: §IV · p. 4
• Together, the two evaluations are presented to demonstrate the quality of SyntheticHLS for generating generalizable synthetic HLS designs.Source: §IV · p. 4

• The fixed GNN-DSE QoR model predicts latency and resource use from source code across five benchmark and synthetic datasets.Each dataset contributes 4,096 design points with a 70%/30% train-test split; cross-dataset configurations are out-of-distribution.Source: A. Generalization of Synthetic HLS… · pp. 4–5
• Synthetic training transfers well to benchmark tests while benchmark-trained models transfer poorly to synthetic tests; Final is the most broadly transferable corpus.Final and Base perform comparably on benchmark tests, with Final slightly better for latency prediction; Final-trained models also transfer better to Base than the reverse.Source: Figure 4 · p. 5, A. Generalization of Synthetic HLS… · p. 6
• Base and Final each contain 68 unique kernels across 10 application domains, and SyntheticHLS can scale with additional LLM and synthesis compute.The combined synthetic datasets were collected in approximately 24 hours.Source: A. Generalization of Synthetic HLS… · p. 5
• All synthetic designs use gpt-oss-120B, and systematic evaluation across LLM families and scales remains future work.Source: A. Generalization of Synthetic HLS… · pp. 5–6
• Across seed generation and nine feedback-driven mutation stages, pass@8 for valid, synthesizable designs remains near 100% in most HLS domains and reaches 100% at the final stage, while pass@1 ranges roughly 30% to 80%.Source: Figure 5 · p. 6

Valid Design Pass Rates

• Average pass@1 rates across mutation stages range from 30% to 80%, while average pass@8 yields at least one valid mutated design for almost every source design.A valid sample compiles, simulates correctly, and synthesizes; multiple valid samples allow ranking and selection for the next stage.Source: B. Synthetic Design Trajectories · p. 6
• Engineering Simulation produces no valid mutated HLS designs after the first mutation stage because its sole valid seed’s mutations all fail validity.The domain had only one valid seed design, so its trajectory ended after none of its mutated samples were valid.Source: B. Synthetic Design Trajectories · pp. 6–7

Metric-Guided Mutation Trajectories

• Target metrics generally improve in their intended directions while non-target metrics remain stable or improve without substantial regression during guided mutation.Code complexity metrics increase and Pareto scores decrease; Fig. 6 shows trajectories under the specified optimization schedule.Source: B. Synthetic Design Trajectories · p. 7, Figure 6 · p. 7
• The mutation schedule is user-specified, with structural objectives ordered before Pareto-score objectives; non-target stability is prompted rather than enforced as a hard constraint.The authors identify dynamic or learned schedules as a promising extension.Source: B. Synthetic Design Trajectories · p. 7

Individual Design Metrics

• Final mutated designs have more diverse program-structure distributions and slightly more diverse latency and resource-usage distributions than synthetic seeds.Fig. 7 compares the distributions of nine metrics spanning post-synthesis QoR and program structure.Source: C. Synthetic Design Dataset… · p. 7, Figure 7 · p. 7
• The authors hypothesize that structural diversity, rather than the marginally higher QoR-metric diversity, enables better QoR-proxy generalization, but say further evaluation is needed.They also flag design-space size and quality via Pareto metrics as aspects not captured by these distributions.Source: C. Synthetic Design Dataset… · p. 7

Overall Design Space

• The PCA projection shows seed designs concentrated largely along one feature direction, whereas final mutated designs occupy a broader, more uniform space across the nine metrics.Each point represents one design described by the nine-metric feature vector.Source: C. Synthetic Design Dataset… · pp. 7–8, Figure 8 · p. 8
• The authors argue that equal-compute zero-shot sampling is unlikely to match mutation’s diversity because additional samples may remain bound to the unconditioned generation distribution.They hypothesize that feedback-guided mutation pushes designs beyond that distribution; Fig. 8 is consistent with this view.Source: C. Synthetic Design Dataset… · p. 8, Figure 8 · p. 8

Conclusion

• Cross-validation found iteratively mutated SyntheticHLS designs to be the most generalizable training corpus among the benchmark and zero-shot designs studied.Design characterization also found that feedback-guided mutation reaches a broader, more diverse design space than zero-shot generation alone.Source: §V · p. 8

Future Work

• Corpus size scales directly with the LLM and synthesis compute budget, and future directions include broader LLM evaluation, compute-matched zero-shot comparisons, adaptive schedules, and agentic mutation.The authors also identify inference-scaling comparisons against zero-shot baselines as future work.Source: §V · p. 8

Conclusion

• The authors hope open-sourcing SyntheticHLS will support extensions to mutation strategies, design metrics, and LLM evaluations, and broader HLS dataset curation.Source: §V · p. 8