Source-linked AI summary
ScaleHLS: A New Scalable High-Level Synthesis Framework on Multi-Level Intermediate Representation
Hanchen Ye, Cong Hao, Jianyi Cheng, Hyunmin Jeong, Jack Huang, Stephen Neuendorffer, Deming Chen
TL;DR
Large HLS designs challenge existing approaches that rely on single-level representations and designer-guided directives or rewriting. ScaleHLS combines multi-level representations, HLS-specific optimization interfaces, and automated design-space exploration; experiments report substantial benchmark improvements, including neural-network throughput gains of 1505.3× to 3825.0× over the stated baseline.
Problem
Large HLS designs are difficult to explore because effective code and directives may depend on designers, and correlated optimizations make their benefits hard to assess in isolation.
Source: §I · pp. 1–2Method
ScaleHLS uses multi-level HLS representations and a tunable transform and analysis library with automated DSE to search across optimization parameters.
Source: §I · p. 2Results
1505.3× to 3825.0× throughput speedups were reported for neural-network designs versus baseline designs without multi-level optimization.
Source: 1) Optimization Results · p. 12Takeaways & Limitations
Experiments report that ScaleHLS can optimize large, sophisticated HLS designs and improve performance and productivity across a benchmark set.
Source: §VIII · p. 13
Abstract
from arXiv · showhide
High-level synthesis (HLS) has been widely adopted as it significantly improves the hardware design productivity and enables efficient design space exploration (DSE). Existing HLS tools are built using compiler infrastructures largely based on a single-level abstraction, such as LLVM. However, as HLS designs typically come with intrinsic structural or functional hierarchies, different HLS optimization problems are often better solved with different levels of abstractions. This paper proposes ScaleHLS, a new scalable and customizable HLS framework, on top of a multi-level compiler infrastructure called MLIR. ScaleHLS represents HLS designs at multiple representation levels and provides an HLS-dedicated analysis and transform library to solve the optimization problems at the suitable levels. Using this library, we provide a DSE engine to generate optimized HLS designs automatically. In addition, we develop an HLS C front-end and a C/C++ emission back-end to translate HLS designs into/from MLIR for enabling an end-to-end compilation flow. Experimental results show that, comparing to the baseline designs without manual directives insertion and code-rewriting, that are only optimized by Xilinx Vivado HLS, ScaleHLS improves the performances with amazing quality-of-results -- up to 768.1x better on computation kernel level programs and up to 3825.0x better on neural network models.
I. INTRODUCTION
Large HLS designs are difficult to represent, optimize, and explore because their hierarchies call for different abstraction levels, while existing approaches often rely on single-level representations, manual rewriting, or designer-set directives. ScaleHLS addresses these challenges with a multi-level HLS flow, automated optimization and DSE, and front- and back-end support.
I. INTRODUCTION
- Existing approaches struggle with large HLS designs because single-level representations miss abstraction-level differences, manual optimization does not scale, and correlated optimizations complicate design-space exploration.Designers may also need to supply effective code and directives to obtain good design quality.Source: §I · pp. 1–2
- ScaleHLS is presented as the first end-to-end automated HLS flow using multiple abstraction levels to honor the structural or functional hierarchies of large designs.The authors describe this as their contribution to the best of their knowledge.Source: §I · p. 2
- ScaleHLS optimizes HLS designs holistically at graph, loop, and directive levels to address design-space complexity.These levels provide a systematic multi-level representation for HLS designs.Source: §I · p. 2
- Its HLS transform and analysis library makes optimization techniques callable and tunable instead of requiring manual code rewriting, establishing a foundation for automated DSE.The library is described as saving human effort.Source: §I · p. 2
- The automated DSE engine searches the latency-area Pareto frontier, with a QoR estimator rapidly evaluating discovered design points.The estimator is developed to evaluate the points found by the DSE engine.Source: §I · p. 2
- An HLS C front-end and synthesizable C/C++ emission back-end bridge multi-level IR and C-based HLS designs for an end-to-end compilation flow.The paper also identifies representation, optimization, and exploration as challenges ScaleHLS tackles.Source: §I · p. 2
II. BACKGROUND
MLIR provides hierarchical program representation through operations, blocks, regions, and dialects, alongside passes for transformation and analysis. Its affine and scf dialects represent structured control flow with different constraints, while lowering can convert these forms to unstructured control flow.
A. MLIR Framework
- MLIR organizes programs using SSA-style operations and nested blocks, regions, and functions, allowing arbitrary design hierarchy.Operations connect typed operands and results; regions contain blocks and are themselves contained by operations.Source: A. MLIR Framework · pp. 2–3
- MLIR dialects group related operations, attributes, and types, while passes perform analysis or transformation; conversions and translations bridge dialects and external representations.Lowering refers to reducing an IR's abstraction level.Source: A. MLIR Framework · p. 3
B. Relevant MLIR Dialects
- The affine dialect constrains loop bounds and conditions to affine values, whereas scf supports bounds and conditions expressed using any SSA values.Both dialects can represent structured control flow; unstructured branches can represent its flattened form.Source: B. Relevant MLIR Dialects · p. 3
C. Relevant MLIR Front-ends
ScaleHLS combines existing model front-ends with graph-, loop-, and directive-level IRs, optimization passes, a tunable library, and automated DSE. Its integration flow translates HLS C into MLIR and emits synthesizable C++ designs.
C. Relevant MLIR Front-ends
- ScaleHLS uses Torch-MLIR and ONNX-MLIR to parse PyTorch and ONNX models into MLIR representations.Torch-MLIR lowers PyTorch through torch dialect to affine dialect; ONNX-MLIR lowers onnx operations through krnl to affine.Source: C. Relevant MLIR Front-ends · p. 3
III. SCALEHLS FRAMEWORK OVERVIEW
- ScaleHLS compiles HLS C and frameworks such as ONNX and PyTorch into optimized, synthesizable HLS C/C++ designs.The framework is organized around representation, optimization, exploration, and integration.Source: §III · p. 3
A. Representation
- Graph-level IR represents tensor computation graphs, loop-level IR represents loop structures, and directive-level IR represents HLS structures and program directives.The levels use onnx, affine/scf, and customized hlscpp dialects, respectively.Source: A. Representation · p. 3
B. Optimization
- Optimization passes operate at suitable abstraction levels, and an analytical QoR estimator predicts latency and resource utilization from structured directive-level IR.The hierarchical IR is intended to reduce processing complexity and improve scalability.Source: B. Optimization · p. 4
C. Exploration
- A parameterized transform and analysis library exposes optimization techniques as tunable interfaces, which the extensible DSE engine uses to search a multi-dimensional Pareto frontier.Each design-space dimension corresponds to a tunable transform-pass parameter.Source: C. Exploration · p. 4
D. Integration
- The Clang-based HLS C front-end emits scf dialect, which is raised to affine where possible; the back-end emits synthesizable C++ and can also generate LLVM IR.LLVM IR enables software simulation and interfacing with LLVM-compatible tools such as Xilinx Vitis HLS.Source: D. Integration · p. 4
IV. SCALEHLS REPRESENTATION
- ScaleHLS applies transforms and analyses across multiple abstraction levels to explore broader design spaces and improve scalability.The representation section details graph-, loop-, and directive-level IRs.Source: §IV · p. 4
A. Graph-level IR
ScaleHLS uses tensor operations for graph-level optimization, structured loop IR for loop transformations, and directive-level representations to guide hardware generation. Function directives express task and operation parallelism, including a target initiation interval.
A. Graph-level IR
- Graph-level ScaleHLS IR uses ONNX tensor operations, enabling define-use analysis and graph transformations such as node merging without memory-dependence analysis.The example convolution consumes input and weight tensors and produces an output tensor.Source: A. Graph-level IR · p. 4
B. Loop-level IR
- Loop-level IR uses affine and scf dialects to preserve nested loop and conditional structure, enabling loop transformations such as tiling and efficient affine analysis.The paper illustrates the representation with an SYRK computation kernel.Source: B. Loop-level IR · p. 4
C. HLS Directives
- ScaleHLS represents function, loop, and array HLS directives to guide hardware generation and tune latency-area tradeoffs.Array partitioning uses affine-based memory typing to express partition factors and fashions such as cyclic and block.Source: C. HLS Directives · p. 4
1) Function Directives:
- The dataflow directive pipelines sub-functions for task parallelism, while the pipeline directive schedules operations across stages and can specify a target initiation interval.The function directive attribute stores dataflow and pipeline flags plus targetII; function inlining is performed directly in the IR.Source: 1) Function Directives · p. 5
2) Loop Directives:
ScaleHLS represents loop directives and array-memory choices in its IR, supporting transformations at loop and memory levels. Array partitioning is encoded through affine memory types, while resource directives specify memory kinds and interfaces.
2) Loop Directives:
- Loop dataflow and pipeline directives tune throughput and latency, while unrolling adds parallelism at a resource cost and merging adjacent loops improves locality and reduces control overhead.Perfectly nested loops can be flattened to further improve pipeline throughput and latency.Source: 2) Loop Directives · p. 5
3) Array Partition:
- Affine memory types encode array partition factors and fashions, enabling ScaleHLS to derive partition and physical indices from logical indices.For cyclic partitioning of dimension 0 by two, the partition and physical indices are d0 % 2 and floor(d0 / 2); unsupported downstream partition fashions are disallowed.Source: 3) Array Partition · p. 5, Figure 3 · p. 5
4) Array Resource and Interface:
- Resource directives map memory kinds to MLIR memory spaces and distinguish single-port, simple dual-port, and true dual-port on-chip memories.For array arguments and returned values, ScaleHLS determines an interface category such as AXI or naive BRAM according to the memory space.Source: 4) Array Resource and Interface · pp. 5–6
V. SCALEHLS OPTIMIZATION
ScaleHLS packages multi-level transforms and QoR estimation in a callable library, enabling targeted optimization and design-space exploration. Its dataflow legalization and function splitting expose tradeoffs between throughput and resource use.
V. SCALEHLS OPTIMIZATION
- Callable transforms can target specific IR locations and tune parameters, supporting DSE across combinations of passes; the integrated QoR estimator addresses unpredictable downstream HLS outcomes.The library supports applying different transform combinations to different targets in the IR.Source: §V · p. 6
1) Legalize Dataflow:
- Dataflow interval falls from 5t to 3t with conservative legalization and to 1t with copy-node insertion, while aggressive legalization consumes more computation and memory resources.The pass analyzes dependencies and legalizes the target function; inserted copies break bypass paths to enable finer-grained stages.Source: 1) Legalize Dataflow · pp. 6–7, Figure 4 · p. 7
2) Split Function:
- Function splitting groups procedures or inserted copy nodes from the same dataflow stage into sub-functions, and merging adjacent stages trades throughput for lower resource use.Merging every two adjacent stages yields a three-stage dataflow with a 2t interval and fewer resources than the five-stage copy-node version.Source: 2) Split Function · p. 7, Figure 4 · p. 7
B. Loop Transform Passes
- The discussion next turns to loop-transform passes, illustrated on the SYRK kernel as the transition from the graph-level representation to loop-level IR.The figure describes the SYRK computation kernel example.Source: B. Loop Transform Passes · p. 7
1) Loop Perfectization:
ScaleHLS transforms loop structure and ordering to support downstream optimization, using affine analysis for legal loop changes. Its directive-level passes then manipulate HLS-specific directives to improve design quality.
1) Loop Perfectization:
- Loop perfectization moves operations between loop statements into the innermost loop, placing them in a new affine.if and hoisting non-state-modifying operations out of the condition.This addresses imperfect loops that can interfere with loop tiling and prevent outer-loop flattening.Source: 1) Loop Perfectization · p. 7
2) Loop Order Optimization:
- Affine memory-dependency analysis permutes loops with loop-carried dependencies outward to increase their distance and can reduce achievable pipeline II and latency.An optional perm-map list specifies each loop's new position.Source: 2) Loop Order Optimization · p. 7
3) Remove Variable Loop Bound:
- The remove-variable-bound pass substitutes bounds with their minimum or maximum values and uses affine.if for conditional execution, enabling optimizations despite MLIR's non-rectangular-loop limitations.The transformation can increase total loop iterations, a side effect that later optimizations may offset.Source: 3) Remove Variable Loop Bound · p. 7
4) Loop Tiling:
- Loop tiling improves data locality and accommodates limited on-chip buffers; affine analysis checks legality and the DSE engine can tune tile size.In the SYRK example, the i-loop is tiled by a factor of two.Source: 4) Loop Tiling · p. 7
C. Directive Transform Passes
- Directive-level passes manipulate HLS-specific directives, with the SYRK example showing the transition from loop-level to directive-level IR.The figure identifies the C-to-MLIR front end and MLIR-to-C/C++ translation back end in the compilation flow.Source: C. Directive Transform Passes · pp. 7–8, Figure 5 · p. 8
1) Function and Loop Pipelining:
ScaleHLS legalizes function and loop pipeline targets before applying directives, and uses access-pattern analysis to select array partitions. It also removes redundant control-flow and memory operations from the IR.
1) Function and Loop Pipelining:
- Legalization fully unrolls contained loops and pipelines sub-functions before applying a loop directive with the specified II; outer perfectly nested loops are flattened.The function-pipelining pass uses the same legalization mechanism, and both passes diagnose illegal targets.Source: 1) Function and Loop Pipelining · p. 8
2) Array Partition:
- For each array dimension, ScaleHLS uses access count and index-distance analysis to select cyclic or block partitioning, with inter-procedural analysis supporting correct scopes and globally optimal strategies.The example applies cyclic partitioning when the index distance is 2; partition factors can also be guided through a parameter.Source: 2) Array Partition · pp. 8–9
D. IR Redundancy Elimination
- Redundancy passes remove dead affine branches, redundant memory accesses, unused memory instances, and identical accesses when dependencies do not conflict.ScaleHLS also uses MLIR canonicalization and common-subexpression elimination passes.Source: D. IR Redundancy Elimination · p. 9
E. Automatic Design Space Exploration
ScaleHLS estimates design quality early and searches a multi-dimensional transform space for latency-area Pareto points. Its flow includes an HLS C front-end that raises suitable constructs into MLIR affine operations.
1) QoR Estimation:
- The QoR estimator uses ALAP scheduling and dependency analysis to estimate latency and resource utilization before costly downstream synthesis.Memory ports are modeled as non-shareable except for reads with identical address indices; function calls and loops are dependency-graph nodes.Source: 1) QoR Estimation · p. 9
2) DSE Algorithm:
- The DSE engine targets the latency-area Pareto frontier and uses neighbor traversal because exhaustive evaluation of the potentially millions of design points is infeasible.PCA profiling of GEMM designs found Pareto points clustered in the reduced design space.Source: 2) DSE Algorithm · p. 9
- Its five-step search samples designs, proposes and evaluates neighbors of current Pareto points, updates the frontier, and selects the lowest-latency point meeting resource constraints.The finalized design is emitted as synthesizable C++ code, and the DSE pass can be extended to other optimization algorithms.Source: 2) DSE Algorithm · pp. 9–10
A. HLS C Front-end
- The C front-end translates synthesizable HLS C into scf operations, raises affine loops and memory statements when possible, and rejects unsupported constructs such as pointer-to-pointer structs.Fixed-size arrays become fixed-size memref types; scalar pointers use 1 × 1 memrefs.Source: A. HLS C Front-end · p. 10
B. HLS C/C++ Code Emission
ScaleHLS emits optimized structured IR as synthesizable C/C++ and evaluates its DSE-generated computation-kernel designs against unoptimized baselines. The reported results include large speedups and a GEMM design close to its theoretical latency bound.
B. HLS C/C++ Code Emission
- The emitter translates affine/scf control flow into C/C++ statements and converts memory and optimization attributes into HLS pragmas or corresponding code annotations.It converts returned values to input pointers to ensure synthesizability.Source: B. HLS C/C++ Code Emission · p. 10
VII. EXPERIMENTAL RESULTS
- Experiments use Vivado HLS 2019.1 to generate RTL, with performance and resource utilization taken from its synthesis reports.Source: §VII · p. 10
1) Automatic DSE results:
- Performance gains combine loop regularization, dependency reduction, tiling and unrolling, pipelining, tuned II, and partitioning matched to transformed access patterns.BICG cannot benefit from loop permutation because critical loop-carried dependencies prevent effective pipeline-II reduction, but gains 41.7× through increased computation parallelism.Source: 1) Automatic DSE results · pp. 10–11
- 41.7×–768.1× speedups were achieved across six computation kernels versus original PolyBench-C designs without DSE or manual rewriting.The kernels were evaluated on an XC7Z020 FPGA at problem size 4096.Source: 1) Automatic DSE results · p. 10, Table III · p. 11
- 0.97× of the theoretical latency bound was achieved by the DSE-generated GEMM design, which was 1.67× better than the manually optimized design.DSE took minutes, compared with about 10 hours for the manual design.Source: Table IV · p. 11, 1) Automatic DSE results · p. 11
2) Comparison with Previous Works:
ScaleHLS is designed to address large HLS designs that challenge prior DSE approaches limited to directive optimization. Its evaluations report scalable computation-kernel optimization and large throughput gains on representative neural networks.
2) Comparison with Previous Works:
- Prior DSE methods focused on directives and could struggle with large problem sizes, while ScaleHLS combines multi-level representation and automated optimization to generate synthesizable designs without manual hints or rewriting.The cited open-source framework either produced designs Vivado HLS could not synthesize or required unreasonable exploration time on the scaled-up benchmarks.Source: 2) Comparison with Previous Works · p. 11
1) Optimization Results:
- Graph, loop, and directive optimization achieved throughput speedups of 1505.3×–3825.0× on HLS designs versus baselines compiled from PyTorch to HLS C/C++ through ScaleHLS without multilevel optimization.Optimizing large HLS designs took 37.3–60.8 seconds with a single command.Source: 1) Optimization Results · p. 12
3) Scalability Study:
- Across problem sizes 32–4096, BICG, GEMM, SYR2K, and SYRK maintained stable speedups; small GESUMMV and TRMM designs had lower speedups because their design spaces limited resource use.The authors characterize the framework as scalable across a wide range of computation-kernel problem sizes.Source: 3) Scalability Study · p. 12
2) Comparison with Previous Works:
ScaleHLS is presented as a general-purpose HLS flow for large designs, with comparisons to prior work, an ablation of its optimization levels, and a conclusion that identifies future directions.
2) Comparison with Previous Works:
- ScaleHLS is described as the first general-purpose HLS flow to generate ResNet-18-level DNN accelerators without human-designed IPs or templates.Prior HLS optimization flows focused on small-scale algorithms, while DNN compilation flows relied on predefined IP libraries or parameterized templates.Source: 2) Comparison with Previous Works · pp. 12–13
- ScaleHLS achieves better DSP efficiency than TVM-VTA while saving hundreds of human hours for dedicated hardware IP design.The comparison concerns DNN accelerators.Source: 2) Comparison with Previous Works · p. 13
3) Ablation Study:
- Directive, loop, and graph optimizations contribute average speedups of 1.8×, 130.9×, and 10.3×, respectively, across three DNN benchmarks.The directive speedup is larger when combined with loop optimizations; increasing optimization level trades greater throughput for more on-chip resource use.Source: 3) Ablation Study · p. 13
VIII. CONCLUSION
- ScaleHLS combines multi-level HLS representation and optimization with an HLS C front end, a C/C++ emission back end, and an automated DSE engine.The conclusion reports strong scalability and performance and productivity improvements on a set of benchmarks.Source: §VIII · p. 13
- Future work includes IP integration, improved DSE algorithms and QoR estimation, and direct RTL code generation within MLIR; the current flow uses external HLS tools for RTL generation.The authors also hope ScaleHLS will become an open infrastructure for future HLS research.Source: §VIII · p. 13