Compiler and Hardware Co-Design for Accelerator Architectures

I. INTRODUCTION

• EAAC targets regular, predictable workloads and lowers the cost of integrating custom accelerators into a full software stack without automating hardware generation.Existing tools can ease hardware implementation but typically produce hardware tied to a specific program or workload.Source: §I · p. 1
• EAAC combines a flexible hardware architecture with phased execution that allows data transfers to overlap with execution.Its MLIR compiler provides static memory allocation and explicit synchronization for data-driven scheduling.Source: §I · p. 2
• The paper presents a prototype GEMM accelerator as a proof-of-concept evaluation target for the EAAC framework.The prototype combines the framework with an accelerator implementation and a minimal-control RISC-V core.Source: §I · p. 2

II. RELATED WORK

• EAAC uses distributed hardware semaphores for control rather than relying on an embedded RISC-V core to schedule and command other units.This contrasts with SNAX’s RISC-V-centered control; both projects use MLIR-based compilers and MiniMalloc for static allocation.Source: §II · p. 2
• EAAC occupies a middle ground between fixed-workload HLS accelerators and more flexible CGRAs, with greater integration effort when adapting to a new workload.The paper describes HLS as suited to fixed workloads and CGRAs as trading some peak efficiency for broader program support.Source: §II · p. 2

A. MLIR

• Compiler backends for domain-specific accelerators may need to be built from the ground up for a specific workload and hardware architecture.The paper links this development burden to fragmentation in hardware acceleration, in contrast with toolchain reuse around a shared ISA.Source: A. MLIR · p. 2
• MLIR supports user-extensible dialects and custom transformation passes, making it suitable for compiler infrastructure targeting novel hardware.Its intermediate representation is described as more flexible than LLVM IR.Source: A. MLIR · p. 2
• MLIR dialects can represent either architecture-specific instructions or domain-specific operations such as tensor operations.The passage gives amdgpu as an architecture-specific example and linalg as a domain-specific one.Source: A. MLIR · pp. 2–3
• Multiple frontends can emit MLIR, including TensorFlow and PyTorch exports that represent common machine-learning primitives and tensor types.The passage identifies GEMM as an example of such a primitive.Source: A. MLIR · p. 3

B. Target Workload Characteristics

• EAAC targets static data-flow workloads with predictable memory access, enabling compiler scheduling optimizations such as prefetching through buffer-lifetime analysis and dependency tracking.GEMM, digital signal processing, and image processing are given as examples of applications with these traits.Source: B. Target Workload Characteristics · p. 3

IV. EXTENSIBLE ACCELERATOR ARCHITECTURE

• EAAC wraps custom accelerator units in a unified data-orchestration framework with software control over memory and dependencies.The architecture illustration shows hardware units, tiered memory, and a semaphore array for synchronization.Source: §IV · p. 3, Figure 1 · p. 3
• EAAC uses data-driven scheduling without a program counter, branches, or loops; execution is represented as a sequential instruction stream.The instruction stream may come from an embedded RISC-V core or a host device.Source: §IV · p. 3

A. Hardware Architecture

• Hardware units share Tier 0 scratchpad memory, while configurable memory tiers are interleaved with programmable DMA units.A default Load and Store unit transfers data to and from a host processor.Source: A. Hardware Architecture · p. 3
• EAAC supports coarse-grained, memory-to-memory operations whose operands are pointers to buffers, with Tier 0 scratchpad serving as working memory for hardware units.Hardware units interpret buffer contents according to the operation being executed.Source: A. Hardware Architecture · p. 3
• Hardware semaphores coordinate asynchronous, hazard-free transfers through shared Tier 0 memory; counting semaphores support streaming partial buffers.Units acquire access to the semaphore guarding the relevant input or output buffer section.Source: A. Hardware Architecture · p. 4
• Semaphore chaining supports fine-grained fences and broadcasts, while generation tags prevent aliasing when units reuse physical semaphore pairs.Compilation can prune semaphore use for statically analyzable workloads, and hardware blocks may retain local dependency tracking.Source: A. Hardware Architecture · p. 4

B. Compiler Design

• The first compiler pass category performs static memory allocation and data planning.Source: B. Compiler Design · p. 4
• The second compiler pass category manages dependencies and allocates hardware semaphores.Source: B. Compiler Design · p. 4
• The final compiler pass category generates code for accelerator units and embedded RISC-V processors.Source: B. Compiler Design · p. 4

• EAAC uses statically allocated memrefs as its compiler entry point, with explicit allocation, deallocation, and well-defined buffer lifetimes.Source: 1) Memory Allocation · p. 4
• Static allocation assigns global-memory offsets to memrefs using MiniMalloc, which avoids external ILP or CP solver dependencies.Source: 1) Memory Allocation · p. 4
• The multi-tiered scratchpad makes Tier 0 the only memory visible to accelerator units, functioning somewhat like a CPU register file.Source: 1) Memory Allocation · p. 4
• A modified linear-scan approach sorts buffers by lifetime start and recursively spills the buffer with the latest next use when a tier lacks space.The spilled buffer is reloaded into its original tier before its next use; Figure 2 illustrates buffer movement across tiers.Source: 1) Memory Allocation · pp. 4–5, Figure 2 · p. 5
• A later pass turns assigned spills into concrete IR operations that are lowered to DMA, deallocating the buffer's former tier space after movement.Source: 1) Memory Allocation · p. 5

• With no hardware context, EAAC assumes operations can execute out of order once dependencies are ready, so it allocates a virtual semaphore for every producer-consumer edge.This can require an infeasibly large number of hardware semaphores.Source: 2) Data Dependency Management · p. 5
• For possible read-after-write hazards from address reuse, the compiler chains the semaphores guarding the buffers so the read and write execute in order.Source: 2) Data Dependency Management · p. 5
• The compiler removes synchronization when hardware implicitly serializes operations or when their data-flow distance is adequate for the target architecture.The applicable methods depend on the accelerator architecture and its design parameters.Source: 2) Data Dependency Management · p. 5
• Virtual semaphores are assigned physical addresses in allocation order; when addresses run out, the compiler recycles one and appends a generation tag to prevent aliasing.It chooses the address most likely to be freed soonest to avoid unnecessary serialization; generation tags distinguish successive allocations of the same address.Source: 2) Data Dependency Management · p. 5

• EAAC supplies the passes for memory management and hardware-semaphore synchronization, while users develop code generation for their application-specific operations.Source: 3) Codegen · pp. 5–6
• When operations are not supported by dedicated hardware, they can be offloaded to RISC-V through LLVM-dialect lowering and compilation into RISC-V assembly.Custom passes provide compatibility for access to statically allocated scratchpad buffers and hardware semaphores.Source: 3) Codegen · p. 6
• The compiler emits a FlatBuffer for the assembler and runtime, and also emits LLVM IR for RISC-V assembly when offloading is needed.Source: 3) Codegen · p. 6

• The prototype re-engineers a GEMM accelerator with Tier 0 memory accessible to compute units and a hardware semaphore bank; memory, bus, semaphore, and systolic-array dimensions are parameterizable.Source: C. Hardware Accelerator Prototype · p. 6
• The functional units include a load/store unit, an 8-bit 16x16 GEMM unit, and a small integer RISC-V processor for workloads such as activation and requantization.The units share an instruction decoder and have separate interfaces for scratchpad memory and semaphores.Source: C. Hardware Accelerator Prototype · p. 6
• Because data supply and synchronization are memory-mapped, a custom hardware unit needs only a memory interface and a way to receive host instructions.Source: C. Hardware Accelerator Prototype · p. 6
• The integrated Sodor-based RISC-V core is intentionally simple and low-performance, chosen to ease system integration rather than maximize baseline performance.The table compares baseline RISC-V and accelerator execution-time breakdowns; GEMM and MCC compute cycles overlap, so the total is measured wall-clock cycles.Source: C. Hardware Accelerator Prototype · p. 6, Table I · p. 7

V. EVALUATION

• The comparison uses a RISC-V-only baseline and a second implementation with a 16 × 16 systolic-array GEMM unit; the prototype targets a Kintex 7 xc7k160tffg676-2 FPGA.The system has 64kB of T0 memory and 128kB each of T1 and T2 memory; the RISC-V core has 64kB of instruction memory and 4kB of data memory.Source: §V · p. 7
• The workload comprises 24 independent 16 × 16 int8 matmul tiles summed in int32, then bias-added, ReLU’d, and requantized to int8.It is a synthetic dense-layer unit inspired by the MLPerf-Tiny anomaly-detection network.Source: §V · p. 7
• Larger workloads often required an infeasible number of hardware semaphores, so the evaluation used a smaller workload to limit semaphore-bank consumption.The system is configured with 16 hardware semaphores.Source: §V · p. 7
• Execution-time measurements include loading the compiled binary into instruction memory, and that transfer takes significantly longer for the accelerator because EAAC IR-to-LLVM lowering is unoptimized.Table II reports resource utilization by top-level module; the text says the GEMM unit accounts for most prototype resources.Source: §V · p. 7, Table II · p. 7
• 28x speedup in total execution time over the RISC-V-only baseline, while GEMM accounts for only 1.64% of total execution time.The prototype is not highly optimized; its maximum achievable clock speed is 104.602 MHz, with the critical path in the RISC-V core.Source: §V · p. 7

1) Compiler Evaluation:

• Long-lived semaphore-guarded buffers in programs with hundreds or thousands of instructions increase the burden on semaphore tracking.The allocation test uses increasingly large programs with worst-case buffer lifetimes.Source: 1) Compiler Evaluation · p. 7
• Worst-case distinct hardware semaphore needs grow linearly with application operations; with a single GEMM unit, serialized GEMM operations make some semaphores unnecessary.The test programs were generated with worst-case buffer lifetimes.Source: 1) Compiler Evaluation · pp. 7–8
• Using static hardware information to remove semaphores between ordered operations and between different units breaks the linear increase in semaphore consumption.The compiler exploits known execution order and known run-ahead between hardware units.Source: 1) Compiler Evaluation · p. 8

VI. CONCLUSION

• EAAC is a minimal, extensible interface for integrating custom hardware units into a full software stack, rather than a system that automates hardware generation.It uses a tiered scratchpad hierarchy, static allocation via MiniMalloc, and distributed hardware-semaphore synchronization under a shared memory-mapped control model.Source: §VI · p. 8
• The shared memory-mapped control model is intended to let accelerators produced through diverse methods coexist.EAAC combines tiered scratchpad memory, static allocation, and distributed synchronization.Source: §VI · p. 8
• A prototype GEMM accelerator combining a systolic array and embedded RISC-V core achieved a 28x execution-time speedup over a RISC-V-only baseline on a synthetic fully connected-layer workload.The demonstration ran end-to-end from MLIR input through hardware execution.Source: §VI · p. 8