Source-linked AI summary
A 420 GOPS/W CGRA with a Configurable MAC and Dynamic Truncation
Yi Sheng Chong, Rakshith Harish, Rajesh Chandrasekhara Panicker, Vishnu P. Nambiar, Anh Tuan Do
TL;DR
Edge workloads need flexible, energy-efficient processing, but conventional CGRAs require two cycles for each MAC. This paper introduces configurable MAC and truncation units in CGRA PEs, achieving 420.6GOPS/W at 0.6V and 21MHz, 1.4 times higher than the state-of-the-art.
Problem
Conventional CGRAs require two cycles for MAC operations, limiting processing speed for workloads with many MACs such as neural-network inference and FFT.
Method
The proposed CGRA uses PE-level configurable MAC units for MUL, ADD, or MAC operations and a truncation block for adjustable MAC-result precision.
Results
420.6GOPS/W at 0.6V and 21MHz is reported for the 40nm CGRA, which is 1.4 times higher than the state-of-the-art.
Takeaways & Limitations
The configurable MAC completes a MAC in one cycle while reusing multiplier and adder hardware, reducing processing cycles for MAC-heavy workloads.
Abstract
from arXiv · showhide
Edge devices demand for highly efficient yet flexible processing capability to handle dynamic real-time workloads. Coarse grain reconfigurable architecture (CGRA) emerges as a suitable accelerator candidate in edge devices, because they are as flexible as general purpose processors and offer high efficiency close to that of domain specific accelerators. However, a typical CGRA requires two cycles for a multiply-and-accumulate (MAC) operation, and workloads such as neural network inference and signal processing involve many MAC operations, resulting in long CGRA processing time. This work proposes a CGRA that has configurable MAC units in the processing elements (PEs) that can perform an addition (ADD) or multiplication (MUL) or a MAC by using the same multiplier and adder, in a single cycle. The readout precision of MAC result can be adjusted by a truncation block. The proposed CGRA is implemented with 40nm CMOS technology. It attains an energy efficiency of 420.6GOPS/W operating at supply of 0.6V and frequency of 21MHz, which is 1.4 times higher than the state-of-the-art.
I. INTRODUCTION
CGRA offers programmable, energy-efficient acceleration for diverse edge workloads, but conventional CGRAs require two cycles per MAC. The proposed configurable MAC and truncation blocks reduce MAC processing cycles while supporting adjustable output precision.
- CGRA combines higher efficiency and flexibility than general-purpose processors and domain-specific accelerators, while parallel PEs support different workloads.Its 2D PE array uses reconfigurable connections and compiler-generated instructions.
- A typical CGRA maps each MAC to separate MUL and ADD instructions, requiring two cycles and slowing workloads with many MAC operations.Such workloads include neural-network image recognition and FFT-based digital signal processing.
- The proposed PE performs MUL, ADD, or MAC through a configurable MAC block and adjusts readout precision with a configurable truncation block.The design reduces processing cycles for MAC-heavy workloads.
A. Structure of the CGRA
The proposed CGRA integrates an 8x8 PE array and data memory with a RISC-V CPU, using configurable PE instructions and meshed communication for programmable computation.
- A. Structure of the CGRA: The CGRA uses an 8x8 PE array, 64KB data memory, meshed PE connections, and a RISC-V CPU for coordination.Each PE communicates with up to four neighboring PEs, while edge PEs connect to data memory.
- B. Structure of the PE: Each PE contains configuration memory, an ALU, a control unit, and a crossbar router that fetches instructions and directs data.The ALU supports 18 arithmetic and logical operations.
- B. Structure of the PE: The configurable MAC adds MUL, ADD, and MAC modes to the PE, reducing execution cycles for workloads with abundant MAC operations.The same PE supports conventional single-operation execution and the enhanced MAC path.
C. Configurable MAC
The configurable MAC reuses one signed integer multiplier and adder to execute MUL, ADD, or MAC according to the PE instruction, while preserving full accumulation precision before readout truncation.
- C. Configurable MAC: The configurable MAC supports MUL, ADD, and MAC modes selected by the decoded PE opcode.Multiplexers configure the datapath for each operation.
- C. Configurable MAC: The design reuses the same signed integer multiplier and adder across all three modes to save hardware resources.MUL and ADD results leave the ALU, while MAC results are stored in an accumulation register.
- C. Configurable MAC: A 32-bit accumulation result is formed from 16-bit inputs, then passed to truncation during readout to match the CGRA data width.The wider accumulator prevents precision loss during accumulation.
D. Configurable truncation block
The configurable truncation block detects overflow and converts full-precision MAC results to user-selected output formats, enabling controlled precision at PE readout.
- D. Configurable truncation block: The MAC accumulator is 32-bit for 16-bit operands, and truncation reduces the result to 16-bit for subsequent PE processing or data-memory storage.Truncation occurs after all accumulation cycles are complete.
- D. Configurable truncation block: The block supports four configuration modes that select the output precision and fractional representation.The cited figure identifies the four modes but does not provide their individual encodings.
- D. Configurable truncation block: FXP⟨4,12⟩ and FXP⟨2,14⟩ are included because prior MNIST CNN accuracy analysis found satisfactory results for these 16-bit formats.The configuration choice is linked to the fractional-part length of 16-bit weights.
- D. Configurable truncation block: The truncation block checks overflow, clips overflowing results to all ones, and otherwise truncates integer and fractional parts separately.A dedicated opcode selects MAC-result truncation.
A. Data flow in conventional and proposed PE
The conventional PE completes MAC across two cycles and instructions, whereas the proposed PE combines multiplication and accumulation in one cycle using a shared multiplier and adder.
- PE execution: The PE decodes one configuration-memory instruction per clock cycle to configure its ALU and router operations.Execution loops through the configuration memory until the stopping criterion is reached.
- A. Conventional PE: The conventional PE uses separate MUL and ADD instructions across two cycles for each MAC operation.The product is generated at t1 and added to the operands at t2.
- B. Proposed PE: One MAC instruction completes multiplication and accumulation in a single cycle within the proposed PE.The product is accumulated with the previous MAC register result using the PE's adder, then stored in the accumulation register.
B. Performance of the proposed PE
The proposed PE is synthesized in 40nm technology and evaluated for area, timing, power, and energy against a conventional PE using GeMM.
- Implementation: 0.019 mm2 is the synthesized area of the proposed PE in 40nm technology.The paper also reports a roughly 3.2ns MAC path delay, 1.86 times the multiplier path delay, while meeting a 100MHz target.
- Energy evaluation: 0.34 times smaller energy consumption is reported for the proposed PE than the conventional PE in the 8x8 GeMM example.The proposed PE consumes 0.74mW while processing 512 MAC operations, similar to the conventional PE's power consumption.
C. Performance of the CGRA
The CGRA is evaluated on GeMM, where configurable MAC execution reduces arithmetic instruction reads and produces high throughput and energy efficiency.
- Instruction reduction: 54% fewer arithmetic instruction reads are required for the proposed CGRA than the conventional CGRA on 8x8 GeMM.The conventional design reads 512 MUL and 448 ADD instructions, while the proposed design reads 512 MAC instructions.
- CGRA results: One MAC is performed in 1.35 clock cycles after compiler control-instruction overhead is included.GeMM is used as the workload, with A as the resultant matrix and B and C as input matrices.
- CGRA results: 420.7 GOPS/W is the peak efficiency at 0.6V and 21MHz, about 1.4 times higher than the state-of-the-art.The proposed CGRA achieves 9.5GOPS while consuming 47.6mW at 1V supply.
IV. CONCLUSION
The proposed CGRA uses configurable MAC and truncation blocks to reduce MAC processing cycles while allowing selectable output precision.
- Conclusion: The configurable MAC completes MAC operations in one cycle while reusing the same hardware for MUL and ADD.The truncation block lets users choose the precision of a full-precision accumulated MAC result when it leaves the PE.