PaperlayerView exact revision on arXiv

Source-linked AI summary

Precision-Aware Variable Bit Processing Elements for Hardware-Efficient Systolic Array Designs

Dantu Nandini Devi, Madhav Rao

arXiv:2608.22378v1cs.ARcs.ETcs.LGeess.IV

TL;DR

Approximate floating-point multipliers in Weight Stationary systolic arrays must balance hardware efficiency with numerical accuracy across diverse formats and workloads. The paper combines mantissa-level approximation with NSGA-II optimization and reports substantial efficiency gains while maintaining acceptable application quality. These results support the approach for error-tolerant deep-learning and signal-processing applications.

  • Problem

    Approximate floating-point multipliers in Weight Stationary systolic arrays must balance hardware efficiency with numerical accuracy across diverse formats and workloads.

  • Method

    The method combines mantissa partial-product truncation, post-truncation approximation, positive and negative compressors, and NSGA-II multiobjective search over processing-element configurations.

  • Results

    Across image processing and CNN workloads, optimized approximate arrays substantially reduced footprint, power, and delay while maintaining acceptable SSIM or comparable classification accuracy.

  • Takeaways & Limitations

    The results support approximate floating-point multiplier design as a hardware-efficiency approach for error-tolerant deep-learning and signal-processing applications.

Abstract

from arXiv · show

Systolic arrays (SAs) have emerged as prominent hardware accelerators for matrix operations in deep learning, while floating point number formats enable precision control across computational domains. This research investigates approximate computing techniques for floating point (FP) multipliers in Weight Stationary Systolic Arrays, focusing on IEEE 754 (FP32), TensorFloat-32 (TF32), and Brain Floating point (BF16) formats. By integrating partial product matrix (PPM) column truncation with positive and negative compressors in the FP multiplier architecture, we optimize the trade-off between computational efficiency and accuracy. NSGA-II optimization algorithm was employed to explore the vast design space for evolving FP multiplier designs, towards achieving substantial hardware improvements while maintaining acceptable output quality. Substantial hardware benefits were observed in the FP multiplier designs across various applications, while preserving output quality. The FP approximated Processing Elements designed in the SA was found to offer comparable CNN accuracy for models trained on MNIST, F-MNIST, and CIFAR-10 dataset. The FP approximated SA designs that fall in the top 10 CNN performance offered substantial hardware gains in the range of 66% to 92% footprint savings, 60% to 93% of power benefits with 21% to 54% improvement in the delay when compared with the corresponding exact implementations mentioned in the literature for running the model trained on CIFAR-10 dataset. The TF32 and BF16 approximated SA designs also achieved substantial gains while maintaining comparable CNN accuracy. Our findings confirm that targeted approximation in FP multiplier design significantly improves the efficiency of hardware accelerators for error-tolerant applications, establishing an effective approach to hardware resource optimization in contemporary computing architectures.

I. INTRODUCTION

Approximate computing targets error-tolerant workloads by trading precision for hardware efficiency. This study extends approximate systolic-array multiplier research to floating-point formats and Weight Stationary architectures.

  • Motivation: Approximate computing trades computational precision for improvements in performance, energy efficiency, and hardware footprint.The approach is motivated by resource constraints and error-tolerant image and signal processing applications.
  • AI Workloads: CNNs and fully connected networks are computationally dominated by multiplier-and-accumulator operations.These operations are central building blocks in many neural-network components.
  • Systolic Arrays: Systolic arrays accelerate GEMM and convolution using two-dimensional arrangements of processing elements containing MAC units and storage registers.The paper distinguishes Input Stationary, Weight Stationary, and Output Stationary data-flow configurations.
  • Research Gap: Prior systolic-array studies largely used single approximate multiplier types or restricted integer and application-specific settings.This limits coverage of diverse floating-point representations and systematically approximated floating-point multipliers.
  • Research Focus: The study focuses on approximate FP multipliers in Weight Stationary systolic arrays using FP32, TF32, and BF16 formats.It evaluates whether these designs can balance hardware efficiency with acceptable numerical accuracy.

II. PROPOSED DESIGN

The proposed design approximates floating-point mantissa multiplication inside Weight Stationary systolic arrays. It combines configurable truncation, post-truncation approximation, and positive or negative compression to manage hardware cost and error.

  • Floating-Point Formats: Floating-point operands use sign, exponent, and mantissa components, while FP32, TF32, and BF16 share exponent width but differ in mantissa width.These formats provide broad numerical range but make multiplication more complex.
  • Mantissa Approximation: The methodology selects the number of partial-product columns to truncate during mantissa multiplication.Truncation is applied internally to the mantissa multiplication stage rather than to the external floating-point representation.
  • Mantissa Approximation: The design selects an ideal number of bits for approximation after truncation in the partial-product reduction stage.This creates adjustable approximation levels within the multiplier architecture.
  • Compressor Design: Positive and negative compressors are combined in the reduction tree to produce error distributions that can offset one another.The approach avoids adding separate error-compensation circuits while preserving hardware approximation benefits.
  • Illustrative Configuration: The proposed reduction tree truncates the least significant 8 bits and approximates the next 3 bits with a PMCSI compressor configuration in one example.The evolutionary algorithm determines the truncation and approximation counts for each evolved design.

III. DESIGN SPACE EXPLORATION

NSGA-II explores a large, multiobjective design space in which each processing element can receive its own truncation, approximation, and compressor settings. The resulting configurations are evaluated against application quality and hardware objectives.

  • Design Space: A 3×3 systolic array with nine processing elements has a design space of (8 × n × (n + 1) ÷ 2)^9 configurations.Each processing element varies truncation bits, approximation bits, and compressor sequence.
  • Chromosome Representation: Each chromosome encodes the complete array configuration through processing-element-level TBITS, ABITS, and COMBO genes.For a 3×3 array, this representation contains 27 genes and captures the approximation strategy for all nine processing elements.
  • Evolved Array: Figure 4 illustrates an NSGA-II-evolved 3×3 FP32 array whose processing elements use varied truncation, approximation, and compressor configurations.The example emphasizes that optimization can produce nonuniform settings across the array.
  • Evolutionary Search: NSGA-II uses crossover and mutation to explore inherited processing-element sequences and perturb truncation, approximation, or compressor choices.Crossover exchanges contiguous processing-element segments, while mutation changes individual design variables.
  • Fitness Objectives: The optimization uses application quality together with worst-case delay, hardware footprint, and power as multiobjective fitness metrics.Quality is represented by SSIM or classification accuracy depending on the target application.
  • Compressor Options: Positive, negative, and exact compressor variants are available, allowing hybrid processing-element configurations without additional compensation circuits.Eight approximate variants are categorized by dominant error direction, while exact compressors remain in the design pool.

IV. EXPERIMENTAL RESULTS AND DISCUSSIONS

The experiments synthesize approximate floating-point systolic arrays for image processing, CNNs, and JPEG compression using a 45 nm ASIC flow. They assess application quality alongside delay, footprint, and power across several array sizes and data formats.

  • Experimental Framework: The evaluation uses Cadence Genus with the gpdk45nm technology node across image processing, CNN, and JPEG compression applications.These domains provide different computational requirements and precision constraints.
  • Optimization Setup: NSGA-II runs with a population size of 100 for 1000 iterations to obtain Pareto-optimal systolic-array configurations.Fitness combines application-level quality with worst-case delay, hardware footprint, and power.
  • Image Processing: Image-processing experiments include averaging and Gaussian filtering, with evaluation focused on visual quality preservation and computational efficiency.Averaging uses a 3×3 array, while Gaussian filtering uses a 5×5 array.
  • Array Configurations: The study uses 3×3 arrays for averaging and CNNs, 5×5 arrays for Gaussian filtering, and 8×8 arrays for DCT operations.The varying sizes and applications are intended to demonstrate generalization of the approach.
  • Precision Assumption: Approximation is applied internally during mantissa multiplication while weights remain stored in their original FP32, TF32, or BF16 formats.Processing-element inputs and outputs remain compliant with the selected floating-point standard.

A. Image Processing

Image-processing SAs use NSGA-II to balance SSIM against delay, footprint, and power across FP32, TF32, and BF16 formats. The reported solutions preserve high image quality while reducing hardware costs, with gains varying by filter, format, and approximation level.

  • Optimization setup: NSGA-II simultaneously optimized SSIM, worst-case delay, hardware footprint, and power for FP32, TF32, and BF16 image-processing SAs.The evaluated applications included 3 × 3 averaging and 5 × 5 Gaussian filters using the cameraman image.
  • Averaging filter: FP32 averaging-filter solutions maintained SSIM of 0.99 or higher while improving critical-path delay by 26%, silicon footprint by 68%, and power consumption by 62%.These improvements were measured against SOTA implementations cited in the paper.
  • Averaging filter: More aggressive FP32 averaging-filter approximation achieved SSIM 0.98 with 23% delay, 73% footprint, and 70% power improvements.The reported values correspond to the same averaging-filter comparison against cited SOTA implementations.
  • Averaging filter: BF16 averaging-filter designs retained SSIM 0.99 with 10% delay, 28% footprint, and 22% power improvements, while stronger reduction produced SSIM 0.98 with 11%, 44%, and 42% gains.The metrics are reported relative to exact implementations cited in the paper.
  • Gaussian filter: FP32 Gaussian-filter solutions maintained SSIM 0.99 or greater with 20% delay, 73% footprint, and 68% power improvements; stronger approximation reached SSIM 0.98 with 24%, 78%, and 75% gains.The values describe critical-path delay, silicon footprint, and power, respectively.

B. Convolutional Neural Networks

The CNN evaluation tested approximate floating-point multipliers across MNIST, Fashion-MNIST, and CIFAR-10, using exact multiplication for training and approximation during inference. The designs generally preserved competitive accuracy while delivering substantial hardware savings, and some approximate configurations exceeded exact-multiplier accuracy.

  • Results presentation: Ten representative configurations were extracted from the CNN solution space for the evaluated datasets.Figure 6 presents the solution space, while Table III reports representative hardware and accuracy characteristics.
  • MNIST: MNIST approximate implementations produced 90%–100% accuracy, with FP32 reducing delay by 27%–60%, footprint by 82%–92%, and power by 80%–93%.The exact-multiplier baseline accuracy was 99%.
  • CIFAR-10: CIFAR-10 FP32 designs achieved 60%–74% accuracy with 21%–54% delay, 66%–92% footprint, and 60%–93% power gains.The exact-multiplier baseline accuracy was 68%.
  • Accuracy comparison: Approximate multipliers increased accuracy from 97% to 99% on Fashion-MNIST and from 68% to 76% on CIFAR-10 in reported cases.The paper describes these improvements as occurring alongside hardware-efficiency gains.

C. JPEG Compression

JPEG-compression experiments evaluated approximate FP32, TF32, and BF16 SAs for grayscale and color images while monitoring SSIM and hardware metrics. FP32 provided the clearest gains, whereas TF32 and BF16 hardware results were often comparable to exact implementations in reported cases.

  • Evaluation setup: JPEG experiments covered grayscale cameraman and color-image compression using DCT-based processing of 8×8 pixel blocks.The evaluation considered both grayscale and RGB compression cases.
  • Grayscale compression: Grayscale FP32 JPEG designs maintained SSIM 0.99 or higher while improving delay by 6%, footprint by 53%, and power by 40%.A more aggressive reduction achieved SSIM 0.98 with 10% delay, 56% footprint, and 44% power improvements.
  • BF16 results: BF16 JPEG designs showed minimal or comparable hardware improvements relative to corresponding exact implementations.This comparison is reported for both grayscale and color JPEG cases.
  • Color compression: Color FP32 JPEG designs maintained SSIM 0.99 or greater with 5% delay, 50% footprint, and 39% power gains.Further precision relaxation reached SSIM 0.98 with 11%, 57%, and 44% improvements, respectively.
  • Color compression: Color TF32 designs preserved SSIM 0.99 with 5% footprint and 3% power gains, while further reduction reached SSIM 0.98 with 10% and 6% gains.Critical-path delay remained comparable to exact multipliers.

V. CONCLUSION

The work applies NSGA-II to approximate floating-point multipliers in weight-stationary systolic arrays, targeting hardware efficiency while preserving output quality. Results span image processing and CNN workloads, and the designs are released for further use.

  • NSGA-II identifies processing-element configurations that reduce power, silicon footprint, and computational delay while maintaining acceptable accuracy.The conclusion frames this as balancing accuracy and efficiency across diverse computational domains.
  • Up to 78% footprint savings and 75% power reduction were achieved for an image-processing application without perceptible SSIM degradation.These results are compared with exact state-of-the-art implementations.
  • Fashion-MNIST CNNs reached peak accuracy of 99%, while CIFAR-10 CNNs achieved classification performance up to 76%.The reported CNN results indicate that approximation can retain strong task performance.
  • Approximation-induced noise may enhance neural-network generalization in addition to providing hardware-efficiency gains.The conclusion presents this as an additional possible benefit beyond reductions in hardware cost and delay.
  • The designs are made freely available to support adoption and further use by researchers and hardware designers.
Loading 2608.22378v1…