Source-linked AI summary
Understanding the Impact of Precision Quantization on the Accuracy and Energy of Neural Networks
Soheil Hashemi, Nicholas Anthony, Hokchhay Tann, R. Iris Bahar, Sherief Reda
TL;DR
DNN accelerators face tight power and memory budgets, while existing work lacks a comprehensive comparison of precision choices across accuracy and hardware metrics. This paper evaluates a broad range of numerical precisions and training approaches, finding that larger low-precision networks can outperform smaller full-precision counterparts in energy and accuracy.
Problem
Existing studies lack a comprehensive comparison of precision choices across both network accuracy and hardware metrics, limiting guidance on their trade-offs.
Source: §I · p. 1Method
The paper evaluates precisions from binary weights to single-precision floating point, applies training techniques to improve accuracy, and assesses accuracy and hardware metrics.
Source: §I · p. 1Results
Lower-precision, larger networks can outperform smaller full-precision counterparts in energy and accuracy; Power of Two++ (6,16) eliminates accuracy loss while saving 35.93% energy.
Source: B. Results · p. 5, §VI · p. 6Takeaways & Limitations
Some energy benefits from lower precision can be spent on increasing network size to recover accuracy while retaining energy savings.
Source: §I · p. 1, B. Results · p. 5
Abstract
from arXiv · showhide
Deep neural networks are gaining in popularity as they are used to generate state-of-the-art results for a variety of computer vision and machine learning applications. At the same time, these networks have grown in depth and complexity in order to solve harder problems. Given the limitations in power budgets dedicated to these networks, the importance of low-power, low-memory solutions has been stressed in recent years. While a large number of dedicated hardware using different precisions has recently been proposed, there exists no comprehensive study of different bit precisions and arithmetic in both inputs and network parameters. In this work, we address this issue and perform a study of different bit-precisions in neural networks (from floating-point to fixed-point, powers of two, and binary). In our evaluation, we consider and analyze the effect of precision scaling on both network accuracy and hardware metrics including memory footprint, power and energy consumption, and design area. We also investigate training-time methodologies to compensate for the reduction in accuracy due to limited bit precision and demonstrate that in most cases, precision scaling can deliver significant benefits in design metrics at the cost of very modest decreases in network accuracy. In addition, we propose that a small portion of the benefits achieved when using lower precisions can be forfeited to increase the network size and therefore the accuracy. We evaluate our experiments, using three well-recognized networks and datasets to show its generality. We investigate the trade-offs and highlight the benefits of using lower precisions in terms of energy and memory footprint.
I. INTRODUCTION
DNNs deliver state-of-the-art results but demand substantial memory and computational power, motivating low-power accelerators. The paper addresses the lack of comprehensive precision comparisons by evaluating accuracy and hardware trade-offs and proposing larger low-precision networks to recover accuracy.
I. INTRODUCTION
- Memory accesses and data-transfer overheads contribute substantially to accelerator energy, motivating bit-width reduction as a hardware efficiency option.The passage notes that the memory subsystem consumes the dominant portion of accelerator power and energy.Source: §I · p. 1
- Existing studies provide little justification for chosen precisions, and no comprehensive evaluation compares network accuracy with hardware metrics across precisions.Such a comparison could give researchers better guidance on available trade-offs.Source: §I · p. 1
- The evaluation spans network precisions from binary weights to single-precision floating point, including intermediate choices.The paper also applies learning techniques to improve accuracy lost to limited precision.Source: §I · p. 1
- The authors assess accuracy, design area, power, and delay, and present results on a Pareto frontier to compare trade-offs.They also propose increasing network size to compensate for accuracy degradation; low-precision larger networks can match smaller floating-point networks in accuracy while improving energy and design area.Source: §I · p. 1
II. BACKGROUND
DNNs process data through connected layers, principally convolutional, pooling, and fully connected. Their operation involves training learned parameters and then using them in inference, with input and weight precisions treated differently.
II. BACKGROUND
- DNNs comprise connected layers that pass processed data forward, with convolutional, pooling, and fully connected layers serving distinct roles.Convolutional and fully connected layers perform most computation, while pooling layers down-sample data.Source: §II · p. 2
- Inputs and intermediate feature maps use similar precisions, while network-parameter precision can be changed independently.A neuron in convolutional and fully connected layers computes a weighted sum before applying a nonlinearity.Source: §II · p. 2
- Training learns network parameters using backpropagation and stochastic gradient descent; inference uses the learned network to classify test data.The passage identifies the learning process as the main complexity of using lower precision.Source: §II · p. 2
III. PREVIOUS WORK
Prior accelerator studies used varied precisions, often with little justification or without evaluating accuracy alongside hardware metrics. Related work explored approximate computing and precision effects, but the paper identifies a need for a comprehensive hardware-focused analysis.
III. PREVIOUS WORK
- FPGA and ASIC accelerator studies used different precisions, often with little or no justification for their chosen bit-widths.Examples include Eyeriss at 16-bit fixed-point, Sankaradas et al. at 16-bit inputs and intermediates with 20-bit weights, and a floating-point FPGA accelerator.Source: §III · p. 2
- Some prior accelerator comparisons omitted accuracy, while other work selected precision empirically or used differing widths across inputs, intermediates, and weights.Chakradhar et al. used 16-bit inputs and outputs with 48-bit intermediate values [1].Source: §III · p. 2
- Approximate-computing and limited-precision methods existed, but prior studies did not comprehensively relate precision or quantization to energy and computation time against network accuracy.Gysel et al. analyzed precision effects on accuracy but did not evaluate design parameters [9].Source: §III · p. 2
IV. METHODOLOGY
The methodology evaluates numerical precision and quantization choices, uses training techniques to limit accuracy loss, and proposes expanded networks to compensate for it.
IV. METHODOLOGY
- The methodology combines evaluation of precision and quantization choices with training techniques to minimize accuracy degradation and expanded architectures to compensate for accuracy loss.The paper describes two expanded network architectures in the methodology.Source: §IV · p. 2
A. Evaluated Precisions and Train-Time Techniques
- The evaluated numerical choices range from 32-bit floating-point arithmetic to binary networks, with several precision points in between.Source: A. Evaluated Precisions and Train-Time… · p. 3
1) Floating-Point Arithmetic:
The section contrasts floating-point accuracy and hardware demands with fixed-point flexibility and power-of-two weight quantization. It describes how these representations alter computation and the available precision choices.
1) Floating-Point Arithmetic:
- Floating-point arithmetic delivers state-of-the-art accuracy but requires complex circuitry and ample memory, making it unsuitable for low-power and embedded devices.Source: 1) Floating-Point Arithmetic · p. 3
2) Fixed-Point Arithmetic:
- Fixed-point arithmetic simplifies logic by fixing the radix point; the study evaluates 4-, 8-, 16-, and 32-bit precisions with separate radix locations for data and parameters.Non-power-of-two bit widths are excluded because inefficient memory use might nullify their benefits.Source: 2) Fixed-Point Arithmetic · p. 3
3) Power-of-Two Quantization:
- Power-of-two weight quantization replaces multiplications with shifts, while inputs remain represented using 16-bit fixed-point arithmetic.Source: 3) Power-of-Two Quantization · p. 3
4) Binary Representation:
The binary network uses one-bit weights but retains multi-bit inputs and feature maps. The section also describes the accelerator and training approaches used to accommodate reduced precision.
4) Binary Representation:
- Binary networks use one-bit weights and 16-bit fixed-point inputs and feature maps, since the input layer is not binarized.Source: 4) Binary Representation · p. 3
- The tile-based accelerator has 16 neuron processing units with 16 synapses each, and its neural functional unit pipelines weight blocks, an adder tree, and a nonlinearity function.For binary precision, the first two pipeline stages are merged; buffer sizes and control logic also vary with precision. Figure 2 shows the weight-block variants.Source: 4) Binary Representation · p. 3, Figure 2 · p. 3
- Retraining from independently trained full-precision weights restricts weights and inputs to the target precision, while assuming local search can approach the full-precision optimum.Source: 4) Binary Representation · p. 3
- Lower-precision training combines fine-tuning from floating-point weights with two weight sets, using full precision during updates and low precision for forward passes.The parameters are initialized from the floating-point counterpart.Source: 4) Binary Representation · pp. 3–4
B. Expanded Network Architectures
The section proposes spending some lower-precision savings on larger networks to recover accuracy. It reports that substantially larger low-precision networks can improve energy use without sacrificing accuracy.
B. Expanded Network Architectures
- Increasing network size can use some lower-precision savings to maintain accuracy while spending less for each operation.The proposal increases the number of computations when needed to maintain accuracy.Source: B. Expanded Network Architectures · p. 4
- Larger low-precision networks can eliminate accuracy degradation while still improving energy consumption, even when network size increases substantially.Source: B. Expanded Network Architectures · p. 4
A. Experimental Setup
The experiments compare precision choices while holding other design parameters constant, and assess both accuracy and hardware metrics. They use three benchmark datasets and include larger CIFAR-10 network variants.
A. Experimental Setup
- The study measures accuracy and design metrics including power, energy, memory requirements, and area, using a modified Ristretto framework to simulate fixed-point operation.All design parameters except bit precision are held constant to isolate its effects.Source: A. Experimental Setup · p. 4
- Hardware designs are synthesized at 250 MHz using a 65 nm technology library, with other network parameters, including frequency, held constant across precision experiments.The full-precision accurate design is set to zero timing slack at nominal processing corner.Source: A. Experimental Setup · p. 4
- Benchmarks pair MNIST with LeNet, SVHN with CONVnet, and CIFAR-10 with ALEX; the study also evaluates ALEX+ and ALEX++ on CIFAR-10.ALEX+ doubles channels in each convolutional layer; ALEX++ doubles channels when feature size is halved. Ten percent of each test-set category is selected as validation data.Source: A. Experimental Setup · p. 4
B. Results
Across MNIST, SVHN, and CIFAR-10, lower precision reduces hardware costs, and reallocating some savings to larger networks can recover accuracy while preserving energy gains.
B. Results
- Buffers consumed 75%–93% of accelerator power and 76%–96% of design area, excluding main-memory power.Figure 3 breaks down area and power across precision settings.Source: B. Results · p. 4, Figure 3 · p. 5
- Table III compares numerical precisions while holding frequency and other accelerator parameters constant; changing them was outside the study's scope.The controlled parameters include the number of hardware neurons.Source: B. Results · p. 4, Table III · p. 5
- MNIST saw little-to-no accuracy loss at nearly all precisions, while 4-bit fixed-point and binary SVHN models failed to converge; powers-of-two SVHN saved more than 84% energy with about 2% accuracy loss.Energy savings are relative to the full-precision implementation for each dataset.Source: Table IV · p. 5, B. Results · p. 5
- Reducing precision cut network memory footprints by 2× to 32×, including memory for parameters and input data.The study did not use parameter encoding or compression techniques, which it describes as orthogonal to its approach.Source: B. Results · p. 5
- 35.93% energy savings with no accuracy drop were achieved by Power of Two++ (6,16), while larger low-precision networks could dominate the full-precision baseline in accuracy and energy.The CIFAR-10 comparison is shown in Figure 4 and summarized in Table V.Source: B. Results · p. 5, Table V · p. 6, Figure 4 · p. 6
VI. CONCLUSION
The paper evaluates neural-network numerical precision across accuracy and hardware metrics, concluding that larger low-precision networks can outperform smaller full-precision counterparts in both energy and accuracy.
VI. CONCLUSION
- The study evaluates floating-point, fixed-point, powers-of-two, and binary numerical representations across accuracy and hardware metrics.The hardware metrics include area, power consumption, and energy requirements.Source: §VI · p. 6
- Larger low-precision networks can outperform smaller full-precision counterparts in both energy and accuracy.Source: §VI · p. 6
- Future work includes predicting lower-precision accuracy and hardware metrics from network–dataset correlations and developing architectures with multiple radix-point locations between layers.The paper states that multiple radix-point locations may reduce accuracy degradation.Source: §VI · p. 6