PaperlayerView exact revision on arXiv

Source-linked AI summary

Ristretto: Hardware-Oriented Approximation of Convolutional Neural Networks

Philipp Gysel

arXiv:1605.06402v1cs.CVcs.LGcs.NE

TL;DR

CNNs’ computational and memory demands complicate hardware deployment, and developers lack an open-source tool to automate compression while avoiding excessive accuracy loss. Ristretto automates network approximation and fixed-point selection. Dynamic fixed point reaches 4-bit or 8-bit widths for LeNet, CIFAR-10, and CaffeNet, with reduced memory bandwidth and footprint.

  • Problem

    CNNs require substantial computation and memory, while no open-source project helps hardware developers quickly determine how to reduce a trained network’s complexity, and aggressive compression can reduce accuracy.

    Source: Introduction · pp. 11–12

  • Method

    Ristretto automates neural-network approximation, tests fixed-point bit-width partitions, and fine-tunes the selected fixed-point network.

    Source: Introduction · p. 12, §4.4.1 · p. 39

  • Results

    4-bit or 8-bit dynamic fixed point is reported for LeNet, CIFAR-10, and CaffeNet, with significantly reduced memory bandwidth and footprint.

    Source: §8.4 · pp. 59–60

  • Takeaways & Limitations

    The reduced memory footprint is expected to yield energy savings in FPGA and ASIC designs, while the framework aims to lower processing-element area and off-chip memory communication.

    Source: Introduction · p. 12, §8.4 · p. 60

    Quantization of weights, layer inputs, and outputs introduces errors that can affect accuracy, and the described truncated-number experiments use a shared fixed-point format.

    Source: §2.6.1 · p. 25, §4.2 · p. 36

Abstract

from arXiv · show

Convolutional neural networks (CNN) have achieved major breakthroughs in recent years. Their performance in computer vision have matched and in some areas even surpassed human capabilities. Deep neural networks can capture complex non-linear features; however this ability comes at the cost of high computational and memory requirements. State-of-art networks require billions of arithmetic operations and millions of parameters. To enable embedded devices such as smartphones, Google glasses and monitoring cameras with the astonishing power of deep learning, dedicated hardware accelerators can be used to decrease both execution time and power consumption. In applications where fast connection to the cloud is not guaranteed or where privacy is important, computation needs to be done locally. Many hardware accelerators for deep neural networks have been proposed recently. A first important step of accelerator design is hardware-oriented approximation of deep networks, which enables energy-efficient inference. We present Ristretto, a fast and automated framework for CNN approximation. Ristretto simulates the hardware arithmetic of a custom hardware accelerator. The framework reduces the bit-width of network parameters and outputs of resource-intense layers, which reduces the chip area for multiplication units significantly. Alternatively, Ristretto can remove the need for multipliers altogether, resulting in an adder-only arithmetic. The tool fine-tunes trimmed networks to achieve high classification accuracy. Since training of deep neural networks can be time-consuming, Ristretto uses highly optimized routines which run on the GPU. This enables fast compression of any given network. Given a maximum tolerance of 1%, Ristretto can successfully condense CaffeNet and SqueezeNet to 8-bit. The code for Ristretto is available.

Introduction

Ristretto addresses the need for an open-source, automated way to condense trained neural networks for hardware while managing the accuracy trade-off. The background introduces CNN structure and the forward and backward phases of training.

Convolutional Neural Networks

  • State-of-the-art CNNs used in image-recognition competitions contain millions of parameters and require billions of arithmetic operations.CNNs automatically extract low- and high-level features, unlike handcrafted feature methods.Source: Introduction · p. 11, Convolutional Neural Networks · p. 13

Introduction

  • Ristretto is an open-source framework for automated neural-network approximation that analyzes CNN resource requirements and compression–accuracy trade-offs.It targets resource-intensive parts without adding decompression or sparse-matrix multiplication complexity.Source: Introduction · p. 12

2.1 Training and Inference

  • CNNs use supervised training to learn parameters, with forward propagation for classification and backward propagation for training.Training large networks can take days or weeks, and GPU throughput and memory capacity support faster training.Source: Introduction · p. 11, §2.1 · p. 13

2.1.1 Forward Propagation

  • AlexNet processes an RGB image through five convolutional layers and three dense layers, ending in 1,000 class outputs.Convolutional filters extract features; pooling reduces feature-map dimensions, and nonlinear layers enable learning nonlinear features.Source: §2.1.1 · p. 14

2.1.2 Backward Propagation

  • Backward propagation computes gradients for layer outputs and then parameters, using the chain rule to support parameter updates.Batch-based stochastic gradient descent averages gradients across labeled examples and uses a learning rate.Source: §2.1.2 · pp. 15–16

2.2 Layer Types

CNN layer types perform feature extraction, dimensionality reduction, and nonlinear transformation. Their computational costs and arithmetic needs differ, motivating approximation of resource-intensive layers while leaving inexpensive pooling unchanged.

Convolutional layer

  • Convolutional layers extract features through multiply-accumulate operations and are a computational bottleneck because their computation scales with output-map area.With stride 1, computation is R · C times the number of parameters.Source: §2.2 · p. 16

Fully connected layer

  • Fully connected layers form weighted sums with a matrix-vector product, and their parameters can account for a large share of network size.Their inputs connect to every output, and they commonly appear near the end of CNNs.Source: §2.2 · p. 17

Rectified Linear Unit (ReLU)

  • ReLU maps negative values to zero and is used instead of costly sigmoid or tanh functions, whose gradients become small at large values.ReLU is associated with faster convergence in deep networks.Source: §2.2 · p. 18

Normalization layers

  • LRN and batch normalization can support faster convergence and better accuracy, but their overhead and large intermediate-value ranges lead this work to leave them in floating point.In AlexNet, LRN intermediate values are 214 times larger than those from any other layer.Source: §2.2 · pp. 18–19

Pooling

  • Pooling reduces feature-map dimensions and encodes translation invariance; because max pooling requires comparisons rather than arithmetic, this work does not approximate it.The example’s 2×2 max-pooling operation reduces data size by 75%.Source: §2.2 · p. 20

2.3 Applications

CNNs support a wide range of applications, but their computation and memory demands make network size and accuracy important design trade-offs. The section introduces ImageNet evaluation and architectural approaches that balance those demands.

2.3 Applications

  • Deep networks have applications including image classification, object detection, speech recognition, games, and art.The text notes that neural networks and recurrent neural networks could potentially be approximated similarly.Source: §2.3 · p. 20

2.4 Computational Complexity and Memory Requirements

  • Convolutional layers account for over 90% of CNN arithmetic operations, while fully connected layers contain over 90% of network parameters.For CaffeNet, convolutional layers require 2.15 G operations and fully connected layers hold 235 MB of its 250 MB of parameters.Source: §2.4 · p. 21

2.5 ImageNet Competition

  • ImageNet provides a large-scale classification and detection challenge, and researchers commonly report top-1 and top-5 accuracy on its 2012 data set.This thesis focuses on top-1 accuracy to capture how CNN approximation affects network accuracy.Source: §2.5 · p. 22

2.5.1 Network Size vs Accuracy

  • Network size and accuracy trade off: GoogLeNet balances both, SqueezeNet is smallest but not outstanding in accuracy, and ResNet is most accurate but largest.SqueezeNet matches AlexNet accuracy with 50× fewer parameters and avoids fully connected layers, reducing parameter size below 5 MB.Source: §2.5.1 · p. 23

2.5.2 Inception Idea

  • GoogLeNet's inception modules combine 1×1, 3×3, and 5×5 convolutions with a max-pooling path; 1×1 convolutions reduce feature maps before more expensive convolutions.The architecture was designed for high ImageNet accuracy within a budget of 1.5 billion MAC operations for inference.Source: §2.5.2 · p. 24

2.6 Neural Networks With Limited Numerical Precision

Limited-precision arithmetic can reduce the resources required by CNN layers, but quantization introduces errors that may affect accuracy. The framework simulates quantized hardware arithmetic and fine-tunes networks in discrete parameter space.

2.6 Neural Networks With Limited Numerical Precision

  • CNNs are often trained and inferred with 32-bit or 64-bit floating point, although they are described as error-resilient and trainable in discrete parameter spaces.The section introduces quantization, rounding schemes, and accuracy optimization for limited-precision networks.Source: §2.6 · pp. 24–25

2.6.1 Quantization

  • The framework quantizes layer inputs and weights, performs multiply-accumulate operations with those values, and quantizes the output.Quantizing weights, layer inputs, and outputs introduces errors that can affect network accuracy.Source: §2.6.1 · pp. 25–26

2.6.1.1 Data path in hardware

  • The quantized data path uses parallel multiplication units, an adder tree for accumulation, and a final quantization step for layer outputs.Using reduced-bit inputs and weights lowers multiplier area; 32-bit accumulation is simulated because adders are cheaper than multipliers, and output quantization reduces memory requirements.Source: §2.6.1.1 · pp. 26–27

2.6.2 Rounding Schemes

  • Round-nearest-even is deterministic and used for inference, while stochastic rounding is used to quantize parameters during fine-tuning.Stochastic rounding adds randomness that can have an averaging effect during training.Source: §2.6.2 · p. 27

2.6.3 Optimization in Discrete Parameter Space

  • Because quantized-network optimization is discrete and NP-hard, the framework first trains in the continuous domain, then quantizes and fine-tunes the parameters.This approach leverages pretrained networks and uses full-precision shadow weights with stochastic sampling during fine-tuning.Source: §2.6.3 · p. 28

Related Work

Related work reduces CNN resource demands through lower-precision arithmetic, pruning, shared or binary weights, and accelerator designs. The comparison highlights different throughput and power trade-offs across GPUs, FPGAs, and ASICs.

3.1.1 Fixed Point Approximation

  • Fixed-point methods reduce arithmetic resources: prior work trained CIFAR-10 networks in 16-bit, trimmed multipliers to 7-bit, or used binary and ternary weights.The cited fixed-point schemes performed well on small networks, but the passage reports limited work on large CNNs such as AlexNet.Source: §3.1.1 · p. 29

3.1.2 Network Pruning and Shared Weights

  • Deep Compression combines iterative pruning and fine-tuning, shared-weight clustering and fine-tuning, and Huffman coding, achieving up to 49× parameter compression.The motivation is to reduce parameter storage and associated off-chip memory access.Source: §3.1.2 · pp. 29–30

3.1.3 Binary Networks

  • Binary approaches replace weights or activations with constrained values, sometimes converting multiply-accumulate operations into XNOR and bit-count operations.On ImageNet, Binary-Weights-Network reports 2× CPU speedup within 3% of AlexNet accuracy, while XNOR-Net reports a 12.4% accuracy drop.Source: §3.1.3 · pp. 30–31

3.2 Accelerators

  • The accelerator survey compares GPUs, FPGAs, and ASICs as platforms for CNN inference.The review distinguishes GPU throughput and memory bandwidth, FPGA throughput per power and development time, and custom architectures' throughput and energy efficiency.Source: §3.2 · p. 31, §3.2.1 · p. 31, §3.2.2 · p. 31

3.2.1 GPU Acceleration

  • GPU methods include clustered filters and low-rank approximation, FFT-based convolution, and pruning with weight sharing.For dense AlexNet layers, the cited compression pipeline makes forward propagation 4× faster and 3× more energy efficient; a later Titan X implementation reports 3.23 TOPs.Source: §3.2.1 · p. 31

3.2.2 FPGA-based Accelerators

  • FPGA designs report over 60 GFLOPs at 20 W for AlexNet convolution, 117.8 GOPs for VGG, and 137 GOPs with less than 10 W using 16-bit fixed point.The cited implementations use Vivado HLS, OpenCL, and a Xilinx-based design, respectively.Source: §3.2.2 · p. 32

3.2.3 Custom Accelerators (ASIC)

  • Custom ASICs offer high throughput and energy efficiency, with Eyeriss reporting 74.6 GOPs at 0.278 W and EIE inferring compressed networks 13× faster than a Titan X at 0.6 W.Eyeriss is about 2× slower than a throughput-optimized embedded GPU but 13× more energy efficient.Source: §3.2.3 · p. 32

3.2.4 Comparison Of Accelerator Platforms

  • GPU acceleration achieves the highest reported throughput at 3.23 TOP/s, while ASICs have the highest throughput per power; FPGA results are close to embedded GPUs.The FPGA implementation has 11% less throughput and 6% less throughput per power than the embedded GPU.Source: Figure 3.1 · p. 33, §3.2.4 · pp. 33–34, Table 3.1 · p. 33

Fixed Point Approximation

The section frames fixed-point arithmetic as a way to reduce the resource demands of CNN inference. It identifies the baseline networks used for approximation experiments.

Fixed Point Approximation

  • Using reduced bit-width formats for intermediate results can increase the throughput and energy efficiency of hardware accelerators.The section contrasts this approach with normal inference in 32-bit floating point.Source: Fixed Point Approximation · p. 35

4.1 Baseline Convolutional Neural Networks

  • The baseline set includes LeNet for MNIST, CIFAR-10 Full for CIFAR-10, and CaffeNet for 1000-class ImageNet classification.CaffeNet has five convolutional layers, three fully connected layers, 60 million parameters, and 650,000 neurons.Source: §4.1 · pp. 35–36
  • The baselines also include GoogLeNet, with 12× fewer parameters than AlexNet and improved accuracy, and SqueezeNet, designed for AlexNet-level accuracy with a smaller network.SqueezeNet uses 1×1 and 3×3 convolutional kernels and has no fully connected or normalization layers.Source: §4.1 · p. 36

4.2 Fixed Point Format

The experiments replace 32-bit floating point with fixed-point representations and show that the best bit allocation and attainable accuracy vary by network. Fine-tuning improves the quantized results, but the supplied outcomes range from a small LeNet loss at 8 bits to a larger remaining CaffeNet gap at 16 bits.

4.2 Fixed Point Format

  • The experiments use a shared fixed-point format for parameters and layer outputs, with total bit-width equal to integer plus fractional length.Floating-point values are quantized by round-nearest using two's-complement representation.Source: §4.2 · p. 36

4.3.1 Dynamic Range in Small CNN

  • For LeNet, 99% of parameters lie between 2^0 and 2^-10, while 99% of fully connected layer outputs lie between 2^5 and 2^-4.The analysis used forward propagation of 100 images, with values truncated to integer powers of two.Source: §4.3.1 · p. 37

4.3.2 Dynamic Range in Large CNN

  • CaffeNet's second parameter layer has values above 2^9, whereas all values in its last parameter layer are below 2^6.The respective layers involve 447.9M and 4.1M MAC operations.Source: §4.3.2 · p. 38, Figure 4.2 · p. 38
  • CaffeNet's best 16-bit format is Q9.7, with 21.23% of parameters truncated to zero and 0.46% of convolutional-layer outputs too large to represent.The results indicate that large layer outputs are more important than small parameters.Source: §4.3.2 · p. 38

4.4.1 Optimal Integer and Fractional Length

  • Jointly optimizing layer-output and parameter formats, then fine-tuning, yields the best results across three networks.The procedure searches integer/fractional bit partitionings at a target bit-width.Source: §4.4.1 · p. 39

4.4.2 Quantization to Fixed Point

  • LeNet's absolute accuracy loss after fine-tuning is 0.27% at 8-bit fixed point.Before fine-tuning, the relative accuracy loss is 10.3%.Source: §4.4.2 · p. 39, Figure 4.3 · p. 40, Table 4.1 · p. 40
  • CIFAR-10 accuracy drops to 10% at 8 bits, while 16-bit fixed point yields a relative accuracy loss below 1%.The best result used 8 integer bits, compared with 4 for LeNet.Source: §4.4.2 · p. 40
  • CaffeNet's 16-bit fixed-point network achieves accuracy within 4.5% of the baseline after fine-tuning.Quantization alone has a relative accuracy loss of 8.4%.Source: Fixed point format · pp. 40–41, Table 4.1 · p. 40

Dynamic Fixed Point Approximation

Dynamic fixed point assigns number formats to network parameters and layer outputs separately, adapting integer and fractional bit allocation to their ranges. On CaffeNet it remains stable at lower bit-widths than traditional fixed point, and fine-tuned networks retain high accuracy with smaller hardware and memory requirements.

Dynamic Fixed Point Approximation

5.1 Mixed Precision Fixed Point

  • Mixed precision assigns different bit-widths to layer outputs and weights, balancing narrower representations against classification accuracy.Products are accumulated through an adder tree, with the bias added at the final stage.Source: §5.1 · pp. 42–43, Figure 5.1 · p. 42

5.2 Dynamic Fixed Point

  • Dynamic fixed point represents values with a bit-width, sign bit, fractional length, and mantissa, grouping values with a shared fractional length.Each layer has separate groups for outputs and weights, whose ranges differ; the format can be implemented with bit shifters.Source: §5.2 · p. 43, Figure 5.2 · p. 44

5.2.1 Choice of Number Format

  • The number format is chosen automatically to avoid saturation, with layer outputs assigned one fewer integer bit than the parameter-based relation specifies.The output integer-length adjustment gave slightly better results in the reported experiments.Source: §5.2.1 · p. 44

5.3.1 Impact of Dynamic Fixed Point

  • Dynamic fixed point stays stable as CaffeNet bit-width falls below 18-bit, whereas traditional fixed-point accuracy drops significantly.Dynamic allocation adapts integer and fractional bits to the ranges of different network components.Source: §5.3.1 · p. 45, Figure 5.3 · p. 45

5.3.2 Quantization of Individual Network Parts

  • Convolution kernels and layer activations can be trimmed to 8-bit with only 0.3% absolute accuracy change, while fully connected weights incur at most 0.9% change at 8-bit.LeNet weights can be reduced to 2-bit with less than 0.4% absolute accuracy change.Source: §5.3.2 · pp. 45–46, Table 5.1 · p. 46

5.3.3 Fine-tuned Dynamic Fixed Point Networks

  • LeNet performs well at 2/4-bit, while CIFAR-10 and the three ImageNet CNNs can be trimmed to 8-bit; relative accuracy drops are below 0.6% for LeNet, CIFAR-10, and SqueezeNet, and 0.9% and 2.3% for CaffeNet and GoogLeNet.The fine-tuned networks perform nearly as well as their floating-point baselines.Source: §5.3.3 · p. 46, Table 5.2 · p. 47
  • 8-bit fixed-point approximation reduces multiplication-unit size by about one order of magnitude, cuts memory bandwidth by 4–8X, and allows 4–8X more parameters in on-chip buffers.These hardware benefits are reported for the five approximated 32-bit floating-point networks.Source: §5.3.3 · p. 47

Minifloat Approximation

Minifloat approximates the smaller floating-point numbers used by neural networks, retaining floating-point representation with fewer bits. The framework chooses exponent and mantissa allocation for a network, and the layer data path multiplies and accumulates in minifloat.

6.1 Motivation

  • Because neural-network training is normally done in floating point, this section analyzes network approximation using minifloats—floating-point numbers with 16 bits or fewer—to condense models to smaller floating-point representations.Source: §6.1 · p. 48

6.3 Minifloat Number Format

  • Ristretto minifloat lowers exponent bias according to exponent width and omits denormalized numbers, INF, and NaN.INF is replaced by saturated numbers, denormalized numbers by zero, and the framework searches for an exponent/mantissa balance; a 16-bit example uses 1 sign, 5 exponent, and 10 mantissa bits.Source: §6.3 · pp. 48–49, Figure 6.1 · p. 49

6.3.1 Network-specific Choice of Number Format

  • The network-specific format uses enough exponent bits to avoid saturation, assuming infinitely many mantissa bits.Exponent and mantissa bits divide the available bit-width.Source: §6.3.1 · p. 49

6.4 Data Path for Accelerator

  • The minifloat layer data path multiplies inputs and parameters, accumulates products in full precision, adds bias in minifloat, and trims the result to minifloat.Multiplier outputs are 3 bits wider than their input numbers.Source: §6.4 · pp. 49–50, Figure 6.2 · p. 50
  • When layers do not fit in on-chip memory, storing minifloat outputs and parameters off-chip reduces data transfer and can save energy.The accelerator data path takes and produces minifloat numbers.Source: §6.4 · pp. 49–50

6.5 Results

The section reports minifloat accuracy at reduced bit-widths, compares it with prior work, and introduces power-of-two weights for multiplier-free arithmetic.

6.5 Results

  • CIFAR-10 and CaffeNet have no accuracy loss at 12-bit minifloat, while LeNet has none at 8-bit.The comparison uses the corresponding 32-bit networks as baselines.Source: §6.5 · p. 51, Figure 6.3 · p. 51
  • After fine-tuning, CIFAR-10’s absolute accuracy drop is below 1%, CaffeNet’s is 4.6%, and LeNet’s accuracy increases by 0.05%.The increases and drops are compared with the 32-bit network; the table caption identifies bracketed values as results without fine-tuning.Source: §6.5 · p. 51, Table 6.1 · p. 51
  • For 8-bit CaffeNet, dynamic fixed point has an absolute accuracy drop below 1%, versus 4.38% for minifloat.Source: §6.5 · p. 52

6.6 Comparison to Previous Work

  • Ristretto’s 8-bit minifloat results outperform Deng et al.’s by using a carefully chosen format and fine-tuning.Source: §6.6 · p. 52

7.1 Multiplier-free Arithmetic

  • Replacing multiplications with shifts can save accelerator energy because multipliers require more chip area than additions.The approach uses integer power-of-two weights, equivalent to minifloats with zero mantissa bits.Source: §7.1 · p. 53, multiplier-free arithmetic · pp. 53–54

7.2 Maximal Number of Shifts

  • The multiplier-free format encodes exponents in 4 bits, representing values from 2^-8 to 2^-1; only 10.97% of CaffeNet parameters and 5.83% of LeNet parameters fall below its minimum.Parameters below 2^-8 are described as having only a minor effect on network output.Source: §7.2 · p. 54

7.3 Data Path for Accelerator

The section describes a multiplier-free data path and evaluates power-of-two weights, then compares fixed-point, dynamic fixed-point, and minifloat approximations.

7.3 Data Path for Accelerator

  • The data path uses 4-bit parameters to specify input shifts, with barrel shifters enabling multiple-bit shifts in one clock cycle and no multipliers.The analysis retains full-precision layer inputs and outputs.Source: §7.3 · p. 55, Figure 7.2 · p. 55

7.4 Results

  • Power-of-two weights cause absolute accuracy drops of 0.1% for LeNet, 4.21% for CIFAR-10, and 3.65% for CaffeNet, while using 4-bit exponents.The weights replace multipliers with bit shifters and reduce weight storage from 32 bits to 4 bits.Source: Table 7.1 · p. 55, §7.4 · p. 56

8.1 Fixed Point Approximation

  • Fixed point requires the least energy and development time but performs worst at small bit-widths; CaffeNet output saturation rises from 0.45% at 15 bits to 2.82% at 14 bits.The passage links saturation of large outputs to a significant accuracy drop.Source: §8.1 · p. 58

8.2 Dynamic Fixed Point Approximation

8.3 Minifloat Approximation

Dynamic fixed point gives the best low-bit accuracy in the reported comparisons, with 4- or 8-bit representations for LeNet, CIFAR-10, and CaffeNet. Ristretto quantizes trained networks and fine-tunes them, with GPU-optimized routines supporting fast evaluation.

8.3 Minifloat Approximation

  • At 5-bit arithmetic, LeNet accuracy is 97.96%, but it drops to 10.09% at 4 bits when no mantissa bits remain.The passage attributes the sharp drop to insufficient exponent range; it says CIFAR-10 and CaffeNet need at least 6-bit minifloat numbers for good classification performance.Source: §8.3 · p. 59

8.4 Summary

  • Dynamic fixed point supports 4-bit or 8-bit representations for LeNet, CIFAR-10, and CaffeNet, reducing memory bandwidth and footprint.The passage says this is expected to yield energy savings for FPGA and ASIC designs.Source: §8.4 · pp. 59–60

9.1 From Caffe to Ristretto

  • Ristretto takes a trained Caffe model and automatically produces a condensed network description and parameters that can be used for a hardware accelerator.The condensed model remains in Caffe format.Source: §9.1 · p. 61

9.2 Quantization Flow

  • Ristretto analyzes weight ranges and layer activations, then searches for bit-widths for convolutional weights, fully connected weights, and layer outputs.For dynamic fixed point, it quantizes one network part at a time while the others remain in floating point, then selects a trade-off between representation size and classification accuracy.Source: §9.2 · pp. 61–62

9.3 Fine-tuning

  • Fine-tuning uses quantized parameters for image classification while retaining full-precision weights to accumulate updates smaller than the quantization step.It uses Adam; layer outputs remain floating point during fine-tuning, while scoring uses reduced-precision outputs.Source: §9.3 · pp. 62–63, Figure 9.2 · p. 63

9.4 Fast Forward and Backward Propagation

  • A fixed-point CaffeNet can be tested on the 50,000-image ILSVRC 2014 validation dataset in less than 2 minutes on one Tesla K-40 GPU.Ristretto uses full floating point for accumulation and optimized matrix-multiplication routines for forward and backward propagation.Source: §9.4 · pp. 63–64

9.5 Ristretto From a User Perspective

Ristretto extends Caffe with a pipeline that compresses trained networks and can fine-tune the resulting models. The release emphasizes automated trimming, supported quantization schemes, GPU speed, and possible future additions including pruning-related work and C-code generation.

9.5 Ristretto From a User Perspective

  • Ristretto starts with a trained network, quantizes it to a small-bit-width representation, and can fine-tune the condensed model to improve accuracy.Users can choose an error margin and approximation strategy; the condensed model uses Caffe format with Ristretto-specific limited-precision layers.Source: §9.5 · p. 65, fixed point approximation · p. 65

9.6 Release of Ristretto

  • The release highlights automatic CNN trimming, multiple trimming schemes, fine-tuning, and GPU execution with optimized CUDA routines.The source code is available.Source: §9.6 · pp. 65–66

9.7 Future Work

  • Ristretto's modular Caffe-based architecture is intended to make it easy to add limited-precision layer types.The section introduces possible future extensions.Source: §9.7 · p. 66

9.7.1 Network Pruning

  • Fully connected layers contain most network weights, and prior work reports that many of their connections can be removed without loss in classification accuracy.The passage identifies off-chip memory access as the most energy-costly operation for CNN accelerators.Source: §9.7.1 · p. 66

9.7.2 Binary Networks

  • Binary weights can approximate very deep networks, with reported accuracy drops of around 3% for CaffeNet and 6% for GoogLeNet.The stated challenge is high prediction accuracy without increasing parameter size or adding computational overhead.Source: §9.7.2 · p. 66

9.7.3 C-Code Generation

  • Ristretto plans to generate C-code and extracted low-precision parameters for condensed networks, which could feed an FPGA or ASIC high-level-synthesis workflow.The described standard workflow converts tested C implementations to System-C before HLS.Source: §9.7.3 · pp. 66–67
  • The authors hope open-source community contributions will add features and make Ristretto more powerful.They also hope the framework will help researchers speed up CNN-accelerator development.Source: §9.7.3 · p. 67
Loading 1605.06402v1…