PaperlayerView exact revision on arXiv

Source-linked AI summary

Origami: A 803 GOp/s/W Convolutional Network Accelerator

Lukas Cavigelli, Luca Benini

arXiv:1512.04295v2cs.CVcs.AIcs.LGcs.NE

TL;DR

Embedded and mobile vision need efficient hardware for increasingly demanding convolutional networks, whose computation and memory bandwidth challenge practical deployment. The paper presents a scalable accelerator architecture with a silicon implementation that reduces bandwidth requirements and reports high throughput and power efficiency. The manufactured ASIC achieves up to 196 GOp/s and 803 GOp/s/W in its high-efficiency configuration.

  • Problem

    Embedded and mobile deployment of increasingly demanding ConvNets is constrained by computational and memory-bandwidth requirements, while storing state-of-the-art parameters on-chip is infeasible.

  • Method

    The paper develops a massively parallel convolutional accelerator that computes multiple input and output channels in parallel using configurable input stripes, SRAM windows, register-based filter windows, and sum-of-product units.

  • Results

    803 GOp/s/W is achieved in the high-efficiency configuration, while the ASIC reaches 196 GOp/s peak throughput in its high-speed configuration.

  • Takeaways & Limitations

    The architecture reduces external-memory bottlenecks and is presented as scalable to multi-TOp/s performance while improving area efficiency and power consumption relative to prior work.

Abstract

from arXiv · show

An ever increasing number of computer vision and image/video processing challenges are being approached using deep convolutional neural networks, obtaining state-of-the-art results in object recognition and detection, semantic segmentation, action recognition, optical flow and superresolution. Hardware acceleration of these algorithms is essential to adopt these improvements in embedded and mobile computer vision systems. We present a new architecture, design and implementation as well as the first reported silicon measurements of such an accelerator, outperforming previous work in terms of power-, area- and I/O-efficiency. The manufactured device provides up to 196 GOp/s on 3.09 mm^2 of silicon in UMC 65nm technology and can achieve a power efficiency of 803 GOp/s/W. The massively reduced bandwidth requirements make it the first architecture scalable to TOp/s performance.

I. INTRODUCTION

Deep convolutional networks are delivering strong results across computer-vision tasks, increasing demand for energy-efficient embedded acceleration. The paper presents a scalable accelerator architecture designed to reduce area, energy, and I/O demands while supporting ConvNet computation.

  • Deep-learning methods are achieving record-breaking results on challenging computer-vision problems and datasets.
  • Embedded vision systems process image data on board, reducing transmitted data and human intervention compared with centralized video processing.
  • The proposed accelerator targets TOP/s scalability while remaining area- and energy-efficient and keeping I/O throughput within economical-package and low-power limits.
  • The architecture is implemented with optimized fixed-point precision for an accelerator-sized ASIC and experimentally characterized through silicon measurements.
  • A ConvNet stage combines convolution, activation, and pooling, with convolution serving as the paper’s key operation.

A. Measuring Computational Complexity

The paper measures ConvNet complexity in operations and distinguishes practical throughput from peak and theoretical throughput. Its reference scene-labeling network has three stages and processes 7.57 GOp per frame.

  • 2 operations are counted for each multiplication and addition when reporting ConvNet complexity.
  • Actual throughput is measured on a real network, peak throughput uses a synthetic benchmark, and theoretical throughput ignores bandwidth limits.
  • The reference scene-labeling ConvNet has three stages, uses 240 × 320 input images, and requires 7.57 GOp/frame.

B. Computational Effort

Real-time embedded ConvNet inference requires substantial computation, while convolution dominates optimized execution time. The section motivates specialized acceleration around this persistent computational bottleneck.

  • 189 GOp/s is required to run the reference scene-labeling ConvNet at 25 frame/s, beyond recent commercial mobile processors.
  • Convolution accounts for approximately 89% of optimized CPU computation time and 79% of GPU computation time.
  • Activation, pooling, and pixel-wise classification with fully connected layers require negligible computation time relative to convolution.
  • ConvNets achieve low top-5 error rates on ImageNet, including 6.67% for GoogLeNet and 7.32% for VGG Oxfordnet at ILSVRC 2014.

A. Software Implementations (CPU, GPU, DSP)

CPU, GPU, DSP, FPGA, and ASIC implementations provide different performance and efficiency trade-offs, but memory bandwidth and general-purpose architecture constrain efficient ConvNet execution. These constraints motivate specialized accelerators.

  • 96 GOp/s is achievable on Tegra K1, with 76 GOp/s on an actual ConvNet and approximately 7 GOp/s/W platform energy efficiency.
  • GPU implementations improve memory bandwidth using half-precision storage, while bandwidth remains a limiting factor.
  • Fourier-transform convolution can help for kernels larger than 9 × 9 but generally worsens bandwidth and memory requirements.
  • General-purpose software is constrained by fixed arithmetic precision, caches, and instruction handling, motivating specialized architectures for power and area efficiency.
  • NeuFlow achieved 147 GOp/s at 11 W on a Virtex 6 VLX240T using 16-bit fixed-point arithmetic.

C. ASIC Implementations

Prior convolutional-network accelerators demonstrated energy efficiency, but their scalability was constrained by performance dependence, wide memory interfaces, and insufficient on-chip storage for modern networks. Origami addresses these constraints while emphasizing silicon-oriented evaluation and comparison.

  • ShiDianNao performance strongly depends on convolutional-layer size, limiting its performance expansion to tiny layers.
  • ConvEngine achieves 409 GOp/s/W in synthesis results, while several commercial vision IP cores had not been implemented in silicon or disclosed their architectures.
  • 1.93 TOp/s/W was reported for K-Brain in 65 nm, but its 216 KB weight and dataset SRAM is far smaller than modern network parameter sets.
  • 299 I/O pins are required to achieve 320 GOp/s using NeuFlow, illustrating the interface burden of prior accelerator architectures.
  • 6.8 M and 135 M parameters in GoogLeNet and VGG Oxfordnet, respectively, make storing learned parameters on-chip infeasible for state-of-the-art networks.
  • Prior work on word width and compression was validated only on very small datasets, including MNIST and CIFAR-10.

IV. ARCHITECTURE

Origami streams channelized image data through SRAM and a register-based image bank into parallel sum-of-products units, accumulating results across input channels. Its tiled organization improves throughput per bandwidth while balancing filter storage, area, and I/O efficiency.

  • Concept of Operation: Input images are fed in configurable-height stripes, stored in SRAM, moved through a register-based image window, and supplied to sum-of-products units.Each sum-of-products unit processes one image channel with a different filter, producing partial sums for distinct output channels.
  • Accumulation: Partial sums are accumulated in channel summer units across input channels, then complete results are transmitted out of the circuit.
  • Tiled Convolution: 2n_ch^2h_kw_k operations are performed every n_ch clock cycles while transmitting and receiving n_ch values instead of n_ch^2.This tiling improves throughput per bandwidth by a factor of n_ch.
  • Image Window SRAM and Image Bank: The image bank stores n_chh_kw_k values and loads a new row of w_k current-channel elements from SRAM into the bank each cycle.
  • Filter Bank: The filter bank uses registers and a multiplexer because all filter values supplied to the sum-of-products units must change every cycle and cannot remain in SRAM.
  • Filter Bank: Doubling n_ch and I/O bandwidth quadruples filter-bank storage while doubling I/O efficiency, exposing an area-versus-bandwidth trade-off.
  • Sum-of-Products Units: The sum-of-products unit computes an image-patch/filter-kernel inner product using h_kw_k multipliers and h_kw_k − 1 adders arranged as a tree.

B. Optimizing for Area Efficiency

Origami improves area efficiency by running computation units at twice the SRAM and I/O frequency, while accounting for stalls from borders, filter loading, and channel blocking. These effects determine throughput on the reference network.

  • Clocking and throughput: Peak throughput is set by the number of SoP units, their wk × hk operations per cycle, and the fast clock, but stalls reduce realized throughput.The architecture is designed so other blocks sustain peak computation, while border effects, loading, and channel utilization can cause stalls.
  • Throughput losses: Border effects make effective throughput depend on image size because the core waits for initial columns and per-column pixel data.The waiting periods preload wk −1 columns at each image start and hk −1 pixels at each new column.
  • Throughput losses: Filter loading adds an efficiency loss because filter weights share the image-data bus and require additional transfer cycles.For nch = 8, the resulting filter-loading factors are 0.99, 0.98, and 0.91 for the three stages.
  • Throughput losses: Channel blocking can reduce utilization when channel counts do not match the nch × nch processing blocks.With nch = 8, Stage 1 has only 3 input channels and loads the core to ηblocks = 3/8, although stages 2 and 3 split perfectly.

D. System Architecture

The proposed system combines Origami chips with FPGA preprocessing, memory control, and post-processing to analyze camera streams. Its architecture reduces memory traffic through shared inputs and early summation, supporting SoC integration and lower system cost and energy use.

  • System goals: The system accepts camera video, processes images with ConvNets, displays results, and transmits alerts or data for further analysis.Targeted ConvNet tasks include scene labeling, object detection, recognition, and tracking.
  • General architecture: A reference configuration uses four Origami chips, a 32 bit 800 MHz DDR3 or LPDDR3 memory, and an FPGA preprocessing core.The FPGA performs rescaling, color-space conversion, and local contrastive normalization.
  • General architecture: The FPGA manages DMA and memory control, while accumulating partial results and applying bias, ReLU, and max-pooling after convolution.Fully connected layers use FPGA multipliers but account for less than 2% of operations in the scene-labeling example.
  • General architecture: Tiling partitions each ConvNet stage into blocks of height hin,max, nch input channels, and nch output channels before reassembly.Blocks are summed over input channels and reassembled across output channels and horizontal stripes.
  • Bandwidth considerations: Sharing input and output feature-map data across chips reduces memory traffic before partial outputs are summed and written back.The same limitations described for the chip also apply at system level.
  • Bandwidth considerations: Low memory bandwidth enables SoC integration with reduced system size and cost and improved energy efficiency.A narrow, moderate-bandwidth memory interface also reduces packaging cost.

V. IMPLEMENTATION

The implementation section introduces the circuit implementation, fixed-point analysis, implementation figures, and whole-system considerations.

  • Implementation overview: The implementation section covers circuit design, fixed-point precision selection, implementation figures, and system-level implementation aspects.It presents these topics in sequence, from general circuit implementation through fixed-point analysis and system considerations.

A. General Implementation

The implementation uses separate clock domains, a 7 × 7 filter configuration, and a 512-pixel maximum stripe height. Fixed-point analysis selects 12-bit signals, while the complete four-chip system requires 750 MB/s full-duplex bandwidth.

  • Clocking: 500 MHz SoP units and channel summers run at twice the 250 MHz SRAM and I/O clock to improve area efficiency.Both clocks originate from the same source, with the slower clock driving memory and other lower-speed elements.
  • Chip configuration: The taped-out chip uses hk = wk = 7 and hin,max = 512, requiring an image-window SRAM of 29k words.The SRAM is split into four 1024-word blocks with 7 · 12-bit words.
  • Fixed-point precision: 12-bit input, output, and filter signals keep implementation accuracy loss below 0.5% for the reference ConvNet.The design fixes filter size at 7 × 7 and processes 8 input and output channels in parallel.
  • System bandwidth: The four-chip implementation requires 750 MB/s full-duplex bandwidth at 250 MHz with 12-bit words and nch = 8.Feature-map sharing saves a factor of 2, while partial sums are read back and accumulated for final convolution results.

VI. RESULTS

Origami was implemented and measured in 65 nm silicon, demonstrating configurable throughput and energy-efficiency trade-offs across operating points. The chip reaches 196 GOp/s peak throughput and 803 GOp/s/W in its high-efficiency configuration.

  • Implementation: More than one-third of the area was devoted to the filter bank, and the sum-of-products units occupied almost another third of the circuit.The filter bank included 0.41 mm^2 of filter-weight registers and 0.03 mm^2 of multiplexers.
  • Silicon Measurements: 196 GOp/s peak throughput was measured at 500 MHz in the high-speed configuration, while a reference ConvNet achieved 145 GOp/s at 448 mW.The ASIC was taped out in UMC 65 nm CMOS technology.
  • Implementation: 1.31 mm^2 was occupied by logic and on-chip memory, while the generously chosen core area was 3.09 mm^2 for the 55-pin pad frame.The corresponding throughput density was 150 GOp/s/mm^2.
  • Silicon Measurements: 803 GOp/s/W was measured in the high-efficiency configuration at 93 mW, with 74 GOp/s peak throughput and 55 GOp/s on the reference ConvNet.This configuration required 142 MB/s full-duplex I/O bandwidth.
  • Operating Points: Throughput and energy efficiency could be continuously traded off by changing core supply voltage, with slightly worse measurements at higher temperature.Static power was about 1.25% across the voltage range at 25°C and about 10.5% in the interval [0.95 V, 1.25 V].

VII. DISCUSSION

The discussion attributes Origami’s efficiency to parallel channel computation, reduced memory traffic, and a focus on external bandwidth. Comparisons indicate strong area and bandwidth efficiency, while power-efficiency comparisons depend substantially on technology and I/O power.

  • A. Area Efficiency: 90.7 GOp/s/MGE made Origami the most area-efficient reported ConvNet accelerator, compared with 33.8 GOp/s/MGE for the next-best NeuFlow design.The standard cells and on-chip SRAM occupied 1.31 mm^2, excluding the pad-frame-limited area.
  • A. Area Efficiency: Computing multiple input and output channels in parallel reduced buffering from 64 input images to a window of 8 input channels for 64 convolutions.Using 12-bit rather than 16-bit words contributed another 25% improvement, mainly in SRAM and filter-buffer size.
  • B. Bandwidth Efficiency: 521 GOp/GB provided more than a 10× improvement over NeuFlow’s 50 GOp/GB bandwidth efficiency.The architecture reuses input data by calculating pixels for multiple output channels in parallel.
  • C. Power Efficiency: Technology scaling substantially affected energy-efficiency comparisons between Origami, NeuFlow, and HWCE.The authors used a simple projection to compare implementations at 28 nm, noting that the model was not highly accurate.
  • C. Power Efficiency: Including I/O power reduced the chip’s efficiency to 632 GOp/s/W in high-efficiency operation and 342 GOp/s/W in high-throughput operation.The comparison estimates I/O power as 42% to 82% of total power for the three evaluated devices.

VIII. CONCLUSIONS & FUTURE WORK

The paper reports first silicon measurements for a convolutional network accelerator and presents an architecture designed to overcome external-memory bottlenecks. It concludes that the architecture improves area efficiency and supports multi-TOp/s scaling, with further configurability and low-precision training left for future work.

  • Conclusions: The work presents the first reported silicon measurement results of a convolutional network accelerator.The conclusion identifies this as a central contribution of the paper.
  • Conclusions: The architecture is described as the first to scale to multi-TOp/s performance by significantly improving on the external memory bottleneck.This addresses the scaling limitation emphasized in the conclusion.
  • Conclusions: The chip is reported to be more area-efficient than previously reported results and to have the lowest reported power consumption after technology scaling compensation.These are comparative conclusions stated by the authors.
  • Future Work: Future work includes newer technologies, programmable logic, greater configurability, and adapting networks for very-low-precision accelerators during training.The authors frame these directions as planned investigations and development.
Loading 1512.04295v2…