PaperlayerView exact revision on arXiv

Source-linked AI summary

YodaNN: An Architecture for Ultra-Low Power Binary-Weight CNN Acceleration

Renzo Andri, Lukas Cavigelli, Davide Rossi, Luca Benini

arXiv:1606.05487v4cs.ARcs.CVcs.NE

TL;DR

CNN accelerators remain too power-hungry for mobile and IoT end-nodes, with weight I/O and storage creating additional constraints. The paper presents a flexible accelerator for binary-weight CNNs, replacing expensive multiplications and reducing weight data. It reports 1510 GOp/s at 1.2 V, 895 μW at 0.6 V, and 61.2 TOp/s/W peak energy efficiency.

  • Problem

    Existing CNN platforms exceed mobile and IoT end-node power envelopes, while CNN weight I/O and storage impose hard limitations.

  • Method

    The paper designs a flexible, performance-scalable convolutional accelerator using binary weights, simpler complement operations and multiplexers, and latch-based on-chip storage.

  • Results

    1510 GOp/s at 1.2 V, 895 μW at 0.6 V, and 61.2 TOp/s/W peak energy efficiency are reported for the accelerator.

  • Takeaways & Limitations

    Binary-weight CNN hardware substantially improves the reported performance, area efficiency, and energy efficiency of CNN acceleration for ultra-low-power applications.

Abstract

from arXiv · show

Convolutional neural networks (CNNs) have revolutionized the world of computer vision over the last few years, pushing image classification beyond human accuracy. The computational effort of today's CNNs requires power-hungry parallel processors or GP-GPUs. Recent developments in CNN accelerators for system-on-chip integration have reduced energy consumption significantly. Unfortunately, even these highly optimized devices are above the power envelope imposed by mobile and deeply embedded applications and face hard limitations caused by CNN weight I/O and storage. This prevents the adoption of CNNs in future ultra-low power Internet of Things end-nodes for near-sensor analytics. Recent algorithmic and theoretical advancements enable competitive classification accuracy even when limiting CNNs to binary (+1/-1) weights during training. These new findings bring major optimization opportunities in the arithmetic core by removing the need for expensive multiplications, as well as reducing I/O bandwidth and storage. In this work, we present an accelerator optimized for binary-weight CNNs that achieves 1510 GOp/s at 1.2 V on a core area of only 1.33 MGE (Million Gate Equivalent) or 0.19 mm$^2$ and with a power dissipation of 895 μW in UMC 65 nm technology at 0.6 V. Our accelerator significantly outperforms the state-of-the-art in terms of energy and area efficiency achieving 61.2 TOp/s/W@0.6 V and 1135 GOp/s/MGE@1.2 V, respectively.

I. INTRODUCTION

CNNs deliver strong accuracy but their computation, weight storage, and data movement exceed the power envelope of mobile and IoT end-nodes. Binary-weight CNNs offer hardware opportunities by simplifying arithmetic and reducing weight storage, motivating YodaNN’s specialized accelerator.

  • Mobile and IoT end-nodes cannot meet the power demands of conventional CNN platforms, while offloading computation requires costly data-stream bandwidth.
  • Binary-weight CNNs replace multipliers with complement operations and multiplexers while drastically reducing weight storage requirements.
  • Binary-weight networks incur only small accuracy losses on several well-known CNN benchmarks.
  • YodaNN introduces a flexible, energy-efficient, performance-scalable accelerator optimized for binary-weight CNNs.
  • 5.1× higher digital-core energy efficiency and 1.3× higher throughput are demonstrated against a 12-bit MAC baseline at 1.2 V.
  • 1.5 TOp/s peak performance, 1.1 TOp/s/MGE peak area efficiency, and 61.2 TOp/s/W peak energy efficiency surpass the cited state-of-the-art accelerators.

B. CNN Acceleration Hardware

CNN accelerators span power-hungry GPUs, programmable FPGAs, and specialized hardware, but IoT end-nodes require substantially better energy efficiency. YodaNN addresses this gap through binary weights, convolution-focused data reuse, and flexible hardware support.

  • GPUs reach up to 6 TOp/s at 250 W, while low-power programmable architectures remain above IoT end-node energy budgets.
  • Convolution layers receive primary hardware focus because they constitute the largest share of CNN computation time.
  • Sliding-window schemes and tile-to-tile caching reuse recent pixel values to reduce convolution data movement.
  • Zero-padding can load unnecessary data and leave hardware unused when supporting filters smaller than a fixed kernel size.
  • Compressed networks, weight sharing, pruning, and PCA reduce data or computation, but can require indices or on-the-fly reconstruction.
  • YodaNN’s binary-weight design is reported as 12× more efficient than prior state-of-the-art and 43× more energy-efficient than the PCA-based architecture cited.
  • The accelerator is reported to outperform the state of the art by 2.7× in performance, 10× in area efficiency, and 32× in energy efficiency.

III. ARCHITECTURE

The architecture processes convolution layers through parallel channel-wise computation, buffered image windows, binary filter storage, accumulation, and interleaved output scaling.

  • Architecture: Convolution layers dominate total computation time, motivating the accelerator’s focus on this layer.The layer maps nin input channels to nout output channels through nin · nout kernels, accumulating per-input-channel convolution terms into each output channel.
  • Architecture: The filter bank stores binary weights for up to 32 input and 32 output channels and supports column-wise circular shifts.Its shift-register organization aligns filter columns with the spatial processing order.
  • Architecture: The image memory stores an image stripe, while the image bank caches each 7 × 7 spatial window to reduce memory accesses.The image bank reuses six previously stored rows when traversing columns, loading only the lowest row from image memory.
  • Architecture: 32 SoP units calculate one partial convolution per output channel while input-channel contributions are accumulated by 32 channel summers.Each SoP calculates a contribution for a new input channel, and its corresponding channel summer accumulates contributions across all input channels.
  • Architecture: After channel accumulation, the Scale-Bias Unit applies per-channel scaling and biasing and streams outputs, while the I/O interface handles 12-bit streams.The input interface manages one 12-bit input stream and two 12-bit output streams using blocking ready-valid handshaking.

A. Dataflow

The dataflow partitions channels and image height into chip-sized blocks, reuses sliding-window data, computes channel contributions in parallel, and streams scaled outputs after accumulation.

  • A. Dataflow: The dataflow splits input and output channels into blocks below 32 × 32 and divides the image into height slices.Processing iterates over output-channel blocks, input-channel blocks, and image-height blocks before loading filters and image columns.
  • A. Dataflow: After preloading border-dependent columns and pixels, SoP units calculate partial sums while changing the input channel every cycle.The channel summers retain the completed sums for all 32 output channels at each row and column.
  • A. Dataflow: Completed output-channel sums are scaled, biased, and streamed out in an interleaved manner.The pseudo-code applies ocout(x, y) = αcout˜ocout(x, y) + βcout before output.
  • A. Dataflow: Partial sums from separate input-channel blocks are summed to produce final output channels.This accumulation occurs after all channel blocks have been processed.
  • A. Dataflow: The sliding-window scheme shifts weights rather than the full image memory, reusing recent pixels across adjacent columns.Only newly needed pixels are inserted while the weight permutation aligns operands for the next column.

B. BinaryConnect Approach

The BinaryConnect approach replaces full-precision convolution weights with binary values, reducing filter-storage and arithmetic requirements relative to a 12-bit MAC baseline.

  • B. BinaryConnect Approach: BinaryConnect remaps accelerator weights to binary values wk,n ∈ {−1, 1}.This is the principal architectural change relative to the equivalent 12-bit implementation.
  • B. BinaryConnect Approach: Binary weights reduce filter-bank storage and allow multipliers to be replaced by two’s-complement operations and multiplexers.The resulting products remain 12-bit wide rather than 24-bit, reducing multiplier and adder-tree size.
  • B. BinaryConnect Approach: The baseline comparison uses 12-bit MAC architecture with 8 × 8 channels and 7 × 7 filters.The binary design feeds the SoP units with 12-bit image windows and 7 × 7 binary weights.

C. Latch-Based SCM

The accelerator replaces SRAM-based image memory with latch-based SCMs to improve voltage scalability and reduce memory energy, despite higher area cost.

  • Organization: 6×8 banks of 128 rows store 12-bit values in the multi-banked SCM image memory.Pre-decoding addresses the appropriate bank of the array.
  • Power control: Hierarchical clock gating and address/data silencing prevent dynamic power consumption in unaccessed SCM banks.Each bank uses a two-ported, single-cycle-latency architecture.
  • Voltage scalability: 0.6 V–1.2 V is the SCM operating range, extending the convolutional engine below the 0.8 V SRAM limit.SCMs operate across the technology’s full range, while SRAM limits voltage scalability to 0.8 V.
  • Energy efficiency: 3.25× lower memory power at 1.2 V is achieved versus SRAM through SCM-based image storage.Only up to 7 of 48 SCM banks consume dynamic power in each cycle.

D. Considering I/O Power in Energy Efficiency

Because I/O power can dominate CNN accelerator consumption, the design increases channel parallelism and reuses data to reduce bandwidth and improve throughput.

  • I/O-power motivation: More than 30% of overall chip power can come from I/O, making bandwidth and pad power increasingly important after binary computation reduces arithmetic complexity.The binary approach makes I/O power more critical rather than eliminating it.
  • Data reuse: Increasing output channels executes more operations on the same data, reducing required bandwidth and pad power consumption.The same increase also improves throughput.

E. Support for Different Filter Sizes, Zero-Padding, Scaling and Biasing

The accelerator supports multiple kernel sizes and common CNN post-processing operations while accounting for border, tiling, and channel-utilization effects in efficiency estimates.

  • Zero-padding: 1×1, 2×2, 4×4, and 6×6 kernels are enabled by zero-padding unused image-memory columns and image-bank rows.The same mechanism adds zeros at image borders for convolution.
  • Scaling and biasing: Up to two output channels are calculated in parallel per SoP unit, followed by channel scaling and biasing in fixed-point arithmetic.Feature maps remain at maximum precision on-chip before scaling and biasing.
  • Efficiency measures: 2×n_out×n_in×h_k×w_k×(h_in−h_k+1)×(w_in−w_k+1) counts separate additions and multiplications for one CNN frame.Memory accesses and zero-padding operations are excluded from this complexity formula.
  • Tiling: Image tiles overlap by h_k−1 rows when the full image height exceeds image-window memory, causing overlapped rows to load twice.This overlap determines the tiling-throughput penalty.
  • Channel utilization: Input and output channels are partitioned into parallel blocks, with block outputs summed pixel-wise outside the accelerator.Channel idling affects throughput but not energy efficiency when fewer input channels are used.
  • Border efficiency: Zero-padding preserves η_border = 1 despite preloading latency, whereas nonzero-padded layers lose border efficiency.The same number of columns processed after the image can overlap with preloading the next image.

B. Experimental Setup

The experimental setup combines a control-signal testbench with a fabricated YodaNN core whose floorplan, memory, and operating-frequency characteristics are reported.

  • Testbench: The testbench streams filters and input images from raw files, then compares chip outputs with expected feature maps.Expected responses are generated using a bit-true quantized spatial convolution implementation.
  • Floorplan: 1.3 MGE (1.9 mm^2) is the reported core area, with SCM memory, binary filter storage, SoP units, and image memory as the main components.The SCM memory occupies 480 kGE, while the binary weights filter bank occupies 333 kGE.
  • Operating point: 480 MHz at 1.2 V and 27.5 MHz at 0.6 V are the reported maximum operating frequencies.The floorplan is shown in Figure 10, which identifies a 9.2 KiB SCM memory computing 32 output channels in parallel.

C. Fixed-Point vs. YodaNN

YodaNN’s binary architecture reduces arithmetic and memory costs relative to a fixed-point baseline, improving throughput and energy efficiency while enabling operation below the SRAM voltage limit.

  • C. Fixed-Point vs. YodaNN: 377 GOp/s at 1.2 V increases peak throughput from 348 GOp/s in the fixed-point baseline.The binary design also reduces core power to 39 mW, a 79% reduction.
  • C. Fixed-Point vs. YodaNN: 5.1× better core energy efficiency and 1.3× better core area efficiency result from reduced arithmetic complexity and latch-based memory.Replacing SRAM with SCM shortens the critical path and reduces pipeline stages from three to one.
  • C. Fixed-Point vs. YodaNN: 59 TOp/s/W at 0.6 V is achieved with SCM, despite peak throughput dropping to 15 GOp/s.The 0.6 V operating point improves energy efficiency by 11.6× relative to the fixed-point architecture at 0.8 V.
  • C. Fixed-Point vs. YodaNN: Power falls from 185 mW to 39 mW when comparing the 8×8-channel fixed-point and binary-weight variants.The reduction is attributed to lower power in SCM, sum-of-products units, and the filter bank.
  • C. Fixed-Point vs. YodaNN: 4× higher throughput from 32×32 rather than 8×8 channels improves energy efficiency by 20%, although core power increases 3.32×.Supporting different kernel sizes increases core area by 11.2% and core power by 38% versus a binary 7×7-only design.

D. Real Applications

YodaNN is evaluated on binary-weight CNNs spanning benchmark and well-known networks, using channel blocking and specialized handling for challenging layers and kernels. Energy-optimal operation delivers high efficiency across most networks, while AlexNet is limited by its first layer.

  • D. Real Applications: BinaryConnect, VGG-13, VGG-19, ResNet-18, ResNet-34, and AlexNet are evaluated as real-life binary-weight CNNs.Layers are split into channel blocks, generally using 32×32 or 32×64 input-output channel configurations.
  • D. Real Applications: 48.1–56.7 TOp/s/W is achieved by most networks, while AlexNet reaches 14.1 TOp/s/W.AlexNet’s lower efficiency is associated with its computationally demanding first layer and accelerator idling.
  • D. Real Applications: 1428 FPS is reached for BinaryConnect-SVHN in the throughput-optimal setting, while VGG-19 reaches 13.3 FPS.The throughput-optimal configuration operates with chip power of 153 mW.
  • D. Real Applications: 0.5–53.2 FPS is achieved across the listed CNNs at the maximum-energy-efficiency corner with 895 µW power.At this operating point, throughput is about half the maximum possible 55 GOp/s for most networks.

E. Comparison with State-of-the-Art

YodaNN compares favorably with state-of-the-art CNN accelerators in peak throughput, area efficiency, and energy efficiency. Its voltage scaling produces a Pareto front while binary arithmetic and SCM support flexible, low-power operation.

  • E. Comparison with State-of-the-Art: 1.5 TOp/s peak throughput exceeds NINEX by 2.7×, while 61.2 TOp/s/W at 0.6 V improves energy efficiency by more than an order of magnitude over prior results.YodaNN forms a clear Pareto front over state-of-the-art accelerators across 1.2–0.6 V.
  • E. Comparison with State-of-the-Art: 12× higher energy efficiency and 28× higher area efficiency than EIE are reported despite EIE assuming 97% sparsity.The comparison is presented as part of the quantitative state-of-the-art evaluation.
  • V. CONCLUSION: Seven kernel sizes from 1×1 through 7×7 increase flexibility for evaluating varied CNNs, while reducing energy efficiency by 29%.The architecture replaces fixed-point MAC units with complement operations and multiplexers and uses latch-based SCMs for lower-voltage scaling.
  • E. Comparison with State-of-the-Art: 1.1 TOp/s/MGE peak area efficiency and 61.2 TOp/s/W peak energy efficiency surpass state-of-the-art accelerators by 10× and 32×, respectively.At 0.6 V, power consumption is 895 µW, with an average frame rate of 11 FPS for state-of-the-art CNNs.
Loading 1606.05487v4…