PaperlayerView exact revision on arXiv

Source-linked AI summary

FracBNN: Accurate and FPGA-Efficient Binary Neural Networks with Fractional Activations

Yichi Zhang, Junhao Pan, Xinheng Liu, Hongzheng Chen, Deming Chen, Zhiru Zhang

arXiv:2012.12206v1cs.LGcs.CV

TL;DR

BNNs offer efficient bitwise computation and reduced model memory, but their accuracy on realistic datasets lags compact CNNs, and their input layer is commonly left unbinarized. FracBNN combines fractional activations with thermometer-encoded input binarization and an FPGA accelerator; on ImageNet it reports gains over prior BNNs while reaching MobileNetV2-level accuracy.

  • Problem

    BNNs have lower accuracy than compact CNNs on realistic datasets, while their input layers are commonly left unbinarized to avoid accuracy loss.

    Source: §1 · p. 1, §2.2 · p. 3

  • Method

    FracBNN uses two-bit fractional activations with an extra sparse binary convolution, thermometer-encodes the input layer, and computes all convolutions in BMACs.

    Source: §1 · p. 2

  • Results

    FracBNN improves ImageNet top-1 accuracy by 28.9% over the best-known FPGA-targeted BNN and 2.4% over ReActNet, while achieving MobileNetV2-level accuracy.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The full network runs on an embedded Xilinx Ultra96 v2 FPGA at 48.1 fps, demonstrating real-time image classification.

    Source: §1 · p. 2

    The ReActNet baseline retains a floating-point input layer and trails compact MobileNetV2 in accuracy.

    Source: §2.2 · p. 3

Abstract

from arXiv · show

Binary neural networks (BNNs) have 1-bit weights and activations. Such networks are well suited for FPGAs, as their dominant computations are bitwise arithmetic and the memory requirement is also significantly reduced. However, compared to start-of-the-art compact convolutional neural network (CNN) models, BNNs tend to produce a much lower accuracy on realistic datasets such as ImageNet. In addition, the input layer of BNNs has gradually become a major compute bottleneck, because it is conventionally excluded from binarization to avoid a large accuracy loss. This work proposes FracBNN, which exploits fractional activations to substantially improve the accuracy of BNNs. Specifically, our approach employs a dual-precision activation scheme to compute features with up to two bits, using an additional sparse binary convolution. We further binarize the input layer using a novel thermometer encoding. Overall, FracBNN preserves the key benefits of conventional BNNs, where all convolutional layers are computed in pure binary MAC operations (BMACs). We design an efficient FPGA-based accelerator for our novel BNN model that supports the fractional activations. To evaluate the performance of FracBNN under a resource-constrained scenario, we implement the entire optimized network architecture on an embedded FPGA (Xilinx Ultra96v2). Our experiments on ImageNet show that FracBNN achieves an accuracy comparable to MobileNetV2, surpassing the best-known BNN design on FPGAs with an increase of 28.9% in top-1 accuracy and a 2.5x reduction in model size. FracBNN also outperforms a recently introduced BNN model with an increase of 2.4% in top-1 accuracy while using the same model size. On the embedded FPGA device, FracBNN demonstrates the ability of real-time image classification.

1 INTRODUCTION

BNNs offer hardware and memory efficiency, but their accuracy on realistic datasets and the floating-point input layer pose challenges. FracBNN addresses both with fractional activations, input binarization, and an FPGA accelerator.

1 INTRODUCTION

  • BNNs’ accuracy on ImageNet remains below compact CNNs: binarized AlexNet reaches 36.1%, FPGA-accelerated AlexNet 42.9%, and ReActNet 69.4%.ReActNet has 29.3 million parameters, while MobileNetV2 reaches 72% top-1 accuracy with 3.4 million parameters.Source: §1 · p. 2
  • FPGA input-layer computation is difficult to parallelize with limited DSP resources, and a dedicated floating-point engine cannot be reused by other layers.Fixed-point input-layer quantization has also incurred accuracy loss on realistic datasets.Source: §1 · p. 2
  • FracBNN uses an extra sparse binary convolution to update a fraction of features with two-bit activations, improving BNN accuracy while retaining BMAC convolution.Its dual-precision scheme computes features with up to two bits.Source: §1 · p. 2
  • The HLS accelerator runs the full FracBNN on an embedded Xilinx Ultra96 v2 FPGA at 48.1 fps.The paper reports real-time image classification and frame-rate advantages over cited accelerators.Source: §1 · p. 2
  • FracBNN improves ImageNet top-1 accuracy by 28.9% over the best-known FPGA-targeted BNN and by 2.4% over ReActNet, while reaching MobileNetV2-level accuracy.The approach keeps all convolutional layers in BMACs.Source: §1 · p. 2
  • Thermometer encoding binarizes the input layer while preserving feature similarity and causing minimal accuracy degradation.This avoids leaving the input layer outside the BMAC computation.Source: §1 · p. 2

2 BNN PRELIMINARIES

The preliminaries introduce conventional BNN computation and ReActNet as a more accurate BNN design. They also identify accuracy, model size, and input-layer precision as remaining concerns for FPGA implementations.

2.1 Conventional BNN Models

  • Conventional BNNs binarize weights and activations, turning multiplication and accumulation into XNOR and popcnt operations that map efficiently to FPGA LUT fabric.Their small memory footprint is useful given embedded FPGAs’ limited on-chip SRAM.Source: §2.1 · p. 2
  • ImageNet BNNs based on binarized AlexNet or VGG often exceed 10 MB and have low accuracy, limiting fit within modern embedded FPGAs’ on-chip SRAM.The stacked block consists of a sign function, binary convolution, and normalization.Source: §2.1 · p. 2

2.2 An Improved BNN Model

  • ReActNet uses MobileNet architecture and BPReLU to shift and reshape features between convolutions; it reaches 69.4% ImageNet top-1 accuracy with 4.82G BMACs and a 4.6 MB model.It retains a floating-point input layer and remains below compact MobileNetV2, which has 72% top-1 accuracy and 3.4 million parameters.Source: §2.2 · p. 3

3 THE FRACTIONAL BNN MODEL

FracBNN combines a binary input layer and sparse fractional convolutions to preserve binary MAC computation across convolutional layers while improving accuracy. On ImageNet, it reaches MobileNetV2-level accuracy and outperforms prior BNNs.

3.1 New Building Blocks

  • On CIFAR-10, the proposed ResNet-20 building block achieves 87.2% accuracy, the highest among the compared binarized ResNet-20 variants.The reported improvement over ReActNet’s structure is attributed to activation placement and balancing both branches’ distributions.Source: §3.1 · p. 4, Table 1 · p. 4

3.2 Binary Input Layer

  • Thermometer encoding binarizes input images using flexible-length vectors, avoiding the per-bit weight loss of direct 8-bit unpacking.With 96 input channels, it preserves model accuracy and uses 2.7× fewer input-layer BMACs than BIL.Source: §3.2 · pp. 4–5
  • Thermometer encoding improves input-layer feature similarity: with 96 channels, pre- and post-binarization dot products are highly correlated, unlike RGB inputs.The section links this preserved similarity to minimal accuracy degradation after binarization.Source: §3.2 · pp. 4–5

3.3 Fractional Convolution

  • FracConv stores 2-bit activations and conditionally applies an additional BMAC using the LSB and the same 1-bit kernels to update base-phase outputs.A channelwise threshold scores output-feature importance; sparse updates keep additional computation limited while weights remain binary.Source: §3.3 · pp. 5–6
  • FracBNN’s update convolution has 56% sparsity, corresponding to approximately 1.4-bit equivalent activation precision on ImageNet.The model has no floating-point MACs; only its classifier uses integer MACs.Source: §3.3 · p. 6
  • On ImageNet, FracBNN is 28.9% higher in top-1 accuracy than FP-BNN and achieves the same accuracy level as MobileNetV2.The section reports that all convolutional layers use pure BMACs.Source: §3 · p. 3, §3.3 · p. 6
  • At similar model size, FracBNN is 6% and 2.4% higher in top-1 accuracy than MeliusNet-29 [2] and ReActNet-A [30], respectively.It is also 2.5× smaller and 10.8% higher in accuracy than the six-model BNN Ensemble.Source: §3.3 · p. 6

4 FRACBNN ACCELERATOR DESIGN

FracBNN’s accelerator supports the network’s convolutional and auxiliary operations on an embedded FPGA. Its fractional-convolution engine uses a threshold-gated update phase, while the input layer uses binary convolution.

4 FRACBNN ACCELERATOR DESIGN

  • The accelerator supports fractional and binary convolutions, pooling, classification, normalization, activation, residual connections, and concatenation on an embedded FPGA.The design is intended to support the operations used by FracBNN.Source: §4 · p. 6

4.1 3×3 and 1×1 Fractional Convolution Engine

  • Fractional convolution first computes an MSB binary convolution, then conditionally computes an LSB output when the MSB output exceeds its channel threshold.The conditional update allows sparsity between 0% and 100%, spanning behavior between 2-bit and binary convolution.Source: §4.1 · pp. 6–7

4.2 3×3 Binary Convolution Engine

  • The input layer reuses the 3×3 convolution engine for binary convolution and executes only its base phase on thermometer-encoded inputs.For ImageNet, the 96 input channels are processed in three iterations of 32 channels.Source: §4.2 · p. 7

4.3 Average Pooling, Classifier, and Other Operations

  • Average pooling uses a pipelined adder tree, while the matrix-multiplication unit serves as the final linear classifier.BatchNorm, BPReLU, and channel concatenation are fused with adjacent operations; classifier work can alternatively run on the CPU in parallel with the accelerator.Source: §4.3 · p. 8

5 EVALUATION

FracBNN is evaluated on CIFAR-10 and ImageNet, with its ImageNet accelerator implemented on an embedded FPGA. The reported results include real-time ImageNet inference and latency trade-offs for fractional convolution.

5 EVALUATION

  • FracNN is evaluated for classification on CIFAR-10 and ILSVRC12 ImageNet, with input images augmented using random horizontal flips and random crops.For FracBNN, both CIFAR-10 and ImageNet accelerator designs run at 250MHz on a Xilinx Ultra96 v2 FPGA board.Source: §5.1 · p. 8, §5.2 · p. 8

5.1 Modeling Training Setup

  • ImageNet training adds a third stage that initializes weights from stage two and applies fractional convolution, excluding the first and last layers.The first two stages binarize activations, while the third uses the fractional layer; ImageNet training uses KL divergence against a pretrained ResNet-50 teacher.Source: §5.1 · p. 8

5.2 FPGA Implementation

  • 48.1 FPS at 16 ms hardware logic time, using 72% of LUTs and 93% of BRAM, accompanies the ImageNet accelerator’s reported best accuracy among comparable FPGA-implemented models.The design uses DDR for intermediate feature maps and double buffers to overlap some communication and computation.Source: §5.2 · p. 8, Table 6 · p. 9
  • 2,806.9 FPS at 72.9% LUT utilization is reported for CIFAR-10, with a 1.4× higher frame rate than ReBNet and 2.1% higher accuracy.The entire network is unrolled on the FPGA, with only input images and prediction results transmitted.Source: Table 7 · p. 9, §5.2 · p. 9

5.3 Ablation Study

  • The binary input layer runs significantly faster than a conventional fixed-point input layer with trivial additional resource consumption.It reuses the fractional-convolution engine, with resource consumption attributed to loading the image and weights.Source: §5.3 · p. 9, Table 9 · p. 9
  • Fractional convolution has slightly worse latency than 1-bit convolution but over 3× better latency than conventional 2-bit convolution.Matching 1-bit concurrency requires extra resources for the fractional model; 2-bit convolution’s resource demand forces lower concurrency.Source: §5.3 · p. 9

6 RELATED WORK

Prior work improves BNN accuracy through architectural and training changes, while FPGA research targets efficient low-precision inference. FracBNN’s related-work distinction is fractional activations and binary computation across all convolutional layers.

Binary Neural Networks

  • Prior BNN work explored architecture and training changes, PReLU-based activations, ensembles, and FPGA-tailored LUTNet; FracBNN instead updates selected features to two bits using precision gating.The precision-gating approach is described as dynamic and threshold-based.Source: §6 · p. 10

Quantization of Input Layer

  • Earlier input-layer approaches include stochastic binarization, DBID variants, and grouped stems, but the cited methods either expand channels, lose bit weights, or retain floating-point input convolution.For VGG-8 on CIFAR-10, the two DBID approaches are reported to degrade accuracy by at least 4.6%; FracBNN uses thermometer encoding and permits arbitrary dimensions from 1 to 255.Source: §6 · p. 10

FPGA-Based CNN Accelerators

  • Low-precision FPGA accelerators and co-designed networks address computation and bandwidth, but many prior works target AlexNet or VGG rather than newer ResNet and MobileNet models.FracBNN’s stated distinction is that all its convolutional layers use pure BMACs.Source: §6 · p. 10

FPGA-Based BNN Accelerators

  • FPGA BNN work includes BitSel, FINN, ReBNet, FBNA, and ImageNet AlexNet implementations; FracBNN uses fractional activations to target higher accuracy.The cited ImageNet binarized AlexNet implementation reports 42.9% top-1 accuracy, 13% below its full-precision model.Source: §6 · p. 10

7 CONCLUSIONS

FracBNN combines fractional activations with thermometer-encoded binary inputs while retaining BMACs for all convolutions. Its reported ImageNet accuracy is comparable to MobileNetV2, and it supports real-time classification on an embedded FPGA.

7 CONCLUSIONS

  • FracBNN uses dual-precision activations, thermometer-encodes the input layer, and computes all convolutions as BMACs.The accelerator supports fractional convolution and is implemented on an embedded Xilinx Ultra96 v2 FPGA.Source: §7 · p. 10

Related papers

Loading 2012.12206v1…