SYQ: Learning Symmetric Quantization For Efficient Deep Neural Networks

• DNN growth increases memory and computational requirements, making deployment on embedded systems more difficult.Source: §1 · p. 1
• Low-precision networks can replace most multiply-accumulate operations with bitwise operations, but large precision reductions lose information and can degrade accuracy.Source: §1 · p. 1
• Quantized weights create a gradient mismatch between forward and backward functions when their gradient estimates are inaccurate, impinging on learning.Source: §1 · p. 1
• The method learns symmetric weight codebooks using gradient-based optimization, with pixel- or row-wise scaling for convolutional layers and layer-wise scaling for fully connected layers.Source: §1 · p. 2
• The approach improves convolutional weights’ ability to learn low-precision representations, addressing layers described as common and typically least redundant in modern networks.Source: §1 · p. 2
• It reduces the computational complexity of fine-grained low-precision scaling while imposing minimal hardware costs relative to layer-wise scaling.Source: §1 · p. 2
• SYQ improves accuracy on AlexNet, ResNet, and VGG with 1–2-bit weights and 2–8-bit activations.Source: §1 · p. 2

• Scaling approaches recover weight dynamic range using layer expectations, optimized thresholds, or gradient-based scaling coefficients [18] [33].Source: §2 · p. 2
• Other work explored incremental weight subgrouping, multiple binarizations, logarithmic representations, activation quantization, and fine-grained clustering or grouping [31] [21] [32] [20].Source: §2 · p. 2
• Sparse representations and hardware implementations, including FINN demonstrations of binarized networks on FPGAs, were also reported [7] [29] [10] [8] [28].Source: §2 · p. 2

3.1. Motivation

• Low-precision DNNs simplify arithmetic and reduce resource demands, but deeper and wider networks remain challenging to deploy on constrained hardware.Each layer computes dot products between weights and inputs; the paper describes MACs being replaced by bit operations for binary or ternary weights.Source: §3.1 · p. 2
• For precisions up to ternary weights and 8-bit activations, FPGA implementations require no high-precision DSP multipliers; storage falls 8–32× as weight and activation precision decrease.The passage also reports that LUT requirements decrease with both precisions.Source: Figure 1 · p. 3, §3.1 · p. 3

3.2. Weight Quantization

• Weight discretization can cause vanishing gradients, so an STE supplies a nonzero surrogate derivative while full-precision weights are updated and later discarded for deployment.During training, quantized weights are used for inference and backpropagation.Source: §3.2 · p. 3

3.3. Scaling

• Scaling coefficients increase model capacity and compensate for information lost to binary or ternary quantization, while reducing gradient mismatch between forward and backward functions.The cited prior approach sets a layer’s coefficient to the mean absolute weight value; another STE incorporates full-precision weight information.Source: §3.3 · pp. 3–4
• Asymmetric scaling can make codebook indices irregular and increase computation; SYQ instead learns symmetric codebooks with structured matrices for efficient fine-grained scaling.The proposed symmetric quantizer learns diagonal scaling elements during training, with storage described as K or K^2.Source: §3.1 · p. 3, §3.3 · p. 4

4. SYQ Structural Representations

• SYQ partitions quantization into weight subgroups and represents scaling with diagonal matrices, which can be decomposed efficiently while storing only the scalar vector.The paper introduces diagonal and square diagonal matrix representations for these subgroups.Source: §4 · p. 4

4.1. Layers

• Convolutional weights are reused across an input feature map, so their quantization errors affect many dot products; finer scaling is used for these layers than for fully connected layers.Convolutional scaling can use K × K or K^2 × K^2 diagonal matrices, while fully connected layers use one coefficient.Source: §4.1 · p. 4

4.2. Subgroups

• Pixel-wise scaling groups spatially equivalent pixels along I × N, while row-wise scaling groups pixels along a row or column and layer-wise scaling is coarser.The subgroup granularities affect both accuracy and computation.Source: §4.2 · p. 4

5.1. Symmetric Quantizer

• Symmetric zero-centered codebooks nearly halve storage for binary/ternary weights, regularize indices, and support pixel- or row-wise scaling while retaining scalar matrix multiplication.Figure 2 depicts the subgrouping as training tensors and deployment matrix decompositions; gradient-based scales reduce weight-quantization gradient mismatch and enhance learning.Source: §5.1 · p. 5, Figure 2 · p. 5

5.2. Initialization

• Subgroup scales are initialized from the mean of corresponding full-precision weights to reduce initial mismatch, then optimized by backpropagation; FC layer-wise scaling uses the mean of all layer weights.Source: §5.2 · p. 5

5.3. Activations Quantization

• For activations, SYQ uniformly quantizes x∈[0,M] to k bits; M is adjusted as a hyperparameter to minimize mismatch, and uniform quantization reduces hardware complexity.Source: §5.3 · p. 5

6.1. Networks

• ImageNet experiments cover AlexNet, ResNet variants, and VGG-16, using single-crop Top-1 and Top-5 accuracy evaluation.The reported SYQ training results are described as achieving accuracy similar to floating point; full-precision weights are retained in the first and last layers, with other CONV layers using pixel-wise scaling and FC layers layer-wise scaling.Source: §6 · p. 6, §6.1 · p. 6

6.2. Changing Granularity Via Weight Subgroups

• Pixel-wise and row-wise scaling are marginally different, especially at higher precisions, and both are considerably more accurate than layer-wise scaling.Row-wise quantization typically uses less memory than pixel-wise quantization, with a small accuracy drop and potential parallelism gains.Source: Table 2 · p. 6, §6.2 · p. 6

6.3. Comparisons To Previous Work

• SYQ's ternary 2w-8act improves on state-of-the-art across AlexNet, ResNet-18, and ResNet-50; ResNet-50 2w-4act also improves on FGQ, while 1w-2act gains 2.7% Top-1 over HWGQ.The comparisons use Tables 3–5 for AlexNet, ResNet-18, and ResNet-50.Source: §6.3 · pp. 6–7, Table 3 · p. 7, Table 4 · p. 7

6.4. Varying Activation Bitwidth

• Lower activation precision does not severely alter training curves, but 2-bit activations make training error more volatile; accuracy remains close to full precision down to 4-bit activations.These observations concern binary AlexNet training and validation error and reported AlexNet and ResNet-50 classification accuracies.Source: Figure 3 · p. 7, §6.4 · p. 7

7.1. Computational and Memory Complexity

• For pixel-wise SYQ scaling, K^2 coefficients and P operations are required, versus K^2N/4 coefficients for FGQ; row-wise SYQ uses K coefficients and P operations.The passage states K = 3 for most CONV layers in modern networks and says SYQ improves computational or memory complexity against other fine-grained methods.Source: §7.1 · pp. 7–8

7.2. Architectural Design

• SYQ divides dot products into sub-dot products over symmetrically constrained weights, then applies each subgroup's scaling factor; binary or ternary multiplies can often be implemented as bit-flips.The described pixel-wise operation count retains Lv − 1 additions while lowering the precision of the first term.Source: §7.2 · p. 8
• The described per-pixel or row-wise hardware adds a ring counter for scaling coefficients while retaining one multiplier; BRAM and LUT usage are almost identical to the layer-wise baseline.For the target Xilinx ZU3, DSP usage increases with parallel output channels; row-wise scaling differs from pixel-wise LUT usage by less than 2%.Source: Table 7 · p. 8, §7.2 · p. 8

8. Conclusions

• The proposed symmetric quantization aims to maximize network learning while minimizing hardware complexity for low-precision weights and activations.Source: §8 · p. 8
• The method constrains solutions to low-precision representations and learns a diagonal scalar matrix using gradient-based optimization.Source: §8 · p. 8
• The approach reduces the computational requirements of fine-grained quantization and achieves state-of-the-art accuracies on modern benchmark networks.Source: §8 · p. 8