Mixed-Precision Neural Network Quantization via Learned Layer-wise Importance

1 Introduction

• Fixed-precision quantization forces layers with different sensitivities to share a bit-width; mixed precision lets insensitive layers use lower bit-widths, offering a flexible accuracy-efficiency trade-off but creating an n^(2L) search space (L layers, n options each).Search-based methods evaluate searched policies on the training set for multiple rounds, including 600 rounds in HAQ.Source: §1 · p. 2
• Prior sensitivity-based methods are described as biased when they estimate sensitivity from full-precision networks, or as limiting search space and requiring manual activation-bit assignment.The paper identifies these limitations for Hessian-based methods and MPQCO, respectively.Source: §1 · pp. 2–3
• Learned quantizer scale factors indicate layer contributions to performance and distinguish quantization-sensitive from insensitive layers.The indicators are learned end-to-end in quantization-aware training and capture quantization transformations such as rounding and clamping.Source: §1 · p. 3
• A joint scheme trains the many weight and activation importance indicators in one QAT process, parallelizing their otherwise separate training processes.The paper says this reduces indicator-training processes by M×.Source: §1 · p. 3
• A one-time ILP uses the learned indicators to assign bit-widths automatically, eliminating iterative search without limiting the bit-width search space.More-sensitive layers receive higher bit-widths than less-sensitive layers.Source: §1 · pp. 3–4
• The method reports about 330× MPQ policy-search speedup over AutoQ on ResNet50 while preventing a 1.7% top-1 accuracy drop.These figures are reported together for the ResNet50 comparison.Source: §1 · p. 4
• ResNet50’s accuracy gap between full-precision and quantized models narrows to 0.6%, while model size is reduced by 12.2×.The paper reports state-of-the-art results for its method.Source: §1 · p. 4

2.1 Neural Network Quantization

• Fixed-precision quantization uses the same bit-width for all or most layers, whereas mixed-precision methods seek layer-specific bit-widths for accuracy-efficiency balance.Prior quantization work also introduced learnable quantizers, activation upper bounds, and scale factors or quantization intervals.Source: Mixed-Precision Quantization · p. 4, Fixed-Precision Quantization · p. 4
• Search-based methods include DRL and NAS approaches that explore bit-width assignments, alongside differentiable quantization methods such as DiffQ.The passage describes these methods through their search formulations and examples.Source: Mixed-Precision Quantization · p. 4
• Criterion-based methods use Hessian eigenvalues or traces, or formulate bit-width selection as an MCKP, reducing search costs but with biased approximations or limited search space.The passage cites HAWQ, HAWQ-v2, and MPQCO as examples.Source: Mixed-Precision Quantization · p. 4

2.2 Indicator-Based Model Compression

• Learned or approximated indicators have been used for pruning and NAS, but quantization differs because it includes numerical precision transformation.Examples include second-derivative information and batch-normalization scale factors for pruning or channel selection.Source: §2.2 · p. 5

3.1 Quantization Preliminary

• The learnable quantizer scale factor s adjusts quantization mappings; each layer has separate weight and activation factors.A larger s makes more continuous values map to the same quantized value, and layers with larger learned scale factors are assigned more bits to preserve numerical diversity.Source: §3.1 · p. 5, §3.3 · p. 7

3.2 From Accuracy to Layer-wise Importance

• MPQ selects layer-wise weight and activation bit-widths to maximize validation accuracy under constraints, but its discrete search space makes iterative training-set evaluation expensive.Prior search can take more than 1,000 GPU-hours for AutoQ, as reported in the paper.Source: §3.2 · p. 6

3.3 Learned Layer-wise Importance Indicators

• MobileNetv1’s depth-wise convolutions showed greater accuracy degradation and larger scale factors than point-wise convolutions at the same bit-width, supporting scale factors as sensitivity indicators.The controlled experiment quantized individual layers at 2 or 4 bits on ImageNet.Source: Feasibility Verification · pp. 7–8, Figure 2 · p. 8

3.4 One-time Training for Importance Derivation

• Joint training obtains indicators for all layers and bit-width options in one training session instead of separate sessions for each indicator.Each update aggregates gradients from n uniform-bit-width passes and one randomly assigned per-layer bit-width pass.Source: §3.4 · p. 9

3.5 Mixed-Precision Quantization Search Through Layer-wise Importance

• 0.06 seconds: the ILP solver finds a ResNet18 mixed-precision policy without training data or iterative accuracy evaluations.The search uses learned indicators and can apply constraints such as BitOps or compression rate.Source: §3.5 · p. 10

4.1 Mixed-Precision Quantization Performance Effectiveness

• For ResNet18 at 3-bit (23.07G BitOps), a 70.5% full-precision initialization yields 69.7% quantized accuracy, surpassing existing methods, especially DNAS by 1.0%, although DNAS starts from 71.0% FP.Source: ResNet18 · pp. 11–12
• On ResNet50, LIMPQ outperforms PACT, LQ-Net, DeepComp, and HAQ with a 7.97 MB model, and has smaller accuracy degradation than HAWQ, HAWQv2, and MPQCO.The comparison includes BitOps and a 12.2× compression-rate constraint.Source: ResNet50 · p. 12, Table 3 · p. 12
• At 4-bit level, MobileNetv1 results show up to 4.39% accuracy improvement over other mixed-precision methods.The experiments report BitOps-constrained settings at 5.78G and 9.68G BitOps.Source: MobileNetv1 · p. 12, Table 4 · p. 13
• For MobileNetv1 weight-only quantization, the 1.79 MB model surpasses the 2.12M HMQ model in accuracy.The table reports weight-only quantization results.Source: MobileNetv1 · p. 12, Table 5 · p. 13

4.2 Mixed-Precision Quantization Policy Search Efficiency

• LIMPQ trains indicators once in 3.3 GPU-hours and solves the ResNet50 ILP in 0.35 seconds; compared with AutoQ, it reports about 330× speedup and higher accuracy.Indicator training takes about 50 minutes on four NVIDIA A100 GPUs using 50% of the training set; AutoQ is reported to require 1000 GPU-hours for one device.Source: Comparison with SOTAs on ResNet50 · p. 13, §4.2 · p. 13
• For one device, LIMPQ requires an additional 20-minute cold-start investment relative to HAWQv2 and reports a 1.1% top-1 accuracy improvement.The comparison notes that for sufficiently many devices, LIMPQ has almost the same time overhead as HAWQv2.Source: Comparison with SOTAs on ResNet50 · pp. 13–14

4.3 Ablation Study

• Reversing the routine bit-width assignment yields 6.59% lower top-1 accuracy under the same BitOps constraint.The reversed assignment gives fewer bits to layers with larger scale factors and more bits to those with smaller scale factors.Source: §4.3 · p. 14

• Quantizer scale factors serve as learned importance indicators for assigning each layer’s bit-width.Source: §5 · p. 14
• Experiments demonstrate an empirical association between the indicators and layers’ quantization sensitivity.Source: §5 · p. 14
• Experiments assess whether the indicators represent layers’ contributions at specific bit-widths and whether the resulting assignments are rational.Source: §5 · p. 14