PaperlayerView exact revision on arXiv

Source-linked AI summary

Generative Low-bitwidth Data Free Quantization

Shoukai Xu, Haokun Li, Bohan Zhuang, Jing Liu, Jiezhang Cao, Chuangrun Liang, Mingkui Tan

arXiv:2003.03603v3cs.CV

TL;DR

Data-free quantization is needed when training data are unavailable, but constructing meaningful calibration data from a full-precision model is difficult. GDFQ generates fake data by matching classification-boundary and distribution knowledge, enabling effective 4-bit quantization with higher accuracy than ZeroQ.

  • Problem

    Existing quantization methods require training data, while constructing meaningful data from a full-precision model remains challenging when privacy or confidentiality prevents access.

  • Method

    GDFQ uses a label-conditioned knowledge-matching generator to produce fake data from classification-boundary knowledge and batch-normalization statistics in a fixed full-precision model.

  • Results

    With 4-bit weight and activation quantization, GDFQ achieves much higher accuracy than ZeroQ on CIFAR-10 and smaller accuracy degradation on CIFAR-100.

  • Takeaways & Limitations

    GDFQ demonstrates effective completely data-free low-bitwidth quantization across image-classification datasets using generated data that retain category and distribution information.

Abstract

from arXiv · show

Neural network quantization is an effective way to compress deep models and improve their execution latency and energy efficiency, so that they can be deployed on mobile or embedded devices. Existing quantization methods require original data for calibration or fine-tuning to get better performance. However, in many real-world scenarios, the data may not be available due to confidential or private issues, thereby making existing quantization methods not applicable. Moreover, due to the absence of original data, the recently developed generative adversarial networks (GANs) cannot be applied to generate data. Although the full-precision model may contain rich data information, such information alone is hard to exploit for recovering the original data or generating new meaningful data. In this paper, we investigate a simple-yet-effective method called Generative Low-bitwidth Data Free Quantization (GDFQ) to remove the data dependence burden. Specifically, we propose a knowledge matching generator to produce meaningful fake data by exploiting classification boundary knowledge and distribution information in the pre-trained model. With the help of generated data, we can quantize a model by learning knowledge from the pre-trained model. Extensive experiments on three data sets demonstrate the effectiveness of our method. More critically, our method achieves much higher accuracy on 4-bit quantization than the existing data free quantization method. Code is available at https://github.com/xushoukai/GDFQ.

1 Introduction

Existing quantization methods depend on real training data, while random inputs and prior data-free methods fail to capture meaningful data distributions. GDFQ addresses this by generating data from classification-boundary knowledge and batch-normalization statistics in a pre-trained model, enabling 4-bit data-free quantization.

  • Motivation: Existing quantization methods generally require training data for calibration or fine-tuning, limiting deployment when original data are unavailable.Quantization reduces model size by converting floating-point values to low precision and can improve execution latency and energy efficiency.
  • Motivation: Random inputs lack semantic information and lie far from the real data distribution, causing severe performance degradation during quantization.The introduction also identifies generative adversarial networks as an alternative, but the supplied passage does not provide further details.
  • Motivation: Existing data-free methods overlook classification-boundary and distribution information, while ZeroQ relies on a single sample and constructs a distribution far from real data.The paper frames constructing meaningful data from a pre-trained model as an open question.
  • Method: GDFQ performs completely data-free quantization by learning a generator from classification-boundary knowledge and batch normalization statistics in a pre-trained full-precision model.Its knowledge matching generator is designed to produce meaningful data that retain classification-boundary knowledge and data distribution.
  • Contributions: 4-bitwidth quantization without any real data is presented as GDFQ’s main low-bitwidth contribution, with experiments reporting superior performance over existing data-free quantization methods.The experiments cover image classification data sets.

2 Related Work

Related work spans low-precision model quantization and data-free model compression. Prior data-free quantization methods address the absence of original data through weight-range equalization, quantization-error correction, and other model-derived strategies.

  • Model quantization: Model quantization compresses weights, activations, and gradients into low precision, replacing expensive multiplications with additions or bit-wise operations.The literature broadly distinguishes binary neural networks from fixed-point quantization according to the accuracy–complexity trade-off.
  • Data-free model compression: Data-free compression includes knowledge distillation, low-rank approximation, and model quantization methods that operate without original training data.Prior work has used generative adversarial networks for data-free distillation and reconstructed datasets from trained-model activation statistics.
  • Data-free model compression: DFQ improves data-free quantization by equalizing per-channel weight ranges and correcting biased quantization error.These methods respond to the practical difficulty of obtaining original data, which conventional quantization methods generally require for stronger performance.

3 Problem Definition

Data-free quantization seeks to construct fake labeled samples from a full-precision model when original data are unavailable, enabling calibration or fine-tuning without real data. The central challenge is generating meaningful data from limited latent information in the pre-trained model, whose quality strongly affects quantization performance.

  • Data-free quantization problem: Data-free quantization constructs fake samples from a full-precision model when original data are unavailable for calibration or fine-tuning.Without data, conventional quantization can lose efficacy or fail, producing an inferior quantized model.
  • Data-free quantization problem: Generated fake samples are paired with labels and optimized using losses such as cross-entropy or mean squared error.The generated sample is denoted ˆx, its corresponding label y, and the loss is denoted ℓ(·, ·).
  • Challenges of constructing data: Constructing meaningful data is difficult because latent information in a pre-trained full-precision model is hard to exploit for recovering original data.Quantization performance highly depends on the quality of the constructed data.
  • Challenges of constructing data: A generator maps random vectors drawn from a prior distribution, such as Gaussian or uniform noise, to fake data, but the exploitable model knowledge and generator-learning strategy remain unresolved.The formulation uses a random vector z sampled from p(z).

4 Generative Low-bitwidth Data Free Quantization

GDFQ removes the need for original calibration data by generating label-conditioned fake data from classification-boundary and distribution information stored in a pre-trained model. It then alternately trains the generator and quantized model using classification, batch-normalization-statistics, and distillation knowledge.

  • Classification boundary information matching: GDFQ generates fake data by conditioning Gaussian noise on labels and training the generator so the pre-trained model assigns each generated sample its conditioning label.The number of categories can be obtained from the pre-trained model’s last layer.
  • Distribution information matching: The generator matches training-data distribution information by aligning generated batch-normalization means and variances with the stored statistics of the pre-trained model.The loss uses the mean and variance encoded at each batch-normalization layer.
  • Optimization problem: The quantized model is fine-tuned on generated data with classification and knowledge-distillation losses so its outputs approximate those of the full-precision model.The generated data convert data-free quantization into a supervised quantization problem, while distillation addresses the limitations of fake-data classification loss.
  • Quantization: Quantization maps 32-bit weights and activations to low-precision values, including 8-bit fixed-point integers, using symmetric k-bit quantization.The method applies the same simple quantization approach to weights and activations.
  • Alternating training: The algorithm alternately updates the generator and quantized model, using generator warm-up and continued generator updates to increase data diversity and stabilize fine-tuning.The pre-trained full-precision model remains fixed throughout training.

5 Experiments

Experiments on CIFAR-10/100 and ImageNet evaluate GDFQ under 4-bit weight-and-activation quantization, showing strong accuracy and benefits from its loss components, fixed batch normalization, alternating training, and extended generator optimization. Comparisons further establish advantages over ZeroQ and other post-training quantization methods.

  • Experimental setup: Experiments use CIFAR-10/100 and ImageNet with 4-bit weights and activations across ResNet-20, ResNet-18, BN-VGG-16, and Inception v3.All layers, including the first and last, are quantized, with per-layer activation clipping.
  • Main comparisons: GDFQ achieves much higher 4-bit accuracy than ZeroQ on CIFAR-10 and suffers a smaller accuracy degradation than ZeroQ as CIFAR-100 categories increase.The authors attribute this to GDFQ extracting more prior knowledge from the full-precision model.
  • Ablation studies: Removing generator guidance causes large performance degradation, while adding cross-entropy guidance produces higher-confidence fake data and a better quantized model.The ablation studies evaluate generator losses on CIFAR-100 with ResNet-20.
  • Ablation studies: Fixing batch-normalization statistics narrows the statistics gap and yields much higher top-1 accuracy than standard batch normalization during fine-tuning.The fixed-BNS ablation is conducted with ResNet-20 on CIFAR-10/100.
  • Training strategies: Alternating generator and quantized-model training performs significantly better than separate training, and higher generator stopping thresholds improve fine-tuning performance.The best stopping-condition performance occurs when generator optimization is not stopped.

6 Conclusion

The paper proposes Generative Low-bitwidth Data Free Quantization to eliminate quantization methods’ data dependence by generating fake data from knowledge in a pretrained full-precision model and using it to fine-tune the quantized model.

  • 6 Conclusion: GDFQ constructs a knowledge matching generator that produces fake data for quantization fine-tuning by learning classification boundary knowledge and distribution information from the pretrained full-precision model.The full-precision model is then quantized, and the quantized model is fine-tuned using the generated fake data.

A More Experimental Results · A.1 Experiments on More Networks

Additional ImageNet experiments evaluate GDFQ on ResNet-50, MobileNetV2, and ShuffleNet at 6-bit and 4-bit precision. GDFQ outperforms ZeroQ across precisions, with larger gains at the lower W4A4 bit-width.

  • A.1 Experiments on More Networks: GDFQ achieves much better accuracy than ZeroQ across the evaluated precisions and networks.The comparison covers the models and bit-widths reported in Table 8.
  • A.1 Experiments on More Networks: GDFQ’s performance gain over ZeroQ is larger at lower bit-width, exemplified by W4A4 across all models.The passage specifically identifies W4A4 as showing larger gains for every evaluated model.
  • A.1 Experiments on More Networks: These results demonstrate the effectiveness of GDFQ on additional network architectures.The conclusion follows from the reported accuracy comparisons with ZeroQ.
  • A.1 Experiments on More Networks: Table 8 evaluates ResNet-50, MobileNetV2, and ShuffleNet on ImageNet after quantization to 6-bit and 4-bit precision using Top-1 accuracy.The table reports experiments on more networks.

A.2 Comparisons with DFQ

The method is compared with DFQ on ImageNet using MobileNetV2 and ResNet-18 quantized to 8-bit, 6-bit, and 4-bit. It achieves much higher performance than DFQ, especially at low bit-width.

  • Comparisons with DFQ: The method achieves much higher performance than DFQ, especially in low bit-width.The comparison uses an implementation of the DFQ algorithm.
  • Comparisons with DFQ: MobileNetV2 and ResNet-18 are quantized to 8-bit, 6-bit, and 4-bit, with Top-1 accuracy reported on ImageNet.
Loading 2003.03603v3…