PaperlayerView exact revision on arXiv

Source-linked AI summary

Diversifying Sample Generation for Accurate Data-Free Quantization

Xiangguo Zhang, Haotong Qin, Yifu Ding, Ruihao Gong, Qinghua Yan, Renshuai Tao, Yuhang Li, Fengwei Yu, Xianglong Liu

arXiv:2103.01049v3cs.CV

TL;DR

Post-training quantization still needs real calibration data, while data-free methods can generate synthetic data but may homogenize it at both distribution and sample levels. DSG relaxes BN-statistics alignment and varies layerwise enhancement across samples. Across architectures and quantization methods, it improves results and can outperform real-data calibration.

  • Problem

    Post-training quantization needs real calibration data that may be unavailable, while existing synthetic data can homogenize at distribution and sample levels.

    Source: §1 · pp. 1–2

  • Method

    DSG combines relaxed BN-statistics alignment with layerwise loss enhancement that differs across samples.

    Source: §1 · p. 2, §3.3 · p. 4, §3.4 · p. 4

  • Results

    DSG consistently improves results across network architectures and quantization methods, surpasses previous methods, and can outperform real-data calibration.

    Source: §1 · p. 2

  • Takeaways & Limitations

    DSG-generated synthetic data can also be applied to post-training quantization methods such as AdaRound.

    Source: §1 · p. 2

    The BN-statistics loss cannot determine activation clip values for quantized models in place of real data; otherwise, a significant performance drop might result.

    Source: §3.2 · p. 4

Abstract

from arXiv · show

Quantization has emerged as one of the most prevalent approaches to compress and accelerate neural networks. Recently, data-free quantization has been widely studied as a practical and promising solution. It synthesizes data for calibrating the quantized model according to the batch normalization (BN) statistics of FP32 ones and significantly relieves the heavy dependency on real training data in traditional quantization methods. Unfortunately, we find that in practice, the synthetic data identically constrained by BN statistics suffers serious homogenization at both distribution level and sample level and further causes a significant performance drop of the quantized model. We propose Diverse Sample Generation (DSG) scheme to mitigate the adverse effects caused by homogenization. Specifically, we slack the alignment of feature statistics in the BN layer to relax the constraint at the distribution level and design a layerwise enhancement to reinforce specific layers for different data samples. Our DSG scheme is versatile and even able to be applied to the state-of-the-art post-training quantization method like AdaRound. We evaluate the DSG scheme on the large-scale image classification task and consistently obtain significant improvements over various network architectures and quantization methods, especially when quantized to lower bits (e.g., up to 22% improvement on W4A4). Moreover, benefiting from the enhanced diversity, models calibrated by synthetic data perform close to those calibrated by real data and even outperform them on W4A4.

1. Introduction

Quantization can make neural networks more efficient, but post-training methods still rely on real calibration data, which may be inaccessible. The paper identifies two forms of homogenization in synthetic calibration data and proposes DSG to improve its diversity.

1. Introduction

  • Post-training quantization avoids retraining but still requires real data for calibration, which may be unavailable for privacy or security reasons.Quantization-aware training is described as time-consuming and computationally intensive.Source: §1 · p. 1
  • Existing synthetic data can homogenize at the distribution level by overfitting BN statistics and at the sample level because every sample uses the same objective.The resulting synthetic samples lack the diversity seen in real data.Source: §1 · p. 2
  • DSG combines Slack Distribution Alignment, which relaxes BN feature-statistics constraints, with Layerwise Sample Enhancement, which reinforces specific layers differently across samples.These techniques target distribution-level and sample-level homogenization, respectively.Source: §1 · p. 2
  • DSG consistently improves results across network architectures and quantization methods, surpasses previous methods, and can outperform real-data calibration.The scheme also applies to AdaRound.Source: §1 · p. 2

2. Related Work

Related work moves from data-driven quantization toward post-training and data-free approaches to reduce training cost or dependence on real data. Existing data-free methods use varied synthesis strategies, with some also relying on distillation.

Data-Driven Quantization

  • Data-driven quantization can recover accuracy after quantization, but retraining or fine-tuning is computationally expensive and original data may be inaccessible.The paper particularly notes private and secure settings.Source: §2 · p. 2

Post-training Quantization

  • Post-training quantization reduces computation and time by avoiding fine-tuning, but the surveyed methods still require limited data to recover performance.Methods include analytical clipping and bit allocation, minimum-error quantization, channel splitting, and AdaRound's data- and task-adaptive weight rounding.Source: §2 · p. 2

Data-Free Quantization

  • Data-free quantization needs neither training nor validation data, but surveyed approaches use differing strategies and some add time-consuming distillation or finetuning.The cited approaches include weight equalization, synthetic-data methods, and adversarial knowledge distillation.Source: §2 · pp. 2–3

3. Diverse Sample Generation

DSG addresses homogenization in synthetic data for data-free quantization by relaxing BN-statistics alignment and enhancing different layers for different samples. Its generated data is reported to improve quantization over existing generative methods and, in some post-training settings, to substitute for real calibration data.

3.1. Preliminaries

  • Typical data-free methods optimize synthetic inputs to match the FP32 model’s BN means and standard deviations, but this does not reproduce real-data behavior.The paper identifies both distribution-level overfitting to BN statistics and sample-level similarity across synthetic examples.Source: §3.1 · p. 3, §3.2 · pp. 3–4

3.3. Slack Distribution Alignment

  • SDA relaxes BN-statistics constraints using margins for feature means and standard deviations, allowing synthetic statistics to vary within a range.The paper estimates the relaxation constants from gaps between Gaussian-initialized synthetic data statistics and corresponding BN statistics.Source: §3.3 · p. 4

3.4. Layerwise Sample Enhancement

  • LSE assigns each synthetic sample a loss that reinforces a particular BN layer, with a batch size equal to the network’s number of BN layers.This varies samples’ attention across layers rather than applying the same objective to every sample.Source: §3.4 · p. 4

N · 1T (XLSEL) (5)

  • The LSE batch formulation uses an enhancement matrix to combine the BN-layer loss terms into sample-specific constraints optimized jointly.The loss vector contains terms for the BN layers, and the matrix is defined as I + 11^T.Source: §3.4 · p. 4, N · 1T (XLSEL) (5) · p. 4

N · 1T (XLSELSDA) (6)

  • The DSG synthesis procedure computes SDA losses from activations, forms the combined objective, then updates synthetic inputs by descending that objective.The resulting samples are intended to address homogenization at both levels.Source: N · 1T (XLSEL) (5) · p. 5, N · 1T (XLSELSDA) (6) · p. 5
  • DSG-generated data performs better in quantization than existing generative data-free methods and can replace real data in state-of-the-art post-training quantization when real data is unavailable.The scheme combines Slack Distribution Alignment (SDA) and Layerwise Sample Enhancement (LSE) to address homogenization at both distribution and sample levels.Source: N · 1T (XLSELSDA) (6) · p. 5

3. Diverse Sample Generation

  • DSG combines relaxed BN-statistics alignment with layer-specific loss reinforcement to address synthetic-data homogenization at distribution and sample levels.SDA relaxes distribution constraints, while LSE reinforces a specific layer for each sample.Source: §3.3 · p. 4, §3.4 · p. 4, N · 1T (XLSELSDA) (6) · p. 5

3.5. Analysis and Discussion

  • In ResNet-18, DSG statistics fluctuate more like real-data statistics and do not strictly fit BN statistics, unlike the more centralized ZeroQ statistics.The figure compares mean and standard deviation of one activation channel across 256 samples.Source: Figure 3 · p. 5, §3.5 · p. 5, N · 1T (XLSELSDA) (6) · p. 5

4. Experiments

ImageNet experiments evaluate DSG through component ablations and comparisons across architectures, bit-widths, quantization methods, and AdaRound. DSG generally outperforms prior methods, with its advantages especially pronounced at lower bit-widths.

4.1.1 Effect of SDA

  • Removing SDA lowers performance by 7.41% versus full DSG; accuracy rises as ϵ goes from 0 to 0.9 but drops at 1.At ϵ = 1, outliers can push the synthetic feature distribution beyond a reasonable range.Source: Table 1 · p. 6, §4.1.1 · p. 6, Figure 4 · p. 6

4.1.2 Effect of LSE

  • LSE adds up to 1.08% over ZeroQ; together with SDA, its gains are compatible and superimposed, reaching 8.49% over vanilla competitors.Source: Table 1 · p. 6, §4.1.2 · p. 6

4.1. Ablation Study

  • Ablations isolate DSG components on ResNet-18/ImageNet at W4A4, where each component's effect is most evident.Source: §4.1 · p. 6, Table 1 · p. 6

4.2. Comparison with SOTA methods

4.3. Evaluation with AdaRound

  • With AdaRound, DSG surpasses ZeroQ under all settings, especially at 3-bit weights, and retains high accuracy with labels and image prior.Source: Table 5 · p. 8, §4.3 · p. 8

5. Conclusion

The conclusion identifies distribution- and sample-level homogenization in prior data-free quantization data as harmful to quantized-model accuracy. DSG combines SDA and LSE to increase synthetic-data diversity, with experiments reporting leading accuracy and versatility, especially at ultra-low bit-widths.

  • Prior data-free synthetic data exhibits distribution- and sample-level homogenization that harms quantized-model accuracy.Source: §5 · p. 8
  • DSG combines Slack Distribution Alignment and Layerwise Sample Enhancement to address the two homogenization levels and increase generated-data diversity.Source: §5 · p. 8
  • Experiments across architectures and post-training quantization methods report leading accuracy and versatility, with up to 22% improvement at W4A4.Source: §5 · p. 8

Related papers

Loading 2103.01049v3…