PaperlayerView exact revision on arXiv

Source-linked AI summary

SynQ: Accurate Zero-shot Quantization by Synthesis-aware Fine-tuning

Minjun Kim, Jongjin Kim, U Kang

arXiv:2603.18423v1cs.CV

TL;DR

Zero-shot quantization seeks accurate low-bit deployment without real data, but synthetic-data fine-tuning faces noise, off-target patterns, and erroneous hard labels. SYNQ filters generated samples, aligns class activation maps, and uses soft labels for difficult samples; it consistently outperforms existing ZSQ methods, with improvements up to 1.74%p.

  • Problem

    Zero-shot quantization must preserve pre-trained-model accuracy without real data, while existing synthetic-data methods face noise, off-target predictions, and erroneous hard labels.

  • Method

    SYNQ applies a low-pass filter, aligns the quantized model's class activation map with the pre-trained model, and uses soft labels for difficult samples.

  • Results

    SYNQ consistently outperforms existing ZSQ methods across diverse models, quantization bits, and datasets, improving classification accuracy by up to 1.74%p.

  • Takeaways & Limitations

    SYNQ is compatible with ZSQ methods that fine-tune on synthetic datasets and adapts across model types, quantization bits, and datasets.

Abstract

from arXiv · show

How can we accurately quantize a pre-trained model without any data? Quantization algorithms are widely used for deploying neural networks on resource-constrained edge devices. Zero-shot Quantization (ZSQ) addresses the crucial and practical scenario where training data are inaccessible for privacy or security reasons. However, three significant challenges hinder the performance of existing ZSQ methods: 1) noise in the synthetic dataset, 2) predictions based on off-target patterns, and the 3) misguidance by erroneous hard labels. In this paper, we propose SynQ (Synthesis-aware Fine-tuning for Zero-shot Quantization), a carefully designed ZSQ framework to overcome the limitations of existing methods. SynQ minimizes the noise from the generated samples by exploiting a low-pass filter. Then, SynQ trains the quantized model to improve accuracy by aligning its class activation map with the pre-trained model. Furthermore, SynQ mitigates misguidance from the pre-trained model's error by leveraging only soft labels for difficult samples. Extensive experiments show that SynQ provides the state-of-the-art accuracy, over existing ZSQ methods.

1 INTRODUCTION

Zero-shot quantization enables low-bit deployment when real training data are unavailable, but synthetic-data fine-tuning suffers from noise, off-target predictions, and erroneous hard labels. SYNQ addresses these limitations with filtering, activation-map alignment, and soft-label training, improving accuracy by up to 1.74%p.

  • Zero-shot quantization targets accurate low-bit models without using real data, supporting deployment on resource-constrained edge devices.
  • Existing synthetic-data methods leverage pre-trained-model statistics, embeddings, or texture distributions to generate samples resembling real data.
  • Synthetic datasets contain high-frequency noise unlike real images, while quantized models may rely on incorrect image patterns for predictions.
  • Erroneous hard labels for difficult synthetic samples can misguide fine-tuning and harm the quantized model.
  • SYNQ applies a low-pass filter, aligns class activation maps, and uses soft labels for difficult samples to address the three limitations.
  • 1.74%p: SYNQ improves image classification accuracy over existing methods and integrates across models, quantization bits, and datasets.

2 PRELIMINARIES AND PROBLEM DEFINITION

ZSQ generates a synthetic dataset from a pre-trained model, then fine-tunes a quantized model on those samples. The framework formalizes both the data-free accuracy objective and image difficulty used to identify unreliable labels.

  • The standard pipeline first generates synthetic samples resembling the original dataset, then fine-tunes the quantized model with them.
  • Noise optimization initializes synthetic images and labels randomly, then updates them using Batch Normalization Statistics and Inception losses.
  • The quantized model is initialized with Rounding-To-Nearest and fine-tuned using hard-label cross-entropy plus knowledge-distillation KL divergence.
  • 2.2 DIFFICULTY OF AN IMAGE: Image difficulty is defined as one minus the pre-trained model's predicted probability for the correct label.
  • 2.2 DIFFICULTY OF AN IMAGE: For difficulty above 0.5, the pre-trained model's error rate increases, indicating incorrect or uncertain predictions.
  • ZSQ optimizes a B-bit quantized model to maximize accuracy without real data, given a pre-trained model.

3 OBSERVATION

The paper identifies three problems in synthetic-data ZSQ: frequency-domain noise, reliance on incorrect image regions, and erroneous labels for difficult samples. These discrepancies hinder recovery of classification performance during quantized-model fine-tuning.

  • Noise in the synthetic dataset: Synthetic samples contain grainy noise and an evenly distributed frequency spectrum, unlike real images whose magnitude concentrates at low frequencies.
  • Noise in the synthetic dataset: This frequency-domain discrepancy challenges quantized models to restore classification performance during fine-tuning.
  • Predictions based on off-target patterns: Existing synthetic-data fine-tuning can make quantized models rely on incorrect image regions, unlike the pre-trained model's critical regions.
  • Misguidance by erroneous hard labels: As image difficulty increases, pre-trained ResNet error rates grow across CIFAR-10, CIFAR-100, and ImageNet, and samples above 0.5 are often mislabeled.
  • Misguidance by erroneous hard labels: Erroneous hard labels for difficult samples damage quantization performance by producing misguided fine-tuning outcomes.

4 PROPOSED METHOD

SYNQ addresses three ZSQ fine-tuning challenges—synthetic-data noise, off-target predictions, and erroneous hard-label guidance—through filtering, CAM alignment, and difficulty-aware soft-label training.

  • Overview: SYNQ targets noisy synthetic data, off-target prediction patterns, and misguidance from erroneous hard labels.These challenges can reduce quantized-model performance during synthetic-data fine-tuning.
  • Low-pass filter: A Gaussian low-pass filter removes high-frequency noise from generated samples before fine-tuning.Filtered samples have amplitude distributions that align more closely with real images.
  • Alignment of class activation map: SYNQ aligns the quantized model’s class activation map with the pre-trained model to improve localization of relevant image regions.The CAM loss compares activation maps, and the reported visualization shows SYNQ targeting correct regions like the pre-trained model.
  • Soft labels for difficult samples: For difficult samples, SYNQ omits cross-entropy with hard labels and uses only soft-label KL divergence.Easy samples use both hard-label cross-entropy and soft-label KL divergence, while difficult samples use only the latter.
  • Combined training objective: SYNQ combines filtered samples, CAM alignment, and difficulty-aware loss selection in its fine-tuning objective.The threshold τ determines whether cross-entropy is applied, while λCE and λCAM balance loss terms.
  • Efficiency: SYNQ scales linearly with the number of training samples N and model layers L and generates 5,120 samples.The paper contrasts this sample count with generator-based methods producing over 1 million samples.

5 EXPERIMENTS

Experiments evaluate SYNQ across CNNs, ViTs, CAM techniques, ablations, and hyperparameters. SYNQ consistently improves zero-shot quantization accuracy across diverse models, datasets, and bit-widths.

  • Accuracy in CNN Quantization (Q1): SYNQ achieves higher accuracy across 3-bit and 4-bit quantization on CIFAR-10, CIFAR-100, and ImageNet, with gains up to 1.74%p over TexQ and PLF.The largest reported gain is for ResNet-18 on ImageNet.
  • Accuracy in ViT Quantization (Q2): SYNQ improves ViT quantization across four ImageNet-pretrained models, increasing average precision by up to 0.58%p when applied to PSAQ-ViT.The evaluated models are DeiT-Tiny, DeiT-Small, Swin-Tiny, and Swin-Small.
  • Analysis on Class Activation Map Techniques (Q3): Grad-CAM outperforms CAM and Grad-CAM++ for 3-bit ResNet-18 quantization, motivating its use for SYNQ’s CAM alignment loss.The paper attributes this to Grad-CAM’s focus on a single object and broader applicability than CAM.
  • Ablation Study (Q4): All three SYNQ components improve accuracy, with the low-pass filter producing the strongest ablation impact of 5.80%p.The components are low-pass filtering, CAM alignment, and soft labels for difficult samples.
  • Hyperparameter Analysis (Q5): Accuracy remains robust across ranges of λCE and λCAM, while τ=0.5 provides the best trade-off between sample usage and ambiguity.Accuracy rises with τ up to 0.5 and declines above 0.5; an appropriate D0 balance is also necessary.

6 RELATED WORK

Network quantization reduces model resource demands by using lower-precision representations, while zero-shot quantization performs this process without access to training data.

  • Network Quantization: Network quantization converts weights, activations, or both from full precision to lower-precision formats, reducing computational complexity and memory footprint.It can also speed inference and reduce power consumption for edge deployment.
  • Zero-shot Quantization: Zero-shot quantization performs quantization without accessing the training data of the full-precision model.Earlier methods calibrated model parameters using model properties alone, but incurred significant performance drops.

7 CONCLUSION

SYNQ addresses three limitations of synthetic-data fine-tuning in zero-shot quantization and reports consistent improvements across models, bit-widths, and datasets. Its procedure generates synthetic data, filters samples, and fine-tunes the quantized model with CAM alignment and difficulty-aware label usage.

  • Conclusion: SYNQ targets noise in synthetic data, off-target predictions, and misguidance from erroneous hard labels.These are identified as the three major limitations of existing synthetic-data fine-tuning methods.
  • Conclusion: SYNQ applies a low-pass filter, aligns class activation maps, and uses soft labels for difficult samples during quantized-model fine-tuning.The method is compatible with synthetic-dataset generation techniques and uses these components to address the identified limitations.
  • Conclusion: SYNQ consistently outperforms existing zero-shot quantization methods across diverse models, quantization bit-widths, and datasets.The paper identifies object detection and diffusion models as future extension settings.
  • Algorithm: Synthetic samples are initialized from Gaussian noise, assigned labels, and optimized with inception and batch-normalization-statistics losses before fine-tuning.The quantized model is initialized with round-to-nearest quantization, and the samples are filtered before training.
  • Algorithm: The fine-tuning loop combines KL divergence, CAM alignment, and conditional cross-entropy based on the sample difficulty threshold.The quantized parameters are updated to minimize the resulting total loss.
  • Complexity Analysis: SYNQ’s total runtime is dominated by quantized-model fine-tuning, with synthetic-data generation and Fourier filtering adding separate computational costs.The proof analyzes dataset generation as O(NTθ), filtering as O(NZ log Z), and fine-tuning as O(N(Tθ + L · Tθ)).

C.2 RUNTIME ANALYSIS

SYNQ adds only modest computational overhead to fine-tuning while improving the adopted zero-shot quantization methods.

  • 17.81% of total fine-tuning time is the average overhead introduced by SYNQ.The comparison covers IntraQ, HAST, and TexQ under noise-optimization settings.
  • SYNQ improves adopted models with minimal sacrifice of quantization time.

C.3 PREVALENT NOISE IN THE SYNTHETIC DATASET

Synthetic datasets produced by existing ZSQ methods contain more high-frequency components than real image datasets, indicating greater noise. Applying a low-pass filter mitigates this distribution discrepancy.

  • Synthetic datasets contain more high-frequency components than corresponding real image datasets across methods and datasets.The analysis compares CIFAR-10, CIFAR-100, and ImageNet with synthetic data from IntraQ, HAST, and TexQ.
  • A low-pass filter with D0 = 50 mitigates the amplitude-distribution discrepancy by removing high-frequency noise.

C.4 FURTHER ANALYSIS ON CAM PATTERN DISCREPANCY

Existing ZSQ methods exhibit systematic discrepancies between the pre-trained model’s and quantized model’s saliency maps. SYNQ addresses this issue through CAM alignment, which outperforms feature alignment in the reported ablation.

  • All three baseline methods show notable CAM discrepancies between pre-trained and quantized models, confirming that off-target prediction patterns are general across the evaluated ZSQ settings.Using real datasets reduces the saliency-map distance by more than 20% compared with synthetic datasets.
  • CAM alignment focuses on prediction-related regions, whereas feature alignment matches broader activation responses.
  • CAM alignment shows superior 3-bit ZSQ accuracy to feature alignment both alone and combined with SYNQ’s other ideas.
  • CAM and feature alignment have the same time complexity, with negligible measured training-time difference.ResNet-18 training takes 113.40 ± 2.28 seconds per epoch with CAM alignment and 113.27 ± 2.34 seconds with feature alignment.
  • SYNQ consistently improves ZSQ accuracy across generator-based and noise-optimization baselines, with gains up to 31.17%p.

C.7 ANALYSIS ON ZERO-SHOT POST-TRAINING QUANTIZATION SETTING

SYNQ is evaluated beyond its primary QAT setting in a zero-shot PTQ comparison with Genie and across robustness and hyperparameter analyses. The supplied passages report consistent improvements with Genie and robustness to several noise conditions, while also examining dataset size and filtering choices.

  • SYNQ optimizes quantized-model parameters in QAT, whereas Genie optimizes PTQ scale factors and zero-points.
  • Uniform quantization linearly scales weights into a lower-bit integer range and rounds them to the nearest integer.The clipping range determines the scaling factor and zero-point; min-max quantization uses the minimum and maximum weight values.
  • Min-max quantization is vulnerable to outliers because they expand the clipping range and reduce precision for most values.
  • SYNQ consistently improves Genie-based zero-shot quantization performance across various quantization bit-widths.The comparison uses a ResNet-18 model on ImageNet.
  • The low-pass filter is evaluated under Gaussian, speckle, Salt-and-Pepper, and uniform noise, while additional analyses vary synthetic-dataset size, filtered samples, and the difficulty threshold.

D DETAILS ON THE EXPERIMENTAL SETUP

The experiments evaluate SYNQ across benchmark image-classification datasets, compare it with established ZSQ competitors, and specify synthetic-data generation and fine-tuning procedures. Hyperparameter searches and repeated runs support the reported results.

  • Datasets: SYNQ is evaluated on CIFAR-10, CIFAR-100, and ImageNet, using real datasets only for evaluation.
  • Competitors: The competitor set includes reconstruction, latent-embedding, intra-class, adaptive-sample, difficult-sample, texture-preserving, and pseudo-labeling methods.Additional comparisons cover PSAQ-ViT for vision transformers and Genie for post-training quantization experiments.
  • Baseline: The baseline combines calibration-center synthesis, difficult-sample generation, and sample-difficulty promotion before replacing existing fine-tuning with SYNQ.It combines the synthetic-data production components of TexQ and HAST to isolate synthesis-aware fine-tuning.
  • Experimental Protocol: Each experiment uses grid-searched hyperparameters, five iterations, and reports the mean and standard deviation of the results.Table 10 lists the searched SYNQ hyperparameter ranges for ImageNet.
  • Training Details: The quantized model is fine-tuned with SGD, momentum 0.9, weight decay 1e-4, batch size 256 for CIFAR-10/100, and 16 for ImageNet.The initial learning rate is searched over {1e-4, 1e-5, 1e-6} and training lasts nep = 100 epochs with decay by 0.1.
Loading 2603.18423v1…