Source-linked AI summary
SCULPT: Training Edge Vision Models for Post-Training Quantization Readiness
Bharadwaj Kavuri, Sourav Babu-PK, Varadhraj Ellapan, Pullarao Maddu, Prasad Deshpande
TL;DR
Low-bit PTQ is challenged by heavy-tailed activation outliers, which make calibration trade off resolution against clipping. SCULPT addresses this during ordinary FP32 fine-tuning with activation regularization and stable percentile clipping, achieving stronger matched-contract PTQ results than standard FP32 PTQ and the tested W8A8 QAT baseline.
Problem
Low-bit PTQ must balance preserving rare activation outliers against retaining resolution for the dense distribution.
Method
SCULPT shapes FP32 training through topology-aware regularization and stable percentile clipping, without simulated quantization or runtime outlier-repair transformations.
Results
99.3631% top-1 accuracy: SCULPT PTQ versus 90.5478% for standard FP32 PTQ and 98.2420% for the tested W8A8 QAT baseline under identical W8A8 conditions.
Takeaways & Limitations
SCULPT supports exporting learned activation bounds into a standard low-bit PTQ workflow, including the evaluated W8A8 setting.
Takeaways & Limitations
The reported evaluation is on EfficientNet-B0/ImageNette, and extending the analysis to more complex perception models remains an important next step.
Abstract
from arXiv · showhide
Edge vision models are difficult to deploy on resource-constrained hardware, making low-bit post-training quantization (PTQ) attractive. In practice, standard FP32 training often produces heavy-tailed activation distributions whose outliers destabilize activation quantization: preserving the full range wastes quantization bins on rare extremes, while aggressive clipping causes information loss. Existing solutions typically rely on quantization-aware training (QAT), which adds training complexity and bit-width coupling, or advanced PTQ procedures that repair the model after training. We present SCULPT (Statistical Clipping and Uniform Loss for Post-Training), a training-time method that improves PTQ readiness during ordinary FP32 fine-tuning. SCULPT combines a topology-aware activation regularizer that suppresses quantization-hostile skewness and kurtosis with a stable percentile-based clipping mechanism that learns deployment-ready activation bounds. Unlike QAT, SCULPT does not simulate quantization during optimization; unlike post hoc outlier-repair PTQ methods, it does not require runtime activation transformations. The learned clipping bounds can be exported directly into a standard PTQ workflow for low-bit deployment, including INT8 and lower-bit settings such as W4A8.
1 Introduction
Low-bit PTQ is vulnerable to activation outliers that make calibration brittle, while existing training- or post-training remedies add complexity. SCULPT instead shapes activation statistics during ordinary FP32 fine-tuning and exports deployment-ready clipping bounds for standard PTQ.
- Motivation: Heavy-tailed activations force a tradeoff between wasting quantization resolution on rare outliers and clipping those extremes.This tradeoff becomes especially damaging at low precision.
- Motivation: Rare extreme values can drive a layer’s calibration maximum far beyond its normal operating scale, making min/max calibration brittle.The representative hard-layer example identifies this mismatch as a major source of PTQ error.
- Existing approaches: QAT learns clipping or quantizer parameters during optimization but adds training complexity and can depend on a specific quantization setting.Post hoc PTQ methods instead apply calibration, reconstruction, rounding, or activation transformations after training.
- SCULPT: SCULPT trains the model in full precision to improve its later PTQ readiness rather than repairing quantization failure only after training.Its central hypothesis links PTQ brittleness to structured activation outliers produced during ordinary FP32 optimization.
- SCULPT: SCULPT combines topology-aware activation regularization with stable percentile clipping to improve PTQ readiness.The method does not simulate quantization during training and does not require runtime activation transformations.
- Evaluation: The evaluation uses a matched quantization contract and examines accuracy, efficiency, and activation outlier behavior.This defines the reported evaluation dimensions rather than a specific result.
2 Related Work
Prior work improves quantization robustness either during training through simulated quantization or after training through corrective PTQ procedures. These approaches motivate SCULPT’s distinct focus on FP32 training-time activation shaping without simulated quantization.
- Training-time methods: Quantization robustness has been studied through training methods such as PACT, TQT, LSQ, and Robust Quantization.These methods generally optimize under simulated quantization and are therefore QAT-style approaches.
- Post-training methods: Post-training methods improve low-bit accuracy through adaptive rounding, block-wise reconstruction, activation smoothing, or specialized activation quantization.Examples include AdaRound, BRECQ, SmoothQuant, and PTQ4ViT.
- Positioning: SCULPT occupies a different design point by improving PTQ readiness during ordinary full-precision fine-tuning.Its approach targets activation statistics before quantization rather than relying on simulated quantization or post hoc correction.
3 Method
SCULPT improves PTQ readiness during ordinary FP32 fine-tuning by jointly shaping activation distributions and learning stable deployment-time bounds. Its two components provide complementary soft distribution regularization and explicit clipping before standard PTQ export.
- Method: SCULPT combines topology-aware activation regularization with stable percentile-based clipping to improve PTQ readiness during ordinary FP32 fine-tuning.The regularizer shapes activation statistics, while the clipping layer learns deployment-ready bounds.
- StablePercentileClip: Adaptive-SPC wraps standard activations with percentile-based clamping whose bounds stabilize over training and are frozen for deployment.The bounds are estimated from observed post-activation statistics rather than fixed in advance.
- StablePercentileClip: Adaptive-SPC estimates percentile bounds on wrapped-activation outputs, preserving the original nonlinearity while adding learned post-activation limits.The wrapped activation may be SiLU, ReLU, ReLU6, or Hardswish.
- StablePercentileClip: Detached running statistics make the clipping bounds nondifferentiable through quantile computation, so backpropagation treats the layer as a hard post-activation clamp.The bounds are updated separately from gradient propagation.
- UADR: UADR suppresses excess skewness and heavy tails in selected learnable-layer outputs using topology-aware one-sided targets rather than enforcing a strictly uniform distribution.Topology-aware refers to an activation’s functional position within a computation block.
- Integration: UADR and StablePercentileClip are applied to complementary parts of the network and produce frozen layerwise activation limits for standard PTQ without simulated quantization or runtime outlier-repair transforms.UADR targets Conv2d and Linear outputs, while Adaptive-SPC follows common activation functions.
4 Experimental Setup
SCULPT is evaluated on EfficientNet-B0/ImageNette using FP32 fine-tuning, activation clipping and UADR, with PTQ and QAT compared under an identical supergroup QDQ contract.
- Model and training: The controlled testbed uses EfficientNet-B0 on ImageNette, with the baseline initialized from Torchvision ImageNet-1K weights, adapted to 10 classes, and trained in FP32 for 80 epochs.Training uses AdamW, label smoothing, RandAugment, AMP, batch size 128, and a cosine OneCycle schedule.
- SCULPT configuration: SCULPT fine-tunes the trained FP32 checkpoint with Adaptive-SPC after nonlinear activations and UADR on Conv2d and Linear outputs.Adaptive-SPC estimates percentile bounds at 0.001 and 0.999 with an inverse-time exponential moving average.
- SCULPT configuration: UADR uses target skewness γT = 0, target kurtosis κT = 3, α = 0.01, β = 0.01, and global regularization weight λ = 5×10−4.The regularization weight is reduced by a factor of 0.1 after SPC freezing.
- Quantization protocol: Standard FP32 PTQ, SCULPT PTQ, and QAT are compared under identical supergroup boundaries with per-channel symmetric weight quantization and per-tensor asymmetric activation quantization.PTQ uses min/max calibration over 32 batches, while QAT uses native PyTorch fake quantizers.
- Evaluation protocol: Checkpoint selection uses quantized validation accuracy under the matched supergroup PTQ contract rather than FP32 validation accuracy.Epoch-level PTQ tracking identifies the best deployment-relevant checkpoint for FP32, SCULPT, and QAT runs.
5 Experimental Results
SCULPT improves low-bit PTQ accuracy and activation statistics under the matched evaluation contract, outperforming naive FP32 PTQ and comparing favorably with the tested QAT baseline while reaching its best result in fewer steps.
- Main quantization results: 99.3631% top-1 accuracy is achieved by SCULPT PTQ in W8A8, versus 90.5478% for standard FP32 PTQ and 98.2420% for the tested QAT baseline.Under the same contract, SCULPT gains 8.8153 percentage points over FP32 PTQ and exceeds QAT by 1.1211 points.
- Main quantization results: 80.2803% top-1 accuracy is achieved by SCULPT PTQ in W4A8, versus 40.5605% for standard FP32 PTQ and 79.4395% for the tested QAT W8A8 baseline.The same trend holds at the lower-weight-bit setting without simulated quantization during training.
- Efficiency: 584 optimizer steps are needed for SCULPT to reach its best W8A8 PTQ result, compared with 2044 for QAT and 3577 for naive FP32 PTQ.The full SCULPT run lasts 16 epochs / 1168 steps, versus 30 epochs / 2190 steps for QAT and 80 epochs / 5840 steps for the original FP32 run.
- Activation distribution analysis: 0.01519 is SCULPT’s mean PTQ NRMSE/std across 131 encoded activation boundaries, down from 0.04818 for standard FP32 PTQ.The analysis examines activation distributions after training to explain the PTQ accuracy improvement.
- Activation distribution analysis: 11.56 is SCULPT’s mean absmax/std across the same boundaries, down from 42.04 for standard FP32 PTQ.These reductions are consistent with suppressing rare extreme values that dominate min/max calibration and stretch the quantization range.
- Activation distribution analysis: 19.99 is SCULPT’s representative-layer outlier score, versus 325.08 for FP32 PTQ and 129.12 for QAT; normalized PTQ error falls to 0.0163 from 0.2841 and 0.1452.The representative hard layer is post.features.4.2.block.1.2, and lower is better for the reported scalar metrics.
- Overall findings: SCULPT yields a substantially stronger PTQ checkpoint than naive FP32 PTQ and compares favorably to the tested W8A8 QAT baseline while requiring fewer optimization steps to reach its best quantized result.The controlled benchmark links the accuracy comparison with the observed activation-statistics changes.
6 Conclusion
SCULPT conditions full-precision training to improve later low-bit PTQ robustness without simulated quantization or runtime outlier-repair transforms, and achieves strong matched-contract results on EfficientNet-B0/ImageNette.
- Conclusion: SCULPT combines topology-aware activation regularization with stable percentile clipping to learn deployment-ready activation bounds during ordinary FP32 fine-tuning.The method suppresses quantization-hostile skewness and heavy tails while preparing bounds for later PTQ.
- Conclusion: 99.3631% top-1 accuracy is achieved by SCULPT PTQ in the primary W8A8 setting, compared with 90.5478% for standard FP32 PTQ and 98.2420% for tested W8A8 QAT.The comparison uses EfficientNet-B0/ImageNette under an identical supergroup QDQ contract.