PaperlayerView exact revision on arXiv

Source-linked AI summary

FP8 Quantization: The Power of the Exponent

Andrey Kuzmin, Mart Van Baalen, Yuwei Ren, Markus Nagel, Jorn Peters, Tijmen Blankevoort

arXiv:2208.09225v2cs.LG

TL;DR

Low-bit integer quantization is efficient, but the paper asks whether floating-point exponent bits can better accommodate neural-network value distributions. It analyzes FP8 design choices, develops efficient simulation and learning procedures, and finds that FP8 generally improves post-training accuracy over INT8 while quantization-aware training reduces the difference between formats.

  • Problem

    The paper examines whether low-bit floating-point quantization can provide inference benefits beyond widely used low-bit integer formats, especially when neural-network distributions contain outliers.

  • Method

    The authors analyze FP8 quantization theoretically, evaluate INT8 and FP8 in post-training and quantization-aware training, and introduce efficient simulation with learnable bias and bit allocations.

  • Results

    Post-training quantization generally favors FP8 over INT8 for neural-network accuracy, with exponent-bit needs increasing for networks with more outliers; 5M2E and 4M3E generally work best.

  • Takeaways & Limitations

    FP8 design should tune exponent bias and balance exponent and mantissa bits to the network’s outlier severity, while quantization-aware training can eliminate format differences.

  • Takeaways & Limitations

    The study compares formats primarily by accuracy and bit width, leaving hardware-specific power and latency impacts unresolved.

Abstract

from arXiv · show

When quantizing neural networks for efficient inference, low-bit integers are the go-to format for efficiency. However, low-bit floating point numbers have an extra degree of freedom, assigning some bits to work on an exponential scale instead. This paper in-depth investigates this benefit of the floating point format for neural network inference. We detail the choices that can be made for the FP8 format, including the important choice of the number of bits for the mantissa and exponent, and show analytically in which settings these choices give better performance. Then we show how these findings translate to real networks, provide an efficient implementation for FP8 simulation, and a new algorithm that enables the learning of both the scale parameters and the number of exponent bits in the FP8 format. Our chief conclusion is that when doing post-training quantization for a wide range of networks, the FP8 format is better than INT8 in terms of accuracy, and the choice of the number of exponent bits is driven by the severity of outliers in the network. We also conduct experiments with quantization-aware training where the difference in formats disappears as the network is trained to reduce the effect of outliers.

1 Introduction

The paper investigates whether low-bit floating-point formats can improve efficient neural-network inference over widely used integer quantization. It analyzes FP8 theoretically and practically across post-training and quantization-aware training settings.

  • 1 Introduction: FP8 adds exponent-based dynamic range to INT8-style quantization, making outliers less harmful to quantized distributions.The paper studies this benefit analytically, in post-training quantization, and in quantization-aware training.
  • 1 Introduction: The study spans analytical distribution and weight models, INT8 and FP8 post-training quantization, and quantization-aware training with multiple FP8 formats.The authors report strong agreement between theoretical predictions and practical results on real networks.
  • 1 Introduction: The proposed FP8 simulation implementation supports rapid experiments across many floating-point formats and learning the exponent bias and mantissa–exponent allocation.This makes FP8 configuration selection possible without manual intervention.
  • 1 Introduction: FP8 can outperform integer quantization for inference, but performance depends on tuning the bias and choosing the exponent–mantissa bit allocation.The paper presents this as its central conclusion from the study.

2 Background

Integer quantization uses scaled, clipped rounding for efficient low-bit computation, whereas floating-point formats trade mantissa precision against exponent-controlled dynamic range. The paper studies FP8 allocations and bias choices as extensions of this trade-off.

  • 2.1 Integer quantization: Integer quantization rounds and clips a scaled matrix into a chosen bit-width, then dequantizes it for approximate efficient computation.The dequantized approximation enables integer matrix multiplication instead of directly using the original values.
  • 2.2 Floating point number system: Floating-point values use a sign bit, mantissa bits, exponent bits, and an exponent bias to represent numbers across power-of-two ranges.Subnormal values reserve an exponent pattern to represent zero and values near zero.
  • 2.2 Floating point number system: More mantissa bits increase precision within each power-of-two interval, while more exponent bits increase representable dynamic range.Fixed-width floating-point formats must trade precision against range.
  • 2.2 Floating point number system: Floating-point formats provide greater precision near zero and can better match peaked or heavy-tailed distributions than integer formats.This allocation of precision and range is illustrated by the comparison of INT4, FP4, INT8, and FP8 grids.
  • 2.3 Assumptions and extensions: The study extends FP8 by allowing flexible exponent bias scales and evaluating 5M2E, 4M3E, 3M4E, and 2M5E allocations.The notation xMyE denotes x mantissa bits and y exponent bits.

3 Expected quantization error

The analysis models quantization error for weights, activations, and scalar products, showing that the best FP8 allocation depends on distributional outliers and that bias selection is important.

  • The expected quantization MSE is computed by separating rounding error from clipping error over the quantization grid and distribution support.The framework evaluates uniform, Gaussian, and heavy-tailed Student’s-t distributions to study the effect of outliers.
  • The scalar-product output error is driven mainly by input quantization errors and the spread of the input distributions.The output error is modeled from quantized weights and activations, including rounding and interaction terms.
  • Choosing the FP8 exponent bias correctly is essential because a fixed bias can cause excessive clipping or waste representable grid points.The paper favors floating-point bias over integer bias and suggests per-channel bias or scale for low-exponent formats.
  • More exponent bits become preferable as Student’s-t distributions develop stronger outliers, whereas INT8 suits uniform data and 2-exponent-bit FP8 suits Gaussian data.The analysis uses expected MSE and SQNR across distributions, scalar products, and a Resnet18 layer to compare formats.
  • FP8 formats with more exponent bits better represent distributions with severe outliers, while well-behaved Gaussian-like layers favor 5M2E.The Resnet18 example is analyzed by fitting Gaussian distributions to sampled weights and activations.

4 FP8 quantization simulation

The paper introduces an efficient FP8 quantization simulator that exposes format parameters for experimentation and learning, while representing FP8 through local uniform grids.

  • The simulation method exposes mantissa bits, exponent bits, and exponent bias, enabling these FP8 parameters to be learned through back-propagation.This supports both efficient framework implementation and experiments across floating-point formats.
  • FP8 quantization is simulated as the union of m-bit uniform grids between consecutive powers of two, with each input element assigned its own scale.The scale depends on the mantissa precision and the exponent range containing the input value.
  • The quantizer clips values outside the representable range, whose maximum and minimum depend on the exponent and mantissa configuration.Values beyond the maximum or below the smallest representable value are clipped to the corresponding limits.
  • For non-unit scaling, the method folds the scale into a reparameterized exponent bias before computing each element’s quantization scale.This incorporates the quantization scale into the FP8 representation rather than treating it separately.
  • Quantization-aware training uses straight-through gradients for rounding and learns the maximum clipping value c instead of the reparameterized bias for greater stability.The method treats the scale-related exponent term as constant during back-propagation so inputs receive the straight-through gradient.

5 Experiments

The experiments evaluate FP8 formats against INT8 across diverse neural networks and examine both post-training and quantization-aware training. PTQ results link exponent allocation to activation outliers, while QAT narrows differences between formats.

  • 5 Experiments: The study evaluates ResNet18, MobileNetV2, ViT, BERT-base, HRNet, DeepLabV3, and SalsaNext across classification, language understanding, and segmentation tasks.INT8 quantization serves as the comparison baseline, with range-estimation variants considered for weights and activations.
  • 5.2 Post-training quantization results: FP8 experiments compare fixed formats, flexible-bias formats, and fully flexible formats, allowing format parameters to vary from globally fixed to tensor- or channel-specific.The fully flexible procedure selects m, e, and bias settings to minimize MSE for each tensor or channel.
  • 5.2 Post-training quantization results: FP8 post-training results favor more exponent bits for networks with large activation outliers, while convolutional networks benefit from more mantissa bits.The trade-off makes per-channel bias selection important when increasing mantissa bits reduces dynamic range.
  • 5.2 Post-training quantization results: Fully flexible formats only sometimes outperform fixed m/e formats, with slim improvements attributed to a local greedy assignment method that may miss globally optimal settings.This limits the observed benefit of format flexibility despite its greater ability to fit network tensors.
  • 5.3 Quantization-aware training: QAT improves FP8 models and generally reduces accuracy differences between formats as weights adapt to the quantizers’ represented distributions.For ResNet18 and MobileNetV2, learning maximum value c and mantissa bits improves some initializations; the effect disappears for fully flexible PTQ initialization and is insignificant or slightly harmful for BERT.
  • 5.3 Quantization-aware training: Figure 6 compares maximum value c and mantissa bits m before and after QAT, starting from c = 60 and m = 3.The experiment illustrates that training can change quantization parameters from their initialization.

6 Related work

The related work covers integer quantization, conventional floating-point studies, and prior FP8 research. The paper positions itself as the first extensive study combining analytical and empirical comparisons of FP8 formats across tasks and data modalities.

  • Integer quantization: Integer quantization research is organized around post-training quantization and quantization-aware training, with INT8 and INT4 prominent because corresponding hardware is widely available.The paper studies floating-point alternatives against this established background.
  • Floating point formats: Earlier floating-point research generally focused on high-bit formats such as FP32 and FP64 rather than neural-network-specific low-bit behavior.The paper addresses this gap for FP8 inference.
  • Floating point formats: Prior FP8 studies explored FP16/FP8 training, hybrid formats, and flexible formats, but some retained fixed bias or mantissa/exponent allocations in the forward path.These works motivate broader investigation of format flexibility.
  • Floating point formats: The paper claims the first extensive study of FP8 formats based on both analytical insights and empirical results across several tasks and data modalities.It also distinguishes its simulation method from prior dedicated simulations by enabling efficient gradient-based learning of bias and bit configurations.

7 Impact and Limitations

The paper discusses FP8’s implications for hardware design while limiting its own evaluation to accuracy. It highlights flexible format choices as potentially useful for hardware decisions but leaves implementation-specific power and latency trade-offs unresolved.

  • Impact: Many networks may benefit from FP8 formats with more mantissa bits and per-channel flexible bias rather than widely supported fixed 3M4E or 2M5E formats.The authors intend these accuracy findings to inform FP8 hardware design decisions.
  • Limitations: The study restricts comparison to model accuracy and does not evaluate hardware-specific power consumption or latency.Data-transfer overhead is similar for the two 8-bit formats, but compute overhead depends on implementation.
  • Limitations: FP8 arithmetic may use more power for additions and multiplications, although surrounding logic or multipurpose hardware can amortize or eliminate the difference.The practical hardware trade-off therefore depends on the exact design and use case.

8 Conclusion

The paper finds that FP8 can outperform INT8 analytically and in post-training quantization, especially when exponent bits accommodate outliers. In quantization-aware training, however, these format benefits largely disappear.

  • 8 Conclusion: FP8 generally improves on INT8 for Gaussian-like neural-network distributions, while higher exponent-bit counts suit distributions with outliers.The conclusion also reports that 5M2E and 4M3E usually work best, whereas transformer-like networks benefit from more exponent bits.
  • 8 Conclusion: The FP8 simulator accelerates FP8 quantization experiments and can learn both the bias and mantissa–exponent bit-width trade-off.This makes the format’s flexibility usable without manual intervention.
  • 8 Conclusion: Quantization-aware training reduces the difference between FP8 and INT8 because networks learn to perform well on the INT8 quantization grid.
  • 8 Conclusion: The analytical quantization-error computation integrates rounding and clipping contributions over the explicitly defined quantization grid.The procedure supports Gaussian, Uniform, and Student’s t distributions.

A.2 Scalar product quantization error.

The scalar-product error is decomposed into rounding, clipping-related, and cross terms under an independence approximation for weights and activations. The dominant contributions are weighted input or weight rounding errors.

  • A.2 Scalar product quantization error.: Assuming independent weights and inputs, the expected scalar-product MSE is analytically decomposed into six terms.The decomposition includes weighted rounding errors, products of rounding errors, and cross terms involving the additional integrals.
  • A.2 Scalar product quantization error.: The first two terms usually dominate because they are the only integrals of non-negative functions, determining most of the MSE magnitude.
  • A.2 Scalar product quantization error.: Weight rounding error is weighted by the non-central second moment of the inputs, which does not depend on the quantization grid.The analogous second term interchanges weights and inputs.
  • A.2 Scalar product quantization error.: The auxiliary I and J functions enable analytical evaluation of rounding and scalar-product errors for different probability distributions.Formulas are provided for distributions including Uniform and Student’s t, with corresponding weight-density versions.

B.1 Quantization error ablation

The experiments connect analytical quantization-error predictions to neural-network tensors and examine how format choice changes with distributional outliers. They identify 5M2E as optimal in one ResNet18 layer and more exponent bits under heavier tails.

  • B.1 Quantization error ablation: The ResNet18 layer study finds 5M2E optimal for both fitted Gaussian weight and activation distributions.The fitted weights and activations are clipped at their stated ranges before comparing formats.
  • B.1 Quantization error ablation: Analytical and empirical quantization errors are compared using SQNR plots for random tensor subsets from ResNet18 and BERT.SQNR is log-proportional to MSE, so minimizing MSE maximizes SQNR.
  • B.1 Quantization error ablation: BERT tensors contain significant outlier tails, motivating separate per-layer SQNR views for weights and activations.
  • B.1 Quantization error ablation: Expanding the quantization range for a Student’s-t distribution with ν = 2 increases the optimal exponent-bit width.The experiment uses a min–max range estimator and clips the distribution at the quantization range.
  • B.1 Quantization error ablation: Output-activation MSE from injected Gaussian weight noise strongly correlates with the model’s final top-1 accuracy.

G Full experimental details

The QAT experiments train ResNet18 and MobileNetV2 with specified optimizers, schedules, and learning rates, while separately testing learned FP8 parameters. Gradients through the FP8 quantizer use straight-through estimation.

  • G Full experimental details: ResNet18 is trained for 20 epochs and MobileNetV2 for 10 using Adam, with starting learning rates of 10^-5 or 10^-6.The learning rates were selected from pilot experiments and decayed during training.
  • G Full experimental details: Experiments that learn FP8 clipping and mantissa parameters use SGD without momentum and learning rates from 10^-2 to 10^-5.No weight decay is applied.
  • G Full experimental details: The baseline INT8 QAT results follow the procedure described in prior work.
  • G Full experimental details: The FP8 quantizer uses a straight-through estimator to pass gradients through non-differentiable rounding operations.

I PTQ results

The PTQ results compare INT8 with several FP8 configurations across models and GLUE tasks, while highlighting model-specific degradation linked to weight-distribution outliers.

  • DeepLabV3 suffers greater degradation under fixed-format PTQ than the other considered models.The paper links this behavior to the distributions of values in its weight tensors.
  • Table 3 reports the best PTQ results across all evaluated models for INT8 and three FP8 configuration families.The comparison includes fixed-bias FP8, flexible-bias FP8, and fully flexible FP8.
  • Table 4 reports per-task BERT results across all GLUE tasks, with the best quantized result marked for each task.
  • Early DeepLabV3 backbone layers contain large outliers requiring more exponent bits, whereas later backbone and decoder layers require fewer.This mismatch also appears in other networks, most notably MobileNetV2.
Loading 2208.09225v2…