PaperlayerView exact revision on arXiv

Source-linked AI summary

Scaling Laws for Post Training Quantized Large Language Models

Zifei Xu, Alexander Lan, Wanzin Yazar, Tristan Webb, Sayeh Sharify, Xin Wang

arXiv:2410.12119v3cs.LGcs.CL

TL;DR

Pre-training scaling laws make LLM quality predictable, but post-training quantization adds factors that leave its quality uncertain and make choosing a model size and format costly. The paper systematically studies these factors across LLMs and develops a statistical predictor; it reports that PTQ outcomes can be predicted reasonably well and describes a model-size/precision Pareto frontier. The study’s predictive rules have so far been tested only on models up to 13 billion parameters.

  • Problem

    Post-training quantization quality remains unpredictable despite pre-training scaling laws, while finding a model size and quantization format under fixed deployment constraints can be time- and compute-intensive.

    Source: §1 · pp. 1–2

  • Method

    The authors systematically study pre-trained loss, local loss landscape, low-precision data formats, and PTQ algorithms, then use these factors in a statistical model to predict quantization outcomes.

    Source: §1 · p. 2

  • Results

    The authors report empirical scaling laws for PTQ factors and reasonable-accuracy predictions of post-quantization NLL across model sizes and distinct model families.

    Source: §1 · p. 2, §5 · p. 7

  • Takeaways & Limitations

    The study identifies a Pareto frontier for trading off larger lower-precision models against smaller higher-precision models.

    Source: §5 · p. 7

    Experiments cover models only up to 13 billion parameters, so predictive power on much larger LLMs remains unvalidated.

    Source: §6 · p. 7

Abstract

from arXiv · show

Generalization abilities of well-trained large language models (LLMs) are known to scale predictably as a function of model size. In contrast to the existence of practical scaling laws governing pre-training, the quality of LLMs after post-training compression remains highly unpredictable, often requiring case-by-case validation in practice. In this work, we attempted to close this gap for post-training weight quantization of LLMs by conducting a systematic empirical study on multiple LLM families quantized to numerous low-precision tensor data types using popular weight quantization techniques. We identified key scaling factors pertaining to characteristics of the local loss landscape, based on which the performance of quantized LLMs can be reasonably well predicted by a statistical model.

1 Introduction

Pre-training scaling laws make model quality predictable, but post-training quantization can obscure that predictability and require costly trial and error. The paper studies PTQ scaling factors and uses them to predict quantization outcomes for unseen LLMs at reasonable accuracy.

1 Introduction

  • Pre-training scaling laws relate generalization to model size, data volume, and computation, whereas PTQ introduces additional factors that can make final quality unpredictable.The authors describe PTQ as preserving a trained model’s generalizability while using low-precision computation.Source: §1 · p. 1
  • Finding a model size and quantization format under fixed memory, compute, and data-format constraints can require many costly iterations.The authors illustrate the search trade-off with a larger, lower-bit model versus a smaller, higher-bit model.Source: §1 · p. 2
  • The study systematically examines pre-trained loss, local loss landscape, quantization data type, and PTQ algorithm as factors in quantized-model quality.The authors note that PTQ algorithms commonly minimize local quantization error rather than directly optimizing global loss.Source: §1 · p. 2
  • The authors report empirical scaling laws for PTQ factors and a statistical model that predicts outcomes on unseen LLMs with reasonable accuracy.The factors include pre-trained loss, local loss landscape, low-precision data type, and PTQ algorithm.Source: §1 · p. 2

2 Factors subject to scaling for LLM PTQ

The section traces how pre-training loss, local loss geometry, quantization format, and GPTQ relate to quantized-model loss. Local landscapes and quantization effects show systematic patterns, although GPTQ’s benefit varies with model and precision.

2.1 Loss of pre-trained LLM

  • For well-trained LLMs with no data limit, GPT-2, OPT, and BLOOM roughly follow one power law, whereas Llama 2/3 track a different but qualitatively similar path.Source: §2.1 · pp. 2–3

2.2 Characteristics of local loss landscape

  • Within a model family, larger LLMs have flatter local loss landscapes, and typical radial loss profiles are step-like across sizes and families.They remain relatively flat near the pretrained weights, then rise rapidly and eventually plateau.Source: Figure 2 · p. 3, §2.2 · p. 4

2.3 Low-precision data type for quantization

  • Across model families and sizes, SQNR varies relatively little, while NLL loss is more nonlinear and less predictable, with lower-precision formats roughly tending toward higher loss.The SQNR pattern instead varies predictably across numerical data types.Source: §2.3 · p. 4

2.4 PTQ optimization method

  • GPTQ generally reduces both SQNR and NLL loss, but the loss reduction varies systematically with model size and quantization precision.For OPT models, the SQNR reduction is relatively consistent across sizes and formats, whereas the NLL-loss reduction is highly variable.Source: §2.4 · p. 4, Figure 4 · p. 5
  • For opt-1.3b, GPTQ greatly improves mxint3_128 quantization but only marginally improves mxint6_128, mxint4_128, and mxint2_128.The effect is non-monotonic across precision; the figure contrasts pre- and post-GPTQ results with mapped loss landscapes.Source: §2.4 · p. 4, Figure 3 · p. 5
  • GPTQ seeks a flatter perturbation direction, but its substantial benefit occurs only within a narrow SNR window that depends on model family, size, and weight format.Linear or quadratic approximations typically fail to characterize the step-like radial profiles below 20 dB.Source: §2.4 · p. 5

3 Building a predictive model

The predictive model combines scaling-related loss-landscape and quantization features to estimate post-GPTQ NLL. Its predictions were reasonably accurate on two held-out LLMs from unseen families.

3 Building a predictive model

  • The empirical rules combine model-size-dependent local loss-landscape characteristics with data-type and PTQ effects on SQNR and loss.The study reports these factors as scaling in a predictable way.Source: §3 · p. 6
  • The random forest uses measured model, loss-landscape, quantization, and numerical-format features to predict the final post-GPTQ NLL.Inputs include parameter count, pre-trained and RTN losses, RTN SQNR, radial loss-landscape slope, and format precision, exponent bits, and block size.Source: §3 · p. 6
  • Predictions were reasonably accurate on two held-out models from unseen LLM families, despite differences in architecture, training paradigm, and local loss landscape.The held-out models were EleutherAI/pythia-1b and mosaicml/mpt-7b.Source: §3 · p. 6

4 Experimental procedures

The experiments covered five LLM families and two held-out models, using WikiText-2 and a broad set of microscaling formats. They applied an enhanced GPTQ procedure and evaluated losses and loss-landscape features at specified precisions.

4.1 Models and dataset

  • Five families—GPT-2, OPT, BLOOM, Llama 2, and Llama 3—were studied, with Pythia-1b and MPT-7b held out for validation on WikiText-2.128 training examples calibrated PTQ, and the validation split was used for evaluation.Source: §4.1 · pp. 6–7

4.2 Numerical tensor data type and notations

  • The study tested 36 microscaling data types, with block sizes from 16 to 128 and element precision from 2 to 6.MX formats use a shared block scaling factor; notation records precision and block size, with mxfp notation also recording element exponent and mantissa bits.Source: §4.2 · p. 7

4.3 GPTQ

  • Enhanced MX-compatible GPTQ tuned each layer’s dampening factor by grid search to minimize output MSE and accumulated Hessians before optimization.The grid spanned 10^-3 through 10^4, and Hessian accumulation used 128 training sequences.Source: §4.3 · p. 7

4.4 Loss landscape mapping

  • NLL losses were evaluated on the full validation split at half precision, while Hessian-vector products were computed in single precision using PyHessian.Source: §4.4 · p. 7

5 Conclusion

The study finds that post-training quantization outcomes can be predictable through scaling laws involving local loss landscapes, numerical formats, and PTQ algorithms. It presents a model-size and precision tradeoff and argues that these findings can inform deployment on resource-constrained devices.

5 Conclusion

  • Post-training quantization outcomes for well-trained LLMs can be predictable through scaling laws governing local loss landscapes, numerical formats, and PTQ effects.Source: §5 · p. 7
  • The study’s comparison across models and quantization formats establishes a Pareto frontier between model size, quantization precision, and quality.Source: §5 · p. 7
  • The authors argue the scaling laws can guide model-size and data-format choices for inference workflows and resource-constrained devices.Source: §5 · p. 7

6 Limitations

The experiments covered models up to 13 billion parameters because of computational-resource constraints, leaving predictive power on much larger LLMs pending validation.

6 Limitations

  • The study’s model-size scope reached 13 billion parameters because of computational-resource constraints.Source: §6 · p. 7
  • Predictive power of the scaling rules on much larger LLMs remains pending further validation.Source: §6 · p. 7
  • Figure 6 presents the tradeoff between quantized model weight size and generalization across the models and quantization formats shown.Source: Figure 6 · p. 8

A Scaling of ℓ2-norms of model weights

Weight ℓ2-norms scale close to a square-root power law in most studied model families, while the accompanying figures show SQNR and NLL across MX formats before and after PTQ.

A Scaling of ℓ2-norms of model weights

  • Weight ℓ2-norms scale close to a square-root power law in all studied families except GPT-2 and OPT.This pattern is correlated with constancy of SQNRs across models for all MX data types.Source: A Scaling of ℓ2-norms of model weights · p. 13

B SQNR and NLL of MX formats

  • Figure 8 shows RTN SQNRs and NLL losses before PTQ across model families, with model size encoded by symbol size.Families are color-coded.Source: Figure 8 · p. 13
  • Figure 9 shows SQNRs and NLL losses after PTQ for GPTQ across model families.Source: Figure 9 · p. 14

C Scaling in the case of PTQ to traditional int quantization

Traditional int quantization produces more variable SQNRs and less predictable GPTQ-related changes in SQNR and NLL than MX quantization. The study varies precision and granularity using calibrated symmetric quantizers.

C Scaling in the case of PTQ to traditional int quantization

  • Int quantization requires calibration of scale and, depending on the scheme, zero point at per-tensor, per-group, or per-channel granularity.These extra parameters and procedures likely add variability to PTQ scaling.Source: C Scaling in the case of PTQ to… · p. 14
  • The experiments use symmetric int quantizers at varied precisions and granularities, calibrated by minimizing quantization MSE on 128 training-split sequences.Source: C Scaling in the case of PTQ to… · p. 14
  • Int-quantization SQNRs are much more variable than MX SQNRs and do not appear to scale monotonically with model size.GPTQ-related changes in SQNR and NLL are also less predictable for int than for MX formats.Source: C Scaling in the case of PTQ to… · p. 14

D Interpretation of the importance of input features to the predictive model

The predictive model’s feature interpretation highlights RTN NLL and quantization-format properties, alongside model and local-loss characteristics. The paper illustrates feature contributions for one MPT-7B quantization.

D Interpretation of the importance of input features to the predictive model

E Cost of loss landscape feature computation

The authors report that extracting predictive features is more economical than running GPTQ, with the overhead gap growing with model size. They measure the comparison using OPT models on one A100 GPU.

E Cost of loss landscape feature computation

F Generalization across different datasets

Experiments on LAMBADA and a subset of C4 found that quantization-related scaling trends were consistent across datasets. Predictive models trained within or across datasets generalized to unseen language models or datasets.

F Generalization across different datasets

Related papers

Loading 2410.12119v3…