Source-linked AI summary
H-Scale: Hessian-Guided Scale Refinement for NVFP4 Sub-Byte LLM Inference
Hao Yu, Zheng Li, Dayiheng Liu, Jianwei Zhang
TL;DR
NVFP4’s fine-grained scale space makes per-group scale selection an underexplored calibration problem, while existing PTQ methods primarily refine quantized values. H-Scale uses calibration-activation-derived diagonal second-order weighting to select hardware-valid scales, and generally improves diverse NVFP4 baselines while narrowing the gap to BF16 in several settings. Its refinement is lightweight and adds no inference cost.
Problem
NVFP4’s fine-grained per-group scales are highly sensitive, while existing PTQ methods primarily refine weight values rather than this scale-selection step.
Method
H-Scale selects hardware-valid per-group scales by minimizing a diagonal-Hessian-weighted reconstruction objective computed from calibration activations.
Results
H-Scale generally yields higher average performance across diverse NVFP4 baselines and narrows the gap to BF16 in several complex reasoning settings.
Takeaways & Limitations
H-Scale provides a plug-and-play NVFP4 scale-refinement step with zero additional inference cost.
Takeaways & Limitations
The study is limited to NVFP4 with g = 16 and text-only LLMs; transfer to other formats, group sizes, or multimodal models remains open.
Abstract
from arXiv · showhide
The NVIDIA Blackwell architecture, with native support for the ultra-fine-grained NVFP4 format, opens new opportunities for accelerating large language model (LLM) inference. NVFP4's micro-block design, such as a group size of 16, offers strong representational flexibility for capturing local weight distributions and isolating outliers, but it also introduces a large and highly sensitive space of per-group scaling factors. Existing post-training quantization (PTQ) methods primarily focus on refining quantized weight values, leaving this scale-selection step underexplored. To address this gap, we propose \textbf{H-Scale}, a lightweight post-processing method for NVFP4 per-group scale refinement. Instead of minimizing plain weight reconstruction error, H-Scale selects hardware-valid group scales using a diagonal second-order proxy derived from calibration activations, thereby targeting layer output perturbation more directly. It is designed as a drop-in replacement for RTN-style scale selection in diverse NVFP4 pipelines, requires only modest offline calibration, and introduces strictly zero overhead at inference time. Under a fixed evaluation protocol, experiments on mainstream LLMs show that H-Scale generally improves a broad range of NVFP4 baselines and brings several variants closer to the BF16 reference.
1 Introduction
NVFP4’s fine-grained scales improve local representation but make scale calibration a major accuracy factor. H-Scale addresses this with activation-derived Hessian-guided scale selection as a lightweight, zero-inference-cost refinement.
- Motivation: NVFP4’s 16-value micro-blocks capture local weight distributions and isolate outliers, but per-group scale miscalibration can strongly affect accuracy.Each group couples E2M1 4-bit values with a shared E4M3 scale.
- Motivation: Existing PTQ methods mainly reduce quantization error by reconstructing or regularizing low-bit weight values, leaving scale selection underexplored.H-Scale treats scaling factors as a separate calibration target.
- Method: H-Scale chooses per-group scales with a diagonal-Hessian-weighted reconstruction objective derived from calibration activations.The objective shifts from plain weight reconstruction toward output-aware weighted reconstruction.
- Practicality: H-Scale is designed as a lightweight post-processing step that can be grafted onto existing NVFP4 baselines, runs in minutes, and adds zero inference cost.It is positioned as a drop-in replacement for baseline scale selection.
- Method: H-Scale enumerates neighboring hardware-valid FP8 local scales around a baseline and retains the candidate minimizing Hessian-weighted reconstruction error.The quantized weights remain in native NVFP4, so inference cost is unchanged.
- Results: Across diverse evaluations, H-Scale generally improves average performance and narrows the NVFP4-to-BF16 gap in several complex reasoning settings.The reported comparison is under a fixed evaluation protocol.
2 Related Work
Prior work addresses low-bit quantization through weight reconstruction, activation-aware scaling, transforms, and fine-grained NVFP4-specific designs. H-Scale instead refines final per-group NVFP4 scales using an output-aware second-order objective, complementing those approaches.
- PTQ for LLMs: GPTQ and related PTQ methods compensate low-bit errors through weight reconstruction or regularization, while other methods protect salient channels or mitigate activation outliers.The related methods include GPTQ, AWQ, GPTAQ, and SpinQuant.
- Fine-Grained Quantization: Blackwell’s native 4-bit floating-point support has motivated NVFP4 algorithms that exploit group size 16 while addressing increased scale sensitivity.MR-GPTQ, 4over6, and ArcQuant represent distinct NVFP4-specific strategies.
- Scale Refinement: Scale-refinement methods range from learned scales during fine-tuning to activation-weight scale migration, but they target different pipeline stages.LSQ incurs training costs, whereas SmoothQuant focuses on scale placement between activations and weights.
- H-Scale’s Positioning: H-Scale differs through a diagonal second-order, output-aware reconstruction loss applied at the final per-group scale-selection step.This placement makes it complementary to prior reconstruction and structural quantization methods.
3 Method
H-Scale refines NVFP4 per-group scales by replacing plain weight MSE with a diagonal-Hessian, output-aware objective and searching hardware-valid neighboring scales. The resulting drop-in search preserves the native quantization pipeline while targeting layer-output error more directly.
- NVFP4 quantization: NVFP4 partitions weights into groups of 16, using E2M1 values with per-group E4M3 scales and a tensor-level FP32 global scale.The effective group scale is the product of the global and local scales.
- Hessian-guided objective: H-Scale weights reconstruction errors by a diagonal Hessian derived from calibration activations instead of using plain weight MSE.The diagonal proxy retains per-channel activation energy while dropping cross-channel correlation terms.
- Group-wise scale selection: For each group, H-Scale independently selects the hardware-valid effective scale that minimizes the corresponding weighted reconstruction error.The candidate set is induced by a bidirectional walk over neighboring E4M3 local scales.
- Efficient scale search: The deterministic search keeps baseline grouping and the FP4 value grid unchanged, refining only the per-group scale around the baseline initialization.The default window uses nine smaller and six larger E4M3 steps, with endpoint clipping and exact hardware representability.
- Output-aware refinement: H-Scale allocates limited FP4 resolution toward weights that matter most for calibration-layer output, while retaining the same hardware-valid scale space.This replaces identity weighting in scale-only search with the diagonal Hessian proxy.
- Validation and integration: On Qwen3-4B-Instruct, H-Scale reduces activation loss by 32.5% relative to Min-Max, versus 40.8% for the Best Scale oracle.Weight-Guided Scale achieves the lowest unweighted weight MSE but only a 12.6% activation-loss reduction; H-Scale can replace RTN-style selectors without inference changes.
4 Experiments
Experiments evaluate H-Scale across NVFP4 baselines, model sizes, model families, ablations, scale shifts, and runtime. H-Scale generally improves average benchmark performance while preserving inference cost and using a compact candidate window.
- Experimental Setup: The evaluation tests average performance, transfer, and offline-cost trade-offs across diverse NVFP4 baselines, models, and benchmark suites.Benchmarks cover reasoning, math, knowledge, and coding, using fixed decoding with three independent runs.
- Main Results: H-Scale yields higher average scores across Qwen3-30A3-Thinking, Qwen3-30A3-Instruct, Qwen3-4B-Instruct, and LLaMA-3.1-8B-Instruct evaluations.The reported gains extend across model sizes and families, although individual benchmark improvements remain method-dependent.
- Ablation Analysis: H-Scale gives the highest average score in the Hessian ablation, while identity-weighted scale-only optimization is less stable and can hurt 4over6 and ArcQuant.The comparison distinguishes Hessian-weighted scale refinement from scale adjustment alone.
- Efficiency and Hardware Deployment: Inference cost remains unchanged, while GPTQ runtime rises from 9.6 h to 9.9 h and representative 4over6 runtime rises from 0.2 h to 0.4 h.The added cost is offline scale search and, for some baselines, a one-time activation pass.
- Ablation Analysis: 47.9% of groups select a larger scale, 31.2% retain max-abs initialization, and 20.9% select a smaller scale.These statistics pool 1.68 × 10^8 groups from the three MLP projections of Qwen3-4B-Instruct.
- Ablation Analysis: Deep contractions δ ≤−4 reduce Hessian-weighted rounding error by 58.7% on average, while 0.11% of groups account for 8.3% of total error.Nearby shifts reduce error more modestly, including 32.8% at δ = −1.
- Ablation Analysis: A candidate window with U ≥6 reaches 100.00% median recovery and keeps P10 above 99.6%, while the full window removes 50.6% of initialization error.The selected default is U = 6 and K = 16; wider windows add little recovery.
5 Conclusion, Limitations, and Future Work
The paper identifies per-group scale selection as an important NVFP4 fidelity bottleneck and presents H-Scale as a plug-and-play diagonal-Hessian refinement. Its scope is limited to NVFP4 with group size 16 and text-only LLMs.
- Conclusion: H-Scale generally improves average performance across diverse NVFP4 baselines, model sizes, and Qwen and LLaMA model families.The method is presented as a plug-and-play post-processing step.
- Limitations and Future Work: The study is limited to NVFP4 with g = 16 and text-only LLMs, leaving transfer to other formats, group sizes, and multimodal models open.The authors identify multimodal scale tuning as future work.
A.1 Calibration Microcase
The microcase shows why output-aware scale selection can favor a different FP4 pattern than weight-centric search. Layer-level curves extend this behavior beyond the microcase, while comparisons distinguish output error from weight MSE.
- A.1 Calibration Microcase: Output-aware scoring selects FP4 code pattern [−4, −2, 4, 6] despite larger unweighted error, reducing output error to 0.079.The corresponding output errors are 3.927 for Min-Max and 3.448 for weight-centric search.
- A.1 Calibration Microcase: The gate-projection layer-wise trend also holds for the other MLP projections, motivating additional up-projection and down-projection curves.Figures 3 and 4 report these additional curves.
- A.1 Calibration Microcase: H-Scale can have higher weight MSE than Min-Max while achieving lower layer-output error, showing that weight MSE is an unreliable output-fidelity proxy.This behavior is reported for down projections at early and final layers.
B Implementation Pseudocode
The implementation inserts a parallel, hardware-valid scale search into NVFP4 quantization pipelines. It uses calibration-derived diagonal Hessian weights, searches neighboring FP8 scales, and leaves GPTQ’s blockwise propagation unchanged.
- Implementation: Calibration activations provide diagonal weights h, computed as diag(X⊤X) or reused from Hessian statistics already available in GPTQ-like baselines.The weights align with each NVFP4 group.
- Implementation: The search is fully parallelizable across row-group pairs and candidates, enabling batched GPU scoring of FP4 projections and element-wise weighted errors.This separability keeps the additional offline computation small.
- GPTQ Integration: In GPTQ integration, H-Scale replaces only the local quantization call; blockwise GPTQ error computation, propagation, and final weight assembly remain unchanged.The GPTQ pseudocode retains inverse-Cholesky processing and standard error propagation around the refined local step.
- HScaleGroupQuantize: The selected scale is the candidate minimizing the weighted objective, after which the group is quantized once with that hardware-valid scale.The quantized group is returned in native NVFP4 representation.
C Evaluation Protocol Details
The evaluation uses model-specific benchmark suites and a fixed decoding protocol, with repeated runs to reduce the impact of stochastic variation.
- Benchmark suites: Qwen3-30A3-Thinking uses five tasks, while Qwen3-30A3-Instruct and Qwen3-4B-Instruct use eight-task suites.
- Benchmark suites: LLaMA-3.1-8B-Instruct is evaluated on a compact seven-task suite that omits tasks with near-random or zero BF16 performance.
- Decoding protocol: Three independent runs use fixed sampling settings, with mean scores reported and no significance assigned to small deltas or slight BF16 exceedances.
D Run-to-Run Variability of the Main Results
Main-text results average three end-to-end evaluations, and suite-level averaging substantially reduces run-to-run fluctuation relative to individual task variability.
- Each configuration is evaluated three times, with main-text results reporting the per-run mean.
- The Avg. standard deviation is computed across the three per-run suite averages rather than propagated from per-task columns.
- 0.37 points: the maximum Avg. standard deviation after averaging five- and eight-task suites, versus per-task deviations reaching 1.00 points.
E Hessian Ablation on Qwen3-30A3-Thinking
The ablation indicates that Hessian-weighted scale optimization outperforms scale-only optimization across two NVFP4 baselines, while the benefit persists with a smaller calibration set.
- Hessian-weighted scale optimization gives the strongest average score for both ArcQuant and 4over6 on Qwen3-30A3-Thinking.
- The ablation supports attributing the gain to calibration-output sensitivity weighting rather than merely shrinking or retuning scales.
- 128 samples: H-Scale still improves over the corresponding NVFP4 baseline on Qwen3-30A3-Instruct.