PaperlayerView exact revision on arXiv

Source-linked AI summary

The Integer Alibi: Localizing Cross-Kernel Divergence in INT8-Quantized LLM Inference

Teng-Ruei Chen

arXiv:2608.13756v1cs.LG

TL;DR

The paper tests whether interchangeable INT8 linear kernels actually produce identical inference results. By isolating CUTLASS and Triton kernels and using accumulator reasoning plus layer-level probes, it finds divergence after accumulation, making integer-kernel interchangeability testable rather than assumed.

  • Problem

    Quantized inference engines often select among implementations of the same linear-layer interface, but evidence that these kernels are interchangeable is limited and selection is typically opaque.

  • Method

    The study swaps only the INT8 kernel under an audited configuration, proves the shared INT32 accumulator exact, and probes scale application and output rounding layer by layer.

  • Results

    188 of 196 layers differ under real scales, with every differing finite element within one bfloat16 ulp, while no greedy sequence matches across the compared runs.

  • Takeaways & Limitations

    For integer paths, kernel interchangeability is checkable through exact accumulator controls, per-layer predictions, and rounding-level tolerances.

  • Takeaways & Limitations

    The evidence covers one consumer GPU, one model family at two sizes, one engine version, and two INT8 implementations, so it does not establish generality across systems.

Abstract

from arXiv · show

Two GPU kernels implementing the same scaled INT8 GEMM interface are usually treated as interchangeable. We test that assumption: holding the checkpoint, prompts, hardware, inference engine, decoding, and quantization configuration fixed, we swap only the INT8 linear kernel (CUTLASS versus Triton) inside vLLM. At 1.7B each arm reproduces itself bit-for-bit across cold restarts, yet the arms agree on no sequence in any end-to-end comparison we ran (0/8, 0/16, and 0/64). What makes this more than a benchmark discrepancy is an integer alibi: for shared INT8 operands under a verified no-overflow bound, the INT32 dot product is exact and order-independent, so the accumulator cannot be the source of any difference. Feeding both kernels identical operands from every linear layer of Qwen3-1.7B and 8B (196 and 252 layers), we find bit-identical outputs under power-of-two scales, confirming a pinned prediction list 196/196 and 252/252 (pre-registered at 1.7B, pinned but not blind at 8B), and observed differences of at most one bfloat16 spacing under the checkpoints' real scales. This localizes the divergence to scale application and output rounding after the exact accumulator. Applied as a probe checkpoint, the same intervention restores end-to-end bitwise agreement (8/8 and 16/16 sequences). Cross-implementation FP8 GEMM shows a different signature: both the prevalence and the magnitude of differences grow with reduction depth, while the INT8 fraction stays at parts per million and within one spacing over a 64x range of K. Teacher-forced replay ties layers to tokens: flips concentrate at small logit margins, which predict flip risk with ROC-AUC 0.94 on 16,384 positions. We will release the pre-registration, per-layer predictions, manifests with kernel-selection evidence, and a conformance procedure that turns these controls into a concrete check for kernel interchangeability.

I. INTRODUCTION

Under tightly controlled INT8 inference, swapping only vLLM’s CUTLASS and Triton linear kernels produces different deterministic greedy continuations, contradicting their assumed interchangeability. An integer-alibi argument localizes the cause beyond exact INT32 accumulation, to scale application and output rounding.

  • Kernel interchangeability: Every one of 64 compared greedy continuations changed when only the selected INT8 linear kernel was swapped.The checkpoint, prompts, engine, decoding, and quantization configuration were held fixed; the swap used one environment variable.
  • Kernel interchangeability: At 1.7B, each kernel reproduced its own outputs bit-for-bit across two cold restarts, showing two deterministic functions rather than noisy execution.The 8B and 64-prompt runs were not repeated for restart reproducibility.
  • Integer alibi: For shared INT8 operands without INT32 overflow, the accumulator is exact and reduction-order independent, so divergence must arise during scale application or output rounding.This verified bound is the paper’s “integer alibi,” providing a falsifiable localization and negative control.
  • Controls and conformance: The paper combines audited kernel-only comparisons, execution-log kernel manifests, and specialized conformance checks to test scaled-INT8 kernel interchangeability.The final teacher-forced comparison records the actually selected kernel class and verifies it through an identity contract.

II. RELATED WORK · A. Platform and configuration

Related work is organized by what deployment variable changes while other factors remain fixed, progressing from platform-level treatments to single-kernel substitutions. Prior studies show that fixed weights and inputs can still yield different inference outputs across hardware, algorithms, batch sizes, GPU configurations, and precision settings.

  • II. RELATED WORK: The comparison axis is which factor varies while everything else is held fixed.The paper organizes prior work from the coarsest treatment variable, the platform, to the finest, a single kernel.
  • II. RELATED WORK: Prior work is compared by numerical path, observation scale, and available control.Table I summarizes these dimensions before the related-work discussion proceeds chronologically.
  • II. RELATED WORK: The paper distinguishes platform-level treatments from variation confined to a single kernel.This ordering places the present kernel comparison at the finest end of the related-work treatment axis.
  • A. Platform and configuration: CPU instruction selection and GPU convolution algorithm choices produced unexpected output deviations across, and sometimes within, platforms.Schlögl et al. reported these deviations despite the deployment comparison being framed around platform and algorithm choices.
  • A. Platform and configuration: Evaluation batch size, GPU count and type, and precision materially changed greedy LLM outcomes.Yuan et al. attributed the effect to floating-point non-associativity under varying reduction orders.
  • A. Platform and configuration: Fixed weights and inputs do not guarantee numerically identical inference across deployment stacks.The cited platform and configuration studies establish this limitation before the paper narrows its focus to kernel-level variation.

B. Execution regime and engine · C. Kernel boundaries, audits, and contracts · D. Quantized GEMM kernels

The paper frames kernel interchangeability as a boundary-semantics problem rather than only a throughput question, building on prior work that isolates determinism, truncation, teacher-forced effects, and integer exactness. Its quantized-GEMM analysis centers on no-overflow constraints and auditable low-level arithmetic semantics.

  • B. Execution regime and engine: Prior work distinguishes fixed-shape run-to-run determinism from invariance across batch sizes.He separated these properties explicitly.
  • B. Execution regime and engine: TBIK aligns reduction trees across tensor-parallel sizes to preserve the same computation.The alignment serves cross-configuration invariance, at a measured throughput cost for batch-invariant kernels.
  • B. Execution regime and engine: The Silent Hyperparameter shows that swapping the whole inference backend can shift benchmark scores by up to double digits.The comparison holds weights, decoding, and hardware fixed, while backend changes bundle multiple execution components.
  • C. Kernel boundaries, audits, and contracts: HEAL attributes heterogeneous-GPU 16-bit discrepancies to truncation at kernel boundaries rather than the FP32 interior.It uses teacher forcing to separate local deviations from autoregressive cascade; this paper reuses both insights for INT8.
  • C. Kernel boundaries, audits, and contracts: Hawkeye reproduces tensor-core arithmetic on CPU and audits rounding, subnormal handling, and accumulation order at instruction level.The work treats low-level arithmetic semantics as testable.
  • D. Quantized GEMM kernels: Quantized-kernel research has mostly prioritized which implementation is fastest, with some work instead examining numerical integer exactness.Ootomo et al. [24] and Abdelfattah et al. [25] emulate FP64 GEMM on integer matrix units.
  • D. Quantized GEMM kernels: Integer-exactness studies derive admissible reduction depth as a function of accumulator width under a no-overflow condition.Their constraint is expressed in the log domain rather than as a magnitude bound, with instantiated thresholds differing between the two works.

E. Concurrent work inside our review window · III. THE EXACTNESS STRUCTURE OF A SCALED INT8 GEMM

Concurrent work makes adjacent INT8-equivalence and margin-trigger arguments without claiming priority, while this section formalizes scaled INT8 GEMM into exact accumulation, representability, and epilogue stages. Under stated bounds, only scale application and output rounding retain implementation freedom, and a power-of-two probe removes it.

  • E. Concurrent work inside our review window: The paper positions several 2026 preprints as concurrent adjacent work and explicitly declines priority claims when their arguments overlap.
  • E. Concurrent work inside our review window: Concurrent work reports byte-identical INT32 accumulators across ARM dispatch paths, but without this paper’s epilogue decomposition or pre-registered per-layer predictions.
  • E. Concurrent work inside our review window: MarginGate triggers verification on low top-1 margin for batch-invariant serving, whereas this work estimates and calibrates the margin-to-
  • III. THE EXACTNESS STRUCTURE OF A SCALED INT8 GEMM: A scaled INT8 GEMM comprises int8 multiplication with per-token and per-channel scales, INT32 accumulation, and a final bfloat16 cast.
  • III. THE EXACTNESS STRUCTURE OF A SCALED INT8 GEMM: K ≤32,768 satisfies the conservative accumulator bound with about fourfold worst-case margin, while the largest observed accumulator is 2,374,517.
  • III. THE EXACTNESS STRUCTURE OF A SCALED INT8 GEMM: Every integer accumulator with |acc| ≤ 2^24 is exactly representable in float32, making the representability condition checkable before execution.
  • III. THE EXACTNESS STRUCTURE OF A SCALED INT8 GEMM: The epilogue is the intentional divergence point because kernels may fuse the same arithmetic differently while remaining conventionally correct.
  • III. THE EXACTNESS STRUCTURE OF A SCALED INT8 GEMM: Under exact accumulation and representability bounds, real-scale differences arise only from two scale multiplications and the output cast; power-of-two weight scales require bitwise equality.Power-of-two scaling commutes with rounding for finite normal values, but the lemma requires no overflow or underflow and matching rounding semantics.

IV. METHODS AND GOVERNANCE

The study pins the hardware, software, model, quantization, and kernel-selection stack while auditing treatment isolation through manifests and runtime evidence. Governance distinguishes the pre-registered 1.7B case from the pinned-but-not-blind 8B replication and records protocol deviations explicitly.

  • Stack: Measurements used one RTX 4090 and pinned vLLM, SGLang, Qwen3, W8A8 INT8, quantization-tool, checkpoint, and kernel-selection configurations.The two arms selected CutlassInt8ScaledMMLinearKernel or TritonInt8ScaledMMLinearKernel while sharing the engine’s activation-quantization operation.
  • Deviations: Four pre-registered items were omitted: locked clocks, the {1, 4, 16} batch sweep, forward-and-reverse ordering rounds, and a related task.Per-layer captures instead used a single prefill-like tile with M=512, leaving batch dependence untested at layer level.
  • Replication timing: The 8B results are a pinned but not independent replication, because an earlier run on the same data was known before the authoritative rerun.The paper treats 1.7B as the pre-registered case and does not present the two model sizes as equivalent evidence.
  • Identity and treatment contract: Manifest joins fail closed on mismatches across prompts, rails, checkpoints, quantization, tokenizers, revisions, engines, containers, and runtime kernel-selection evidence.Treatment evidence includes the arm label, engine selection log, selection environment, and log digest.

V. LAYER-LEVEL RESULTS: THE ALIBI AT WORK

The integer alibi’s layer-level predictions held across Qwen3-1.7B and 8B: exact accumulators were safe, and identical operands produced bit-identical outputs under power-of-two scales. With real checkpoint scales, differences were limited to rare one-ulp effects in isolated probes.

  • Prediction and verification: 448 layers stayed below the accumulator threshold, with at least 2.8 bits of headroom, marking every tested layer bitwise-safe.The 196-layer 1.7B and 252-layer 8B models both cleared the 224 threshold.
  • Prediction and verification: All captured layers received identical int8 operands under both power-of-two and checkpoint-real scales, using a prefill-like M=512 tile.The decode regime M=1 was not covered by these layer captures.
  • Prediction and verification: 448 layers were bit-identical across CUTLASS and Triton under power-of-two scales, matching the alibi’s prediction for finite normal outputs.No non-finite outputs were recorded, though subnormal outputs were not separately counted.
  • Real-scale behavior: A handful of one-ulp differences appeared among half a million random-operand elements under real scales, while power-of-two outputs remained bit-identical.Smaller isolated shapes, including M=1, showed no differences, but the probe had little power at roughly 5 × 10−6 per-element rates.

VI. BOUNDARY VALIDATION

Boundary tests on the measurement hardware and pinned container confirm the arithmetic conditions underlying the claims. Integer accumulation is exact across tested GPU paths and matrices, while power-of-two commutation holds for normal values and signed zeros but fails for subnormals.

  • Arithmetic boundary: The 224 boundary behaves exactly as IEEE 754 prescribes: ±(224−1) and ±224 round-trip int→float32 exactly, while ±(224+1) does not and ±(224+2) does.These tests were run on the measurement hardware in the pinned container with an environment-stamped record.
  • Integer accumulation: Integer accumulation is exact against an INT64 reference for random and all-extreme matrices on torch._int_mm and fp64 emulation.The tests include activations at −128, product bound 16256, and K up to 32,768.
  • Power-of-two commutation: Power-of-two commutation holds bitwise for finite normal values and both signed zeros across all tested exponents, but fails in the subnormal range.This matches the conditional lemma described in the passage.

VII. DIVERGENCE STRUCTURE VERSUS REDUCTION DEPTH

INT8 divergence shows no detectable accumulation with reduction depth because exact accumulation leaves only fixed epilogue roundings. Across K = 512 to 32,768, INT8 differences remain rare and within one bf16 spacing, unlike FP8’s depth-dependent signature.

  • Mechanistic prediction: INT8 should avoid reduction-order rounding with K because each element receives a fixed number of epilogue roundings, although prevalence need not remain constant.The prediction is the absence of an accumulation mechanism, not constant prevalence.
  • Prevalence versus K: 18 differing INT8 elements appeared across K = 512 to 32,768, totaling 1.9 to 7.6 parts per million.Counts were one to four per grid point out of 524,288 in a single unreplicated sweep; χ2 = 3.8 with six degrees of freedom was consistent with a constant-rate model.
  • Magnitude versus K: ≤7.4 × 10−3 was the INT8 maximum relative difference at every K, with per-layer verification measuring exactly one bf16 ulp.The median relative difference among differing elements was near one bf16 spacing in both formats, while FP8’s tail grows.

VIII. FROM ULPS TO TOKENS

One-ulp layer differences transfer to token decisions: the kernel arms agree on no sequence, while teacher-forced flips occur mainly at small logit margins. Margin alone predicts flip risk strongly, but position-exact transfer fails across replay and free-running generation.

  • Sequence-level transfer: 0 sequences agree under real scales at either model size, while each kernel arm reproduces itself exactly across cold restarts.Adding SGLang yields three mutually distinct deterministic functions.
  • Margin dependence: ROC-AUC 0.942 shows that logit margin ranks flip risk strongly across 16,384 teacher-forced positions.A margin-binned calibration achieves held-out Brier 0.0352 versus a 0.0440 constant-prevalence baseline.
  • Teacher-forced transfer: 4.7% of teacher-forced positions flip across identical contexts, and flips concentrate almost entirely at small logit margins.The analysis removes autoregressive cascade by replaying identical contexts.
  • Limits of transfer: Position-exact prediction fails because replay evaluates all positions in one prefill pass, whereas generation proceeds incrementally with different noise draws.The transfer claim therefore holds at the distribution level, not per position.

IX. THE POW2 INTERVENTION END TO END

Rewriting every weight scale to the nearest power of two restores bitwise end-to-end agreement while leaving weights, activation quantization, prompts, and kernel arms unchanged. The probe supports localization of divergence to real-scale behavior, but is a conformance instrument rather than a deployment recipe and does not establish engine interchangeability or production effects.

  • Counterfactual intervention: 8/8 sequences at 1.7B and 16/16 at 8B regain bitwise agreement after all weight scales are rewritten to the nearest power of two.Weights, activation quantization, prompts, and both kernel arms remain untouched.
  • Scope and limitations: The probe is a conformance instrument, not a deployment recipe, because power-of-two scale rounding perturbs the model.The study did not measure production accuracy, calibration, or throughput consequences.
  • Scope and limitations: The intervention repairs kernel interchangeability, not engine interchangeability: SGLang still disagrees with the vLLM arms under the probe.The engines differ in attention, sampling, and other components beyond the linear kernels.
  • Scope and limitations: The end-to-end probe used one engine-process configuration, with manifests recording kernel-selection evidence.This configuration made kernel-selection evidence capturable.

X. DISCUSSION · XI. LIMITATIONS · XII. CONCLUSION

The paper frames kernel interchangeability as a conformance question: exact integer accumulation makes reproducibility specifiable and testable, while observed divergence remains an auditability risk rather than an automatic quality failure. Its evidence is limited to the tested configurations, and the proposed probe and procedure require further validation and replication.

  • X. DISCUSSION: An artifact-backed conformance procedure combines seven checks with falsifiable outcomes and diagnoses indicating what a violation first implicates.The scripts instantiate the checks for one kernel pair and require adaptation for another.
  • X. DISCUSSION: Exact, order-free accumulation confines implementation freedom to per-element epilogue roundings, unlike floating accumulation whose divergence grows with reduction depth.Format choice determines which reproducibility contract is available: integer paths permit a specifiable, testable near-bitwise contract.
  • X. DISCUSSION: Token-level disagreement is a reproducibility and auditability risk, not automatically a quality problem, and the study makes no claim of task-quality degradation.Margin analysis quantifies divergence risk rather than text quality.
  • XI. LIMITATIONS: The causal evidence covers one sm_89 GPU, one model family at two sizes, one engine version, and two INT8 implementations plus one FP8 pair and one cross-engine comparison.These results do not license claims across GPU generations, model families, or future kernels.
  • XI. LIMITATIONS: The 2.9% residual remains unresolved, the probe lacks accuracy, calibration, and performance validation, and whole-model inference has no bitwise guarantee beyond the integer GEMM stage.Attention, normalization, rotary embeddings, and the unquantized head remain outside the alibi’s jurisdiction; future work includes broader replication and four-bit formats.
  • XII. CONCLUSION: 188 of 196 layers differ in at least one output element, each differing finite element by at most one bfloat16 ulp, and no greedy sequence matches.The integer alibi attributes the mechanism to per-layer representability and epilogue rounding after acquitting the exact reduction stage.
  • XII. CONCLUSION: Kernel choice is outcome-relevant and made at load time, so practitioners should record which implementation served each result and condition evaluation numbers on it.Table IV provides the conformance check before substituting one implementation for another.
  • XII. CONCLUSION: Bitwise agreement appeared only under the unvalidated probe, whereas margin analysis bounds how often differences reach a decision boundary when reproducibility is not required.The probe has not been evaluated as a deployment configuration.

ARTIFACT STATEMENT

The artifact package preserves the study’s preregistration, measurements, implementation, tests, and analysis code in a version-controlled repository. It also records provenance through SHA-256 digests, per-run manifests, and separate plan-and-code pins for final validation iterations.

  • Artifact contents: The repository contains the pre-registration, per-layer prediction lists, measurement artifacts, identity-contract implementation, regression tests, and analysis code.The pre-registration was locked before backend-comparison measurements and received nine append-only amendments.
  • Validation artifacts: 72 regression checks and a clean-container record accompany the identity-contract implementation.These checks are maintained with the implementation in the version-controlled repository.
  • Provenance: SHA-256 digests, per-run manifests with kernel-selection evidence, and separate plan-and-code pins document artifact and validation provenance.The final validation iterations used separate plan-and-code pins.
Loading 2608.13756v1…