PaperlayerView exact revision on arXiv

Source-linked AI summary

LoopQuant: Accurate and Efficient Post-Training Quantization for Looped Transformers

Zhixiong Zhao, Zukang Xu, Xing Hu, Lifeng Liu, Guangyu Sun, Dawei Yang

arXiv:2610.16368v1cs.LGcs.AI

TL;DR

LoopLMs reuse Transformer modules across evolving recurrent states, creating quantization challenges that feed-forward PTQ methods do not explicitly address. LoopQuant combines cross-loop distribution canonicalization, loop-wise precision allocation, and trajectory-aware calibration; across LoopLMs, it outperforms low-bit PTQ methods, with W4A4 matching or surpassing W4A8 baselines.

  • Problem

    Existing PTQ methods largely target feed-forward Transformers and do not jointly address LoopLM cross-loop activation dynamics, heterogeneous loop sensitivity, and recursive error propagation.

    Source: §1 · pp. 1–2

  • Method

    LoopQuant combines shared cross-loop distribution canonicalization, interaction-aware loop-wise activation precision allocation under a fixed bit budget, and task calibration along the low-bit recursive trajectory.

    Source: §1 · p. 2

  • Results

    Across LoopLMs, LoopQuant consistently outperforms low-bit PTQ methods, with W4A4 matching or surpassing W4A8 baselines.

    Source: §1 · p. 3, §6 · p. 10

  • Takeaways & Limitations

    LoopQuant offers accurate, deployment-friendly low-bit LoopLM inference with memory savings, decoding speedup, and negligible storage overhead.

    Source: §6 · p. 10

    The experiments assess predictive performance and system efficiency, not safety, fairness, privacy, or robustness guarantees.

    Source: ETHICS STATEMENT · p. 11

Abstract

from arXiv · show

Looped Language Models (LoopLMs) improve parameter efficiency by recursively reusing shared Transformer modules, yet their repeated computation over evolving recurrent states poses new challenges for low-bit deployment. Existing post-training quantization (PTQ) methods, largely designed for feed-forward Transformers, fail to account for cross-loop activation dynamics, heterogeneous loop sensitivity, and recursive error propagation, leading to substantial performance degradation at low precision. To address these challenges, we propose LoopQuant, a loop-aware PTQ framework for accurate and efficient low-bit quantization of LoopLMs. LoopQuant is built on three core components: 1) Cross-Loop Distribution Canonicalization (CLDC) learns a shared quantization-friendly representation across recurrent states while preserving parameter sharing; 2) Interaction-Aware Loop-wise Precision Allocation (ILPA) allocates activation precision under a fixed bit budget by modeling task-level loop sensitivity and cross-loop interactions; and 3) Loop-Conditioned Task Calibration (LCTC) further aligns quantized inference with the low-bit recursive trajectory through lightweight task-space calibration. Extensive experiments across LoopLM architectures demonstrate that LoopQuant consistently outperforms existing low-bit PTQ methods. On Ouro-2.6B, LoopQuant under W4A4 surpasses all W4A8 baselines in performance, while reducing model memory by 65.3% with only 3.4% additional storage overhead over naive INT4 and achieving a 1.7 times decoding speedup over BF16. These results demonstrate a strong balance among accuracy, compression, and inference efficiency, providing a practical and deployment-friendly solution for low-bit LoopLM inference. The code is available at [https://github.com/Kishon-zzx/LoopQuant].

1 INTRODUCTION

LoopLMs reduce parameter storage by reusing Transformer modules, but their evolving recurrent states make low-bit quantization difficult. LoopQuant addresses this with three loop-aware components and achieves higher accuracy than existing PTQ baselines, including at W4A4 versus W4A8.

1 INTRODUCTION

  • LoopLMs reuse Transformer modules across loops to increase effective computation depth under a nearly fixed parameter budget, but repeated access makes low-bit quantization important for deployment.Quantization must preserve recursive reasoning while reducing computation and data movement costs.Source: §1 · p. 1
  • Existing PTQ methods largely target feed-forward Transformers, while LoopLM quantization must handle changing cross-loop activations, loop-dependent sensitivity, and recursive error propagation.Applying existing methods directly can substantially degrade performance at low bit-widths.Source: §1 · p. 2
  • LoopQuant’s three components address cross-loop activation dynamics, heterogeneous loop sensitivity, and the mismatch between local quantization objectives and recursive task behavior.CLDC canonicalizes distributions, ILPA allocates precision, and LCTC calibrates along the low-bit recursive trajectory.Source: §1 · pp. 2–3
  • Across LoopLMs, LoopQuant achieves a superior accuracy–efficiency trade-off, with W4A4 outperforming existing PTQ baselines at W4A8.The paper also presents the implementation as, to its authors’ knowledge, the first open-source quantization framework for LoopLMs.Source: Figure 1 · p. 2, §1 · p. 3

2 RELATED WORK

Prior PTQ methods develop transformations and allocation strategies for Transformer quantization, while LoopLM research explores recursively shared modules and emerging quantization approaches. The paper identifies evolving loop dynamics, heterogeneous sensitivity, and recursive error propagation as remaining challenges.

2.1 POST-TRAINING QUANTIZATION FOR LARGE LANGUAGE MODELS

  • Recent PTQ methods address activation outliers with channel scaling, rotations, learned transformations, and distribution-aware calibration, but typically rely on local activation distributions.The cited approaches include SmoothQuant, QuaRot, SpinQuant, FlatQuant (Sun et al., 2024), and DartQuant.Source: §2.1 · p. 3

2.2 LOOPED LANGUAGE MODELS

  • LoopLMs reuse shared Transformer modules to increase effective depth, with architectures such as Huginn, Ouro, and Nanbeige4.2-3B applying recursive computation.LoopQ (Fang et al., 2026) studies cross-loop distribution shifts and recursive quantization errors, while loop-wise sensitivity and trajectory error propagation remain challenges.Source: §2.2 · p. 3

3 PRELIMINARIES

LoopLMs increase computation depth by recursively applying shared Transformer layers to evolving hidden states. The shared parameters remain fixed across loops even as the recurrent states change.

3 PRELIMINARIES

  • A LoopLM recursively applies the same Transformer module to the hidden state for multiple loops, obtaining its final output from the last-loop state.The module has L layers and is applied for T loops.Source: §3 · p. 3
  • Sharing parameters across T loops gives approximately LT effective depth while storing only L layers, although the shared layers receive different recurrent-state activations.Figure 2 presents LoopQuant as three stages: CLDC canonicalizes cross-loop activation distributions, ILPA allocates activation precision, and LCTC aligns quantized models with recursive task behavior.Source: §3 · p. 4, Figure 2 · p. 4

4 METHOD

LoopQuant formulates recursive quantization error in terms of local perturbations, loop sensitivity and interactions, and trajectory-dependent error. Its three stages address these sources through shared distribution canonicalization, loop-wise precision allocation, and task-aligned calibration.

4.1 QUANTIZATION UNDER RECURSIVE COMPUTATION

  • The recursive quantization formulation separates loop-dependent local errors, task sensitivity and cross-loop interactions, and trajectory-dependent errors.It connects these error sources to CLDC, ILPA, and LCTC, respectively.Source: §4.1 · pp. 4–5

4.2 CROSS-LOOP DISTRIBUTION CANONICALIZATION (CLDC)

  • CLDC learns a shared quantization-friendly geometry across recurrent states, applying transformations to every loop while preserving full-precision computation.It minimizes a smooth worst-loop objective because distortion at one recurrent position can propagate forward, and synchronizes cross-loop anomaly profiles.Source: §4.2 · p. 5

4.3 INTERACTION-AWARE LOOP-WISE PRECISION ALLOCATION (ILPA)

  • ILPA assigns loop-wise activation precision from complete recursive trajectories under a fixed bit budget, estimating task-level loop sensitivity and pairwise recursive interactions.It uses exact enumeration for small T and a greedy solver for larger models; the resulting assignment remains fixed during inference.Source: §4.3 · p. 6

4.4 LOOP-CONDITIONED TASK CALIBRATION (LCTC)

  • After CLDC and ILPA, preceding low-bit computation can shift states so that small local quantization error does not ensure robust recursive-trajectory behavior.LCTC therefore targets trajectory-dependent error rather than retraining the quantized model.Source: §4.4 · pp. 6–7
  • LCTC calibrates loop-dependent quantization responses while keeping the quantization structure and precision assignment fixed.It optimizes lightweight loop-indexed clipping, gain, and bias parameters to align quantization defects with task behavior, including trajectory-dependent effects.Source: §4.4 · pp. 6–7

5 EXPERIMENTS

The experiments evaluate LoopQuant across LoopLM architectures, benchmark tasks, low-bit PTQ baselines, component ablations, and deployment-efficiency measures. Results report stronger low-bit performance alongside memory savings and faster decoding.

5.1 EXPERIMENT SETUP

  • The evaluation covers Ouro, Huginn, and Nanbeige4.2-3B on language-modeling, zero-shot, reasoning, and generation benchmarks, comparing LoopQuant with representative low-bit PTQ methods and BF16.The methods include SmoothQuant, QuaRot, SpinQuant, FlatQuant, DartQuant, and LoopQ.Source: §5.1 · p. 7

5.2 MAIN RESULTS

  • 70.20% average zero-shot accuracy on Ouro-2.6B under W4A6 beats the strongest baseline by 1.90 percentage points, while WikiText2 PPL falls from 9.09 to 8.04.The comparison is against lower-activation-precision W4A8 baselines; Table 1 presents comparisons across LoopLMs.Source: §5.2 · pp. 7–8, Table 1 · p. 8

5.3 ABLATION STUDIES

  • On Nanbeige4.2-3B under W4A6, CLDC reduces WikiText2 PPL from 47.39 to 28.41, and adding ILPA lowers WikiText2 and C4 PPL to 27.05 and 34.17.The ablation reports improvements from each component, with CLDC regularizing activation distributions and ILPA allocating precision by loop-specific sensitivity.Source: §5.3 · p. 8, Table 3 · p. 9
  • The ablation also compares loop-wise bit-allocation strategies, but the supplied results do not report their outcomes.Table 4 is identified as a comparison of loop-wise bit-allocation strategies.Source: §5.3 · p. 9, Table 4 · p. 9

5.4 EFFICIENCY ANALYSIS OF LOOPQUANT

  • LoopQuant reduces overall model memory by 53.7–65.3% versus BF16 and adds at most 3.4% memory overhead versus naive INT4 RTN.Quantized linear layers alone achieve approximately 74% memory savings; Figure 3 compares memory consumption and C4 PPL.Source: §5.4 · p. 9, Figure 3 · p. 9
  • LoopQuant improves decoding throughput by 1.5–1.7× over BF16 across three models, while keeping zero-shot accuracy close to BF16.The comparison uses batch size 4, prefills 4,096 tokens, then generates 1,024 tokens; naive INT4 is slightly faster but has severe accuracy degradation.Source: Figure 3 · p. 9, §5.4 · p. 10

6 CONCLUSION

The conclusion presents LoopQuant as a loop-aware PTQ framework and reports accuracy, memory, and decoding benefits across LoopLMs. It also cautions that the evaluation does not establish safety or other broader deployment guarantees.

6 CONCLUSION

  • LoopQuant combines shared cross-loop representation learning, loop-wise activation precision allocation, and calibration along the low-bit recursive trajectory.The conclusion characterizes the framework as addressing cross-loop activation dynamics, heterogeneous loop sensitivity, and misalignment between local objectives and recursive task behavior.Source: §6 · p. 10
  • Across LoopLMs, W4A4 matches or surpasses W4A8 baselines while preserving reasoning capability, with memory savings and decoding speedup at negligible storage overhead.The conclusion describes the resulting balance as supporting accurate, deployment-friendly low-bit inference.Source: §6 · p. 10
  • Generative AI tools were used only for language-related manuscript assistance, and authors retain responsibility for the technical content and reported results.The authors say AI-supported edits were manually inspected and revised when necessary.Source: AI USE STATEMENT · p. 11
  • The experiments assess predictive performance and system efficiency, not safety, fairness, privacy, or robustness guarantees.The authors recommend application-specific evaluation and safeguards before real-world deployment.Source: ETHICS STATEMENT · p. 11

APPENDIX OVERVIEW

The appendix organizes material into pseudocode, derivations and implementation details, and additional experimental details.

APPENDIX OVERVIEW

  • Section A.1 contains LoopQuant pseudocode, including its overall workflow and the CLDC, ILPA, and LCTC components.The listed subsections are A.1.1 through A.1.4.Source: APPENDIX OVERVIEW · p. 14
  • Section A.2 provides detailed derivations and implementation material, including canonical geometry, loop sensitivity, recursive interaction modeling, and calibration.Its listed topics include precision allocation and task-space defect transport.Source: APPENDIX OVERVIEW · p. 14
  • Section A.3 collects additional experimental details, including storage overhead, main results, calibration-data ablations, and further component analyses.The listed topics also include comparison with LoopQ and quantization error during reasoning.Source: APPENDIX OVERVIEW · p. 14

A.1 PSEUDOCODE OF LOOPQUANT

The pseudocode section presents LoopQuant as a three-stage quantization procedure for recursive states, loop-wise precision, and task behavior.

A.1 PSEUDOCODE OF LOOPQUANT

  • CLDC learns a shared quantization-friendly representation across recursive states.It is the first stage of the complete quantization procedure.Source: §A.1 · p. 14
  • ILPA determines loop-wise activation precision under a fixed bit-width budget.It follows CLDC in the complete procedure.Source: §A.1 · p. 14
  • LCTC aligns the quantized model with recursive task behavior through task-space calibration.It is the final stage of the complete quantization procedure.Source: §A.1 · p. 14

A.1.1 OVERALL WORKFLOW OF LOOPQUANT

LoopQuant turns a pretrained LoopLM and calibration data into a quantized model through three stages: CLDC, ILPA, and LCTC.

A.1.1 OVERALL WORKFLOW OF LOOPQUANT

  • LoopQuant applies CLDC, then budgeted loop-wise activation precision allocation with ILPA, and finally task-aligned calibration with LCTC.The pipeline takes a pretrained LoopLM, calibration dataset, and target activation bit-width budget as input.Source: §A.1.1 · pp. 14–15
  • ILPA selects a loop-wise precision configuration under the bit-width budget; LCTC then optimizes task-space calibration with the weights, transformations, and assignments frozen.For small T, the algorithm uses exact enumeration; otherwise, it uses greedy allocation.Source: §A.1.1 · pp. 14–15
  • CLDC learns a shared canonical transformation from loop-wise calibration activations, applies it to the weights, and quantizes the canonicalized weights with GPTQ.The algorithm specifies optimizing cross-loop reconstruction robustness, anomaly synchronization, and group-aware criteria.Source: §A.1.1 · p. 15

A.1.2 CROSS-LOOP DISTRIBUTION CANONICALIZATION (CLDC)

CLDC creates a shared quantization-friendly representation across recursive states while preserving LoopLM parameter sharing.

A.1.2 CROSS-LOOP DISTRIBUTION CANONICALIZATION (CLDC)

  • CLDC learns shared canonical geometry from loop-wise calibration states while preserving the model’s parameter-sharing structure.Its transformation is shared across all loops and fixed for the later ILPA and LCTC stages.Source: §A.1.2 · p. 15
  • Robust synchronization reduces cross-loop activation-distribution mismatch in the shared representation.The procedure aligns distributions across recursive states.Source: §A.1.2 · p. 15
  • Group-aware channel reordering improves compatibility with group-wise quantization.Source: §A.1.2 · p. 15

A.1.3 INTERACTION-AWARE LOOP-WISE PRECISION ALLOCATION (ILPA)

ILPA allocates loop-wise activation precision under a fixed bit-width budget by evaluating task impact along the recursive trajectory and accounting for loop sensitivity and cross-loop interaction.

A.1.3 INTERACTION-AWARE LOOP-WISE PRECISION ALLOCATION (ILPA)

  • ILPA estimates loop-wise sensitivity and cross-loop interaction to inform its precision allocation.Source: §A.1.3 · p. 15
  • The allocation procedure is budget-aware and uses task impact from precision configurations evaluated along the recursive trajectory.Source: §A.1.3 · p. 15
  • ILPA evaluates precision configurations on the complete recursive trajectory before allocating activation precision under the target bit-width budget.Its resulting configuration remains fixed during inference and is used by LCTC.Source: §A.1.3 · p. 15

1 Initialize the shared transformation: Pcont

This section describes LoopQuant’s shared quantization geometry, loop-wise precision allocation, task-space calibration, and associated optimization and deployment-storage procedures. It also reports supplemental evaluations of LoopQuant across four LoopLMs, examining storage overhead, comparisons with LoopQ, calibration behavior, component contributions, recursive error, and CLDC activation transformations.

1 Initialize the shared transformation: Pcont

  • Initialization jointly optimizes Dm and Rm over all loop calibration states, then synchronizes robust errors and anomalies, orders channels by worst-case range, and canonicalizes the result.Source: §1 · p. 16

A.1.4 LOOP-CONDITIONED TASK CALIBRATION (LCTC)

  • LCTC performs final task-level calibration while keeping quantization structure and precision assignments fixed, optimizing lightweight loop-conditioned parameters for task-space and trajectory-dependent defects.Its objective jointly considers teacher alignment, language-modeling loss, clean-state task defects, and trajectory-dependent defects; only static calibration metadata are retained after calibration.Source: §A.1.4 · p. 16

A.2.1 ORTHOGONAL CANONICAL GEOMETRY PARAMETERIZATION

  • CLDC uses one shared channel scaling and orthogonal rotation, parameterized with structured block corrections, to improve quantization geometry while preserving the original full-precision computation.The Cayley parameterization keeps the learned rotation strictly orthogonal during optimization, and the same transformation is applied across recurrent states with inverse weight compensation.Source: §A.2.1 · pp. 16, 18

A.2.2 ROBUST CROSS-LOOP SYNCHRONIZATION OBJECTIVE

  • CLDC combines smooth worst-loop reconstruction control with cross-loop anomaly synchronization, yielding consensus geometry for the full recursive trajectory rather than an individual-loop optimum.A scale regularizer prevents activation scaling from simply shifting quantization difficulty to weights.Source: §A.2.2 · pp. 19–20

A.2.3 GROUP-AWARE CHANNEL REORDERING

  • Group-aware reordering addresses within-group dynamic-range mismatch by optimizing a hard channel assignment, initialized by worst-case range sorting and refined with accepted cross-group swaps.The resulting permutation is shared across recurrent states and adds no continuous deployment parameters.Source: §A.2.3 · pp. 20–22

A.2.4 SHAPLEY-STYLE LOOP SENSITIVITY ESTIMATION

  • ILPA estimates each loop’s activation-precision sensitivity from task-level degradation across recursive precision contexts, averaging marginal effects over loop orderings.Its task-level cost is calibration-set token-level negative log-likelihood, and sensitivity reflects the degradation from lowering a loop’s precision across different preceding low-precision loops.Source: §A.2.4 · p. 22

A.2.4 SHAPLEY-STYLE LOOP SENSITIVITY ESTIMATION; A.2.5 RECURSIVE INTERACTION MODELING

  • ILPA models loop precision effects with marginal sensitivity and pairwise non-additive interactions, making its allocation objective a discrete second-order surrogate of trajectory-level error.It estimates context-aware sensitivity by sampling permutations and using Monte Carlo approximation rather than enumerating all permutations.Source: §A.2.4 · pp. 22–23, §A.2.5 · p. 23

A.2.6 EXACT AND GREEDY PRECISION ALLOCATION

  • Under a fixed activation-bit budget, ILPA uses exact search for small loop counts and a greedy solver for larger ones, prioritizing loops with lower sensitivity and weaker interactions for low precision.The greedy solver uses at most T selection steps and avoids exhaustive search.Source: §A.2.6 · p. 24

A.2.7 LOOP-CONDITIONED CALIBRATION PARAMETERIZATION

  • LCTC adjusts loop-conditioned clipping, gain, and bias parameters for each module while leaving quantized weights and backbone parameters unchanged.The parameters are static lookups indexed by physical module and loop, so they require no token-wise state cache or online optimization.Source: §A.2.7 · p. 25

A.2.8 TASK-SPACE DEFECT TRANSPORT

  • LCTC transports each loop’s quantization defect into task space and uses a secant along inherited trajectory drift to capture how prior errors alter the current loop’s response.The trajectory-dependent term is zero for the first loop because no previous quantization perturbation exists.Source: §A.2.8 · pp. 25–26

A.2.9 FISHER-WEIGHTED TRAJECTORY CALIBRATION

  • LCTC uses Fisher-weighted task-space defects to approximate prediction-distribution shifts, with separate losses for direct clean-state defects and sensitivity to trajectory drift.The Fisher energy is a second-order approximation of task-level distribution shift, and the trajectory loss constrains sensitivity to inherited drift.Source: §A.2.9 · pp. 26–27

A.3.1 CLDC OPTIMIZATION DETAILS

  • CLDC jointly optimizes loop-shared diagonal scaling and block-diagonal orthogonal transformations, using 32-channel rotation blocks and 80 Adam steps at a learning rate of 1 × 10^-2.The transformations start as identity, and gradient norms are clipped to 1.0.Source: §A.3.1 · p. 28

A.3.2 LCTC OPTIMIZATION DETAILS

  • LCTC samples trajectory interpolation α from {0.5, 1.0, 1.5}, searches clipping ratios from 0.50 to 1.00, and disables the trajectory term for the first loop.Calibration uses fixed loss weights λNLL = 0.1, λa = 0.1, λt = 0.03, and λreg = 10^-4.Source: §A.3.2 · p. 28

A.3.3 DEPLOYMENT STORAGE OVERHEAD

  • Deployment retains CLDC transformations, ILPA’s final precision assignment, and LCTC calibration parameters, while discarding calibration-time statistics, trajectory caches, and optimization states.The additional storage is accounted for as persistent metadata alongside the W4 model storage.Source: §A.3.3 · pp. 28–29
  • W4A4 and W4A6 use the same retained tensor shapes and therefore essentially identical deployment storage, with overhead scaling by transformed channel dimensions, recurrent precision assignments, and calibrated groups.Retained quantities are static model metadata and require no token-dependent cache or recurrent-history storage during inference.Source: §A.3.3 · p. 29

A.3.6 ABLATION ON CALIBRATION DATA

  • Perplexity varies by less than 0.05 across calibration-set sizes and by at most approximately 0.02 across sampling seeds; calibration-corpus choice has moderate impact on zero-shot accuracy but favors perplexity in its matching domain.WikiText2 calibration lowers WikiText2 perplexity versus C4 calibration from 10.57 to 9.86 on Ouro-1.4B and 9.41 to 8.27 on Ouro-2.6B; C4 calibration lowers C4 perplexity from 14.00 to 11.86 and 12.24 to 10.15, respectively.Source: §A.3.6 · pp. 30–31, Figure 4 · p. 31, Table 7 · p. 40

A.3.9 LAYER-WISE QUANTIZATION ERROR DURING REASONING

  • On GSM8K under W4A4, LoopQuant suppresses representation-error accumulation and recurrent-boundary spikes, while QuaRot’s relative error exceeds 1.2 after the first boundary on Ouro-1.4B-thinking.Several baselines’ peaks approach 0.6–1.0, whereas LoopQuant keeps corresponding peaks substantially lower and returns rapidly to a lower-error trajectory; repeated application of shared parameters can amplify inherited quantization perturbations.Source: Figure 5 · p. 32, §A.3.9 · pp. 32–33

A.3.10 ACTIVATION DISTRIBUTION TRANSFORMATION BY CLDC

  • CLDC transforms channel-wise outlier-heavy activations into more balanced, compact distributions across recurrent loops, including attention and MLP projections.Raw down-projection inputs include first-loop channels above magnitude 20 despite central percentiles near zero; CLDC reduces these outliers to a compact sub-unit range.Source: §A.3.10 · p. 33, Figure 6 · p. 35, Figure 7 · p. 36

A.3.4 DETAILED MAIN RESULTS

  • Under W4A4, LoopQuant beats the strongest baseline by 5.91, 5.31, 5.41, and 3.83 percentage points across four LoopLMs and leads on all seven tasks per model.Under W4A6, it also exceeds the strongest W4A8 competitor by 1.93, 1.90, 0.64, and 1.36 percentage points on Ouro-1.4B, Ouro-2.6B, Nanbeige4.2-3B, and Huginn-0125, respectively.Source: §A.3.4 · pp. 29–30, Table 5 · p. 39

A.3.5 COMPARISON WITH LOOPQ

  • On Ouro-2.6B under W4A4, LoopQuant improves LoopQ’s four zero-shot metrics from 68.60/62.27/56.80/50.34 to 75.07/66.75/65.88/56.09 and WikiText2 perplexity from 15.89 to 8.27.LoopQ is the first work on recursive Transformer quantization (Fang et al. (2026)); because its implementation and complete configurations are unavailable, the comparison covers only overlapping models and metrics and is not a controlled reproduction.Source: §A.3.5 · p. 30, Table 6 · p. 40

A.3.7 EFFECTIVENESS OF ILPA WITH DIFFERENT QUANTIZATION BACKBONES

  • ILPA reduces QuaRot WikiText2 perplexity from 14.02 to 13.59 and raises average accuracy from 46.28% to 47.35% under the same 6-bit average activation budget.It also raises SpinQuant average accuracy from 45.97% to 47.04%; with CLDC and LCTC, LoopQuant reaches 12.47 WikiText2 perplexity and 49.09% average accuracy on Huginn-0125 (T = 6).Source: §A.3.7 · p. 31, Table 8 · p. 40

A.3.8 ABLATION ON LCTC CALIBRATION PARAMETERS

  • With CLDC and ILPA fixed, full LCTC reduces WikiText2 and C4 perplexity by 1.21 and 1.98 versus removing LCTC, using 1.46 MB additional storage (about 0.08% of the W4 base-model footprint).Clipping, gain, and bias progressively improve perplexity from 27.05/34.17 without LCTC to 26.48/33.38, 26.09/32.74, and finally 25.84/32.19 on WikiText2/C4.Source: §A.3.8 · p. 31, Table 9 · p. 40

Related papers

Loading 2610.16368v1…