PaperlayerView exact revision on arXiv

Source-linked AI summary

Security-Enhanced Seed-Based Weight Quantization for Large Language Models

Qiuyu Ren, Sudipta Paria, Aritra Dasgupta, Swarup Bhunia

arXiv:2609.38477v1cs.CRcs.AIcs.LG

TL;DR

Seed-based compression reduces the storage and bandwidth costs of large language models, but existing methods do not account for non-uniform weight sensitivity. Seed-Q allocates representation budgets according to sensitivity and lets the decoder reproduce that allocation without per-block metadata. The paper reports reduced compression-related degradation and storage, amplified bit-flip effects, and modest ASIC overhead.

  • Problem

    Seed-based compression does not account for non-uniform weight sensitivity, despite the substantial memory and bandwidth demands of large language models.

    Source: §1 · pp. 1–2

  • Method

    Seed-Q assigns larger representation budgets to sensitive regions and deterministically reconstructs each block’s allocation without per-block metadata.

    Source: §1 · p. 2

  • Results

    Seed-Q reports up to 42% lower perplexity degradation and 55% lower accuracy degradation, alongside up to 6% lower bits per weight.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Seed-Q reduces storage and memory-access energy in bandwidth-bound inference, while amplified bit-flip effects improve detectability and ASIC overhead remains modest.

    Source: §1 · p. 2

    Robust protection must cover the compressed payload and all inputs that determine its interpretation, and proposed countermeasures require separate evaluation of coverage and overhead.

    Source: §4.3.2 · p. 9

Abstract

from arXiv · show

Large language models (LLMs) incur substantial storage, memory-bandwidth and energy costs, motivating compact weight representations. Existing seed-based compression methods reconstruct weights from compact pseudo-random representations but do not explicitly account for the non-uniform sensitivity of model weights. We introduce Seed-Q, a security-enhanced sensitivity-aware seed-based weight compression framework that uses lightweight Linear Feedback Shift Register (LFSR)-based weight generation with non-uniform bit allocation. Our approach assigns larger representation budgets to sensitive weights while aggressively compressing less sensitive regions. Importantly, this non-uniform allocation requires no side-information: the decoder deterministically reconstructs the bit-allocation schedule, with no rung depending on the decoded weights, eliminating the need to store per-block metadata or use calibration data while preserving the baseline coding rate. Experiments across diverse LLMs show that Seed-Q matches 4-bit perplexity of SeedLM with fewer bits, while at the same 4 bits/weight it reduces both perplexity degradation and zero-shot accuracy loss relative to SeedLM. We also show that Seed-Q simultaneously achieves high security against bit-flip attacks on model parameters, as bit corruption affects multiple reconstructed weights, greatly amplifying its impact and making it easier to detect. We further implement Seed-Q in an ASIC-based accelerator and demonstrate modest hardware overhead compared to prior seed-based approaches.

1 INTRODUCTION

LLMs make inference costly through weight storage and repeated memory transfers, while existing seed-based methods do not allocate representation budgets according to model sensitivity. Seed-Q introduces decoder-reproducible sensitivity-aware allocation and reports improved compression quality, lower storage use, and amplified bit-flip effects.

1 INTRODUCTION

  • For memory-bound inference, fewer weight bits reduce storage and off-chip traffic, lowering memory-access energy and bandwidth demand.The passage contrasts DRAM access energy of 1–2 nJ with roughly 10 pJ for an internal cache access or functional operation.Source: §1 · p. 1
  • SeedLM uses uniform budgets, while S-Quant adapts to block reconstructability; neither directly allocates according to model sensitivity.Similar reconstruction errors can have different effects on model quality depending on their location.Source: §1 · p. 2
  • Seed-Q assigns larger representation budgets to sensitive regions and compresses less-sensitive regions more aggressively without per-block allocation metadata.The decoder deterministically derives each block’s representation from stored gains and compact allocation tables.Source: §1 · p. 2
  • Up to 42% lower perplexity degradation and 55% lower accuracy degradation, with up to 6% lower bits per weight, versus existing seed-based configurations.Seed-Q also preserves lightweight LFSR-based reconstruction with modest ASIC overhead.Source: §1 · p. 2
  • Seed-Q studies bit-flip effects on reconstructed weights and model performance, with amplified corruption proposed as making attacks easier to detect.The contribution describes quantifiable robustness against bit-flip attacks by making an attack easily detectable.Source: §1 · p. 2

2 RELATED WORK

Related work spans seed-based generation, sensitivity-aware compression, and bit-level integrity. Seed-Q applies sensitivity allocation to seed-generated weights and measures corruption footprints, while noting that attack detection itself is not evaluated.

2 RELATED WORK

  • SeedLM uses compact seeds and coefficients with an LFSR-generated basis, while S-Quant varies generated bases according to block reconstruction characteristics.Earlier random-basis approaches use larger bases and are not designed for lightweight on-the-fly LLM inference.Source: §2 · p. 3
  • HAWQ-V2 and SqueezeLLM use sensitivity-aware precision or quantization, whereas Seed-Q applies non-uniform allocation to seed-generated weights.Seed-Q derives block importance from checkpoint information rather than token-level activation calibration, and its schedule needs no per-block allocation map.Source: §2 · p. 3
  • Prior work reports targeted bit flips degrading model accuracy; Seed-Q measures multi-weight seed-corruption effects but does not evaluate attack detection.The measured fault footprint may motivate a future integrity check.Source: §2 · p. 3

3 METHODOLOGY

Seed-Q retains LFSR-based seed reconstruction while assigning blocks different representation budgets according to importance and rung error. Its decoder reconstructs the allocation without per-block headers from information it already holds.

3.1 SEED-BASED BLOCK CODING

  • Each eight-weight block uses an LFSR seed and quantized coefficients; Seed-Q varies seed length and coefficient count by rung, avoiding the per-block cost of signaling rung choices.A rung is a seed-length and coefficient-count configuration with a fixed rate and quality for a given model.Source: §3.1 · pp. 3–4

3.2 PROBLEM STATEMENT

  • The allocator minimizes block-importance-weighted rung error under a target average rate, using measured uniform-rung excess loss as an empirical loss surrogate.Allocation quantities must be decoder-available without per-block entries, ruling out scores based on a block's own weights or calibration data.Source: §3.2 · p. 4

3.3 DATA-FREE BLOCK IMPORTANCE

  • Checkpoint-derived channel scales and a backward gradient proxy from isotropic logits inform block importance; input channels are sorted before blocking so within-block scales are similar.The method does not read tokens, and the backward proxy omits factors requiring activations.Source: §3.3 · pp. 4–5
  • A floor clips low block-importance estimates at their f-quantile, with f = 0.06 selected on Llama-2-7B and fixed across models.The estimate is least reliable at its low end, where small differences can determine demotions.Source: §3.3 · p. 5

3.4 RATE–DISTORTION ALLOCATION

  • The allocator uses a lower convex hull of measured rate–distortion points; a rung advances when block importance times the hull slope reaches λ, with deterministic tie handling.Bisection on λ meets the rate budget, and tied blocks are demoted in block order until it is met exactly.Source: §3.4 · p. 5

3.5 OUTLIER COLUMNS

  • Four checkpoint-identified outlier columns are stored in FP16 because shared-exponent blocks leak extreme peaks onto near-zero block-mates.Candidate columns are selected using dynamic-range or activation-scale thresholds.Source: §3.5 · p. 5

3.6 DECODING WITHOUT PER-BLOCK SIDE INFORMATION

  • The decoder rebuilds block rungs from gains and tables, reads variable-length blocks without headers, and can decode tensors on demand in forward-pass order.The tables add at most 0.005 bits per weight.Source: §3.6 · pp. 5–6

4 EXPERIMENTAL RESULTS

Across four LLMs, Seed-Q reaches SeedLM’s 4-bit perplexity at lower rates and improves perplexity and zero-shot accuracy at 4 bits/weight. Hardware measurements show trade-offs against prior seed-based designs, while simulated bit flips can affect many reconstructed weights and sharply degrade model performance.

4.1 PERFORMANCE ANALYSIS

  • Mode 1 matches SeedLM’s 4-bit perplexity at 3.77, 3.75, 3.88, and 3.78 bits/weight across the four models, using 3–6% fewer bits.On Llama-2-13B, this configuration uses 0.40 GB of weight storage.Source: §4.1 · p. 6, Table 1 · p. 6
  • At 4.0 bits/weight, Seed-Q lowers SeedLM perplexity from 5.85 to 5.68, 5.18 to 5.06, 7.17 to 6.92, and 6.32 to 5.96.These reductions remove 42%, 38%, 23%, and 32% of SeedLM’s excess loss over FP16, respectively.Source: §4.1 · p. 6, Table 1 · p. 6
  • Mode 2 raises mean zero-shot accuracy by 0.5–1.1 points across all four models, recovering 27–55% of SeedLM’s loss relative to FP16.Mode 1 remains within about one point of SeedLM, comparable to the stated single-task sampling error.Source: §4.1 · p. 6, Table 2 · p. 7
  • In the ablation, perplexity falls steeply through 3.75 bits/weight and flattens beyond 4.0; the floor raises removed excess loss from 32% to 37.5%, while four outlier columns remove 6%.The floor value f = 0.06 is used for every model; adding more outlier columns changes nothing.Source: §4.1 · p. 6, Figure 2 · p. 8

4.2 OVERHEAD ANALYSIS

  • Against the implemented S-Quant variant, Seed-Q uses 52.6% less cell area and 39.6% less estimated power, with a 42.8% shorter register path; versus fixed SeedLM, it adds 30.8% area and 30.5% power.The SeedLM comparison is for fixed configurations; Seed-Q supports runtime seed widths of 8–16 bits and basis counts of 2–6.Source: §4.2 · p. 7, Table 3 · p. 7
  • At 3.77 bits/weight, Seed-Q reduces ideal weight-transfer volume by 76.4% versus FP16, corresponding to a theoretical 4.24× increase in bandwidth-limited weight-delivery throughput.The reported hardware comparisons use 16-bit seeds, k = 3, and 561 cycles per eight-weight block.Source: §4.2 · p. 7

4.3.1 FAULT MODEL AND PROPAGATION

  • For a selected single-bit fault, simulated propagation affects 99.99% of weights in Seed-Q’s Llama-3-8B query projection, versus 43.56% for S-Quant; reconstructed-weight effects reach 16.77 M and 7.31 M weights, respectively.This software fault study excludes physical memory injection and attacks chosen to induce a prediction.Source: Figure 3 · p. 8, §4.3.1 · p. 8

4.3.2 SECURITY IMPLICATIONS OF THE RESULTS

  • After the selected fault, Seed-Q perplexity rises from 6.89 to 108.11 (+1,468.43%), while its accuracy falls from 74% to 57%; the comparison does not rank fault positions.The larger perplexity response depends in part on Seed-Q’s earlier allocation mismatch; accuracy uses a 100-question sample.Source: §4.3.2 · p. 9, Figure 4 · p. 9

5 CONCLUSION

Seed-Q combines sensitivity-aware seed allocation with LFSR-based compression to improve compression and bit-flip protection while retaining simple, low-cost decoding hardware.

5 CONCLUSION

  • Seed-Q improves compression over existing seed-based solutions while providing quantifiable protection against bit-flip attacks.It allocates seed budgets according to weight-sensitivity skew, avoids allocation metadata, and supports diverse model architectures and sizes.Source: §5 · p. 9

A METHOD DETAILS

The appendix details approximations and allocation choices used to estimate block importance and assign representation rates, including a floor and a shared damage curve.

A.1 APPROXIMATIONS BEHIND THE IMPORTANCE

  • The importance proxy assumes isotropic normalized channels, follows only the attention value path, treats MLP gate and up branches independently, and uses diagonal input covariance.Its backward proxy also omits activation-dependent factors and the softmax Jacobian; propagating importance to Wv, Wgate, and Wup lowers Llama-2-7B damage from 0.0412 to 0.0395, so the conservative variant is reported.Source: §A.1 · p. 12

A.2 NORMALIZATION WITHIN TENSORS

  • Between-tensor importance scales are discarded because depth-dependent propagation artifacts make them unreliable; adding calibrated Fisher scale worsens allocation by 5% on Llama-2-7B.Unit-mean normalization keeps ranking within tensors; unit-mean Fisher still removes 27% of SeedLM’s excess loss on Llama-3-8B.Source: §A.2 · p. 12

A.3 THE FLOOR

  • With the column factor alone on Llama-2-7B at 4.0 bits per weight, perplexity varies by at most 0.0002 over f ∈[0.055, 0.08], while damage varies by at most 2% over f ∈[0.05, 0.10].Adding the row factor lowers damage from 0.0428 to 0.0395; the model-specific floor value q is carried in the global table, so the decoder does not compute a quantile.Source: §A.3 · p. 12

A.4 THE DAMAGE CURVE

  • The shared damage curve uses loss-space rung errors and assumes block contributions add; measured damages on training data shift by 0–3.4% on Llama-3-8B.The assumption fails for blocks no rung can represent, which are handled outside the hull; tested model hulls contain four to eight rungs.Source: §A.4 · pp. 12–13

A.5 EVERETT’S THEOREM AND TIES

The allocation procedure selects block rungs under a rate budget, handles discrete ties, and accommodates unrepresentable outlier columns while retaining deterministic decoding.

A.5 EVERETT’S THEOREM AND TIES

  • Bisection adjusts λ to the rate budget, and a tie pass demotes blocks at the threshold to close gaps between attainable rates; the allocation is optimal up to one block’s rate step.The implementation limits blocks to rungs within 1.0 bit per weight below and 1.5 above the uniform rung, a band that does not bind on the grids used.Source: §A.5 · p. 13
  • A small number of sensitive blocks are promoted and paid for by demotions; on Llama-2-7B, promotions concentrate in high-activation residual-stream and MLP channels.Because the data-free importance is less dispersed than true sensitivity on this model, most blocks remain at the uniform rung.Source: §A.5 · p. 13

A.6 OUTLIER COLUMNS

  • Seed-Q stores the first four selected outlier columns in FP16 and overwrites their codec reconstructions, addressing columns that no block rung can represent.The conservative storage cost is 3 × 10−5 bits per weight on Llama-2-7B and below 5 × 10−4 on every tested model.Source: §A.6 · p. 13

A.7 ENCODING AND DECODING

  • The decoder recomputes block lengths from stored gains and tables rather than reconstructed weights, enabling tensor-order-independent decoding and random access between tensors.Effective-rate overhead from tables and outlier data is at most 0.005 bits per weight; encoder and decoder allocations agree when they use the same stored numbers and arithmetic rules.Source: §A.7 · pp. 13–14

A.8 COST

  • Encoder cost is dominated by SeedLM’s exhaustive seed search across every rung, paid once per model; importance estimation and allocation require one checkpoint pass and one comparison per block.On Llama-3-8B, the (16, 3) rung alone took 6.6 hours on four GPUs.Source: §A.8 · p. 14
  • At 4.0 bits per weight, mean coefficient counts are 3.02, 3.06, and 3.00 on Llama-2-13B, Llama-3-8B, and Mistral-7B, respectively, and below 3.13 on Llama-2-7B.The decoder additionally evaluates the ladder, parses variable-length blocks, and writes outlier columns; its reported reconstruction-kernel measurement excludes ladder evaluation and inverse column permutation.Source: §A.8 · p. 15

B.1 TAIL PRECISION OF THE ESTIMATOR

The appendix assesses how the importance ranking captures Fisher-sensitive blocks, how the allocation objective tracks measured loss, and how rung grids vary across models.

B.1 TAIL PRECISION OF THE ESTIMATOR

  • Seed-Q’s top decile captures 80% of the Fisher’s top-decile mass at every budget, despite membership precision of only 16–24%; the Fisher is highly concentrated in a small number of blocks.The Fisher’s top 0.01% of blocks holds 59% of its mass, while the top 1% of Wdown and Wv holds 42% and 24%, respectively.Source: §B.1 · p. 15, Table 4 · p. 15
  • With Fisher importance, the floor neutralizes exactly the region where inversion is worst; at the optimum f ∈ [0.055, 0.08], it changes nothing, with damage 0.0315 at f = 0 and f = 0.06.Source: §B.1 · p. 15

B.2 OBJECTIVE VERSUS MEASURED LOSS

  • The allocation objective can be optimistic when many blocks move: on Qwen2.5-7B, rung interactions omitted by the additive estimate make its prediction optimistic.On the five reported Llama-2-13B and Llama-3-8B allocations, which move 4–27% of blocks, measured loss is 6–24% below prediction; the authors do not report the objective as a result.Source: §B.2 · p. 16, Table 5 · p. 16

B.3 RUNG GRIDS

  • The full Llama-2-7B rung grid has a steep, convex loss curve: removing two seed bits multiplies excess loss by 1.8–3.2, while each coefficient added beyond k = 3 cuts it by about half or more.Later models use reduced grids built from the rungs time allowed, with one rung above 4.0 bits per weight: (14, 4).Source: §B.3 · p. 16, Table 6 · p. 16

B.4 PER-BLOCK ERROR ACROSS RUNGS

The measured block errors broadly follow the rung ordering assumed by the objective, but substantial block-level scatter limits that approximation. The S-Quant replication also finds higher reconstruction error than SeedLM at lower rates.

B.4 PER-BLOCK ERROR ACROSS RUNGS

  • The objective assumes each block’s error factorizes into a block term and a rung term, so one e(g) represents every block at a rung.The observed scatter is one reason measured loss departs from the objective.Source: §B.4 · pp. 16–17
  • Typical-block rung-error ratios were 3.43, 2.21, 1.49, 0.79, and 0.29; the central 80% spans factors of 3.5–3.9 across rungs.Across all rungs, block-level scatter has a log standard deviation of about 0.5, consistent with independent variation in how well the best seed fits each block.Source: Table 7 · p. 17, §B.4 · p. 17

B.5 S-QUANT’S RATE AND OUR REPLICATION

  • At 3.8 bits, S-Quant’s median relative error is 2.1–2.3× SeedLM’s at 4.0 bits, reaching SeedLM’s 4-bit error at 5.2 bits on both models.The measurements cover Qwen2.5-7B and Llama-3-70B.Source: §B.5 · p. 17
  • At B = 16, explicitly storing S-Quant’s per-block kb adds 0.19 bits per weight, making its nominal 3.8-bit configuration about 3.99 effective bits.The reported rate counts seeds, coefficients, and group scales, but not the per-block coefficient count needed to parse the stream.Source: §B.5 · p. 17
  • Within S-Quant’s format, adaptive kb gives 8.7% lower error than fixed k at 3% more bits on every tested tensor; int8 coefficients use twice SeedLM’s mantissa width.The int8 coefficients buy 35% fewer basis vectors at any rate.Source: §B.5 · p. 17

C HARDWARE IMPLEMENTATION

The hardware implementation evaluates Seed-Q’s LFSR weight reconstruction separately from a shared PE array. Its reconstruction kernels use common arithmetic conventions, but omit several stages needed for a complete integrated accelerator.

C HARDWARE IMPLEMENTATION

  • Seed-Q reconstructs one weight at a time from an externally supplied seed and allocation parameters, using a configurable LFSR and one reused integer MAC lane.The kernel applies exponent scaling and normalizes the accumulated value to produce signed outputs.Source: C HARDWARE IMPLEMENTATION · p. 17
  • At S = 16 and k = 3, all three kernels have a 561-cycle request interval per eight-weight block, estimated at a 10 ns clock period.SeedLM fixes S = 16 and k = 3; Seed-Q supports S ∈ [8, 16] and k ∈ [2, 6], while S-Quant fixes S = 16 and supports k ∈ [2, 6].Source: C HARDWARE IMPLEMENTATION · p. 17
  • The comparison evaluates reconstruction kernels, not a complete integrated accelerator: sensitivity estimation, schedule generation, bitstream unpacking, and full memory-to-array scheduling are excluded.The implementations share arithmetic conventions and are not bit-exact reproductions of the original hardware; the S-Quant kernel uses direct LFSR generation rather than its published LUT architecture.Source: C HARDWARE IMPLEMENTATION · pp. 17–18
  • Figure 5 shows the weight generator and shared (16×16) PE array evaluated separately; the PE multiplies reconstructed weights by activations and accumulates partial sums.The array’s products are accumulated in a partial-sum register.Source: C HARDWARE IMPLEMENTATION · p. 17, Figure 5 · p. 18

D SECURITY EVALUATION SETUP

The security evaluation measures selected reconstruction faults in one Llama-3-8B query-projection tensor using WikiText-2 perplexity and sampled zero-shot ARC-Easy accuracy. Its simulated faults and restricted Seed-Q schedule do not establish deployed or intrinsic security rankings.

D SECURITY EVALUATION SETUP

  • The experiment replaces only Llama-3-8B’s layer-0 query projection with clean or corrupted reconstructions, leaving all other checkpoint parameters unchanged.It measures WikiText-2 perplexity on four 2,048-token windows and zero-shot ARC-Easy accuracy on 100 sampled questions, with CPU BF16 inference and random seed zero.Source: D SECURITY EVALUATION SETUP · p. 18
  • No-Q flips a sampled BF16 mantissa bit, AWQ flips a scale’s exponent bit, SeedLM flips a seed bit affecting every block weight, and S-Quant uses a modeled midpoint count mismatch.AWQ is evaluated at 4.13 bits per weight without activation-aware calibration; SeedLM uses 4 bits per weight, while S-Quant uses 4.50 bits per weight in a setup differing from its published format.Source: D SECURITY EVALUATION SETUP · p. 18
  • The Seed-Q fault test uses a restricted gain-derived schedule, not the full importance estimate, rung-hull allocation, or decoder’s cross-tensor dependencies.The sampler perturbs an FP32 copy of γ although the checkpoint stores BF16, and the successful channel and bit were not logged, so a physical stored-bit match cannot be verified.Source: D SECURITY EVALUATION SETUP · p. 18
  • Substituted blocks model incorrect boundaries but are not outputs of a corrupted packed decoder, so the measured degradation does not establish a single-bit fault’s footprint.The study covers one selected fault per scheme in one tensor; unequal trigger locations prevent an intrinsic security ranking, and stored-bit injection, matched triggers, and repeated cross-layer faults remain needed.Source: D SECURITY EVALUATION SETUP · pp. 18–19
Loading 2609.38477v1…