Where Post-Training Quantization Breaks Text Embedders: A Measured Map Across Four Embedder Families

• At INT2, checkpoints with reconstruction error between 0.315 and 0.337 retain 1.3% to 65.9% of retrieval quality, while the dominant isolated damage varies by family.Embedding-only quantization retains 97.6% to 100.5% across the five checkpoints.Source: §1 · pp. 2–3
• At INT4/g16, the largest single-module effect is 1.01 points; at INT3, module ordering varies by family and joint damage is non-additive.At INT3/g16, feed-forward is costliest in the Qwen lineage, while attention is costliest in EmbeddingGemma, BGE-M3, and E5.Source: §1 · pp. 2–3
• Reconstruction error tracks uniform-quantization damage well (r = 0.80 to 0.92) but individual-module damage poorly (r = −0.04 to 0.42).Pooling module arms across bit widths raises the correlation to 0.66 through range extension.Source: §1 · p. 3
• At INT4/g16, 13% to 22% of top-10 documents are replaced despite less than one point of average NDCG change; 97.8% of pooled evictions are non-relevant.Before a model’s cliff, the gold set survives; for collapsing checkpoints, relevant-only overlap falls to 0.001–0.006.Source: §1 · p. 3
• Fine-tuning changes quantization tolerance by at most 1.0 point in two-corpus averages and does not create the INT3 decline or INT2 collapse.A pooling intervention leaves the cliff unexplained; tokenization behavior remains untested.Source: §1 · p. 3

2.1 Mixed-precision and post-training quantization

• Prior quantization work studies sensitivity and non-separable losses; this paper tests whether sensitivity signals remain useful for preserving retrieval order across embedder families.The paper does not claim quantization interactions or their effects on bit allocation are new observations.Source: §2.1 · p. 4

2.2 Low-bit Transformer encoders

• Encoder quantization methods include Q-BERT [29], but the paper instead quantizes existing checkpoints without fine-tuning, quantization-aware training, or distillation.This is the setting where a practitioner already has an embedder and wants to make it smaller.Source: §2.2 · p. 4

2.3 Extreme low-bit LLMs

• Specialized two- and three-bit generative-LLM methods differ from the uniform group-wise scalar PTQ of finished models measured here.Their low-bit success is not evidence that these PTQ arms should survive, and the measured collapse is not evidence against those methods.Source: §2.3 · p. 4

2.4 Low-bit text embedders

• Existing low-bit text-embedder approaches change the checkpoint through quantization-aware training or continued pre-training and distillation, unlike post-training quantization.A surviving ternary PTQ control does not contradict ternary embedders trained to be ternary.Source: §2.4 · p. 5

2.5 Compressing representations rather than weights

• Representation compression reduces index vectors, whereas this paper compresses model weights and keeps the index full precision.JPQ [41] and RepCONC [42] use ranking losses for representations; whether that carries over to weights is the question tested here.Source: §2.5 · p. 5

• The study includes Qwen3-Embedding-0.6B, EmbeddingGemma-300M, BGE-M3, and E5-base-v2, plus a fine-tuned Qwen3 checkpoint as a paired control.The paired control varies training while holding architecture, tokenizer, and hidden size fixed, and is excluded from cross-family evidence.Source: §3 · p. 5
• Evaluation uses SciFact, NFCorpus, and the SkillRet test split, reporting NDCG@10; the SkillRet evaluation uses a pinned dataset revision.The pinned split contains 4,392 queries over 6,006 documents.Source: §3 · p. 5
• Full-precision FP32 NDCG@10 supplies the retention baselines; W8/g16 and W4/g16 are approximately neutral, while models separate at W3 and diverge at W2.The reported ternary comparison is only for E5 because other checkpoints’ ternary and symmetric arms were retracted after a quantizer defect.Source: Table 1 · p. 6, Table 2 · p. 6
• Quantization is group-wise affine weight-only PTQ with FP16 scales and zero-points; activations remain FP32 and each model uses its own prompt convention.The experiments do not establish integer-kernel latency or end-to-end deployed accuracy.Source: §3 · p. 6

4.1 Module sensitivity appears late, and its ordering does not transfer reliably

• At INT4/g16 module effects are small; at INT3 the costliest module varies by family and joint costs are non-additive, while the embedding table remains nearly free through INT2.At INT2, quantizing only the embedding table retains 97.6% to 100.5% of retrieval quality, including in checkpoints whose full-model retention is under 2%.Source: Table 3 · p. 8, §4.1 · pp. 8–9

4.2 Reconstruction error predicts one axis, not the other

• Uniform reconstruction error tracks loss (Pearson r = 0.80 to 0.92), but fixed-width module correlations never exceed 0.42; pooling across widths inflates them to 0.658.At INT2, similar reconstruction errors of 0.315 to 0.337 accompany retained NDCG ranging from 1.3% to 65.9%.Source: §4.2 · pp. 10–11, Table 5 · p. 11

4.3 Task fine-tuning does not create the cliff

• At most 1.0 point in two-corpus retained-NDCG averages separates the base checkpoint from its fine-tuned version, while the base already shows an INT3 decline and INT2 collapse.The comparison averages SciFact and NFCorpus, which are out-of-domain for both checkpoints.Source: §4.3 · pp. 11–12, Table 6 · p. 12

4.4 NDCG@10 does not see what quantization moves

• At INT4/g16, 13% to 22% of the top-10 is replaced and the rank-10 document leaves the list for roughly half of queries, despite NDCG changing by less than one point.E5-base-v2 is absent from the ranking-stability table because its rank-preservation arm was not measured.Source: §4.4 · p. 12, Table 7 · p. 13
• Boundary margins predict rank flips, but the relationship is shallow; margin-proportional weighting would be close to uniform weighting outside the top decile.Boundary-preservation objectives target the teacher's entire ranking, including reorderings that do not affect the metric.Source: §4.4 · pp. 13–14, Figure 3 · p. 14
• At INT4/g16, relevance judgments show mostly non-relevant evictions; on INT2/NFCorpus, relevant-only overlap separates EmbeddingGemma (0.500) from BGE-M3 (0.000).On the hard corpus, the share of evictions that are relevant does not distinguish healthy from broken models; relevant-only overlap does.Source: §4.4 · pp. 12–13, Table 8 · p. 15

5 Controls, Reproducibility, and Cliff Probes

• A train/evaluation prompt mismatch cost 8.84 NDCG@10 points on one checkpoint and 4.23 on another, while quantization partly concealed the mismatch.The evaluation bundle records each checkpoint's resolved query prefix and refuses to run without it; the section notes that consistency, not the chosen prefix, matters after fine-tuning.Source: §5 · p. 15
• Reported sizes use actual on-disk artifact bytes, and fake-quantized quality measurements do not establish integer-kernel latency.Packed quantized files, rather than arithmetic size estimates, are reported; all quality numbers use fake quantization with FP32 arithmetic.Source: §5 · p. 16
• Splitting by only a query's first gold document left 100% of nominally held-out documents reachable through other golds and inverted a model-selection decision.The corrected splits are taken over all gold documents.Source: §5 · p. 16

5.1 Vocabulary size and embedding-table fragility do not explain the INT2 cliff

• BGE-M3 and EmbeddingGemma have similarly large vocabularies, 250,002 and 262,144, yet their INT2 retention differs by 64.1 points.E5-base-v2, with a vocabulary of 30,522, retains 44.8%, while EmbeddingGemma retains 65.9%.Source: §5.1 · p. 16
• Embedding-only INT2 retains 97.6% to 100.5% of retrieval quality, while the isolated source of INT2 damage varies by model family.BGE-M3 is nearly attention-dominated, the Qwen lineage is feed-forward-dominated, and no isolated module approaches the joint loss for EmbeddingGemma or E5-base-v2.Source: §5.1 · p. 16, Table 9 · p. 17

5.2 What sets the INT2 cliff is not pooling

• Mean pooling does not rescue collapsing models at INT2, and the evidence does not establish what separates surviving from collapsing checkpoints.Forcing mean pooling moves BGE-M3 from 2.4% to 4.1% retention and Qwen3-Embedding from 1.6% to 9.5%; removing mean pooling from EmbeddingGemma leaves a weakly interpretable 30.5% result because the intervention damages its full-precision model.Source: §5.2 · pp. 17–18

6 Discussion: distil small, or squeeze large?

• Outside SkillRet, the 109M student scores 7.79 on NFCorpus versus 31 to 39 for general-purpose models, and 50.73 on SciFact versus 64 to 78.The two students' out-of-domain scores are not monotonic in student size, and the authors do not attribute the difference to capacity.Source: §6 · p. 18
• The practical conclusion is limited to a fixed, known retrieval task: the measured student frontier advantage does not hold for a general-purpose embedder.The students were distilled on SkillRet pairs alone, so the out-of-domain degradation may reflect the training distribution rather than distillation itself.Source: §6 · p. 18
• On SkillRet, the 109M INT3 student occupies 68.4 MB and scores 78.04, versus 297.9 MB and 64.46 for the teacher's INT3/g32 PTQ arm.The student also beats the teacher's INT3/g32 arm on both size and score; the comparison is task-bound, not a general-purpose result.Source: §6 · p. 18, Table 10 · p. 19

6.1 Audit of the measurement code

• The audit found that headline verdicts remain unchanged, although three measurements were not repeated and remain from the pre-fix evaluator.E5 and EmbeddingGemma were remeasured under their prompt contracts; the un-repeated results are their Table 8 top-10 turnover, EmbeddingGemma's rank-flip curve, and the pooling intervention. EmbeddingGemma's dense output layers stayed full precision, but quantizing them changed INT2 retention by only 1.1 points.Source: §6.1 · pp. 18–20

7 Limitations

• The study covers three retrieval corpora, not full MTEB, and no multilingual benchmark, so BGE-M3’s multilingual capability is untested.Source: §7 · p. 20
• The experiments quantize weights only; they do not test activation quantization or implement a mixed-precision allocator.The INT4/g16 result concerns limited sensitivity headroom, not a demonstration that a specific allocator fails.Source: §7 · p. 20
• No integer-kernel latency is reported, and low-bit storage reduction does not imply proportional CPU speedup.Closing that systems gap is described as a separate problem addressed by specialised kernels such as T-MAC [34].Source: §7 · p. 20
• Tokenization behaviour—including segmentation, vocabulary coverage, and sequence-length inflation—remains unmeasured.Source: §7 · p. 20
• Cross-family divergence is based on four families, with the fifth checkpoint serving as a paired control, limiting generalization to embedders broadly.The results show that a policy measured on one family failed to describe another, but do not identify the architectural cause.Source: §7 · p. 20

8 Conclusion

• Reconstruction error screens uniform bit widths reasonably well but is substantially less reliable for choosing tensors to protect.The module axis is where a retrieval-aware sensitivity signal must prove itself.Source: §8 · p. 20
• At INT4/g16, module-level sensitivity is too small to allocate against; at INT3, its ordering inverts between families.Source: §8 · p. 20
• At INT2, five checkpoints with reconstruction error from 0.315 to 0.337 retain 1.3% to 65.9% of retrieval quality.Source: §8 · p. 21
• Per-module costs do not predict joint damage additively; interaction magnitude and, at INT3/g16, even its sign vary across families.Source: §8 · p. 21
• The embedding table is never the dominant isolated protection priority across the four families, despite being the largest tensor in several.Feed-forward protection helps in some families and hurts in others, while INT3 robustness varies sharply by family.Source: §8 · p. 21