PaperlayerView exact revision on arXiv

Source-linked AI summary

Uncertainty Drives Social Bias Changes in Quantized Large Language Models

Stanley Z. Hua, Sanae Lotfi, Irene Y. Chen

arXiv:2602.06181v1cs.CL

TL;DR

Post-training quantization can change large language models’ social biases, but existing aggregate evaluations provide limited evidence about these behavioral shifts. Across 50 quantized models and 13 bias datasets, the study finds substantial response flipping and asymmetric demographic impacts despite stable aggregate bias metrics.

  • Problem

    Evidence on how post-training quantization affects social bias is limited, with inconsistent benchmarking and response-evaluation practices complicating comparisons.

  • Method

    The study introduces PostTrainingBiasBench and paired response-level and aggregate analyses across 50 quantized models and 13 closed- and open-ended bias datasets.

  • Results

    Up to 21% of responses flip between biased and unbiased states after quantization while aggregate metrics remain unchanged, with asymmetric effects across demographic groups.

  • Takeaways & Limitations

    Post-quantization bias evaluation should examine uncertainty-linked response changes and demographic subgroups rather than relying solely on aggregate metrics.

  • Takeaways & Limitations

    The English, often hypothetical benchmarks and deterministic generation limit generality to real-world conversational, multilingual, intersectional, and nondeterministic settings.

Abstract

from arXiv · show

Post-training quantization reduces the computational cost of large language models but fundamentally alters their social biases in ways that aggregate metrics fail to capture. We present the first large-scale study of 50 quantized models evaluated on PostTrainingBiasBench, a unified benchmark of 13 closed- and open-ended bias datasets. We identify a phenomenon we term quantization-induced masked bias flipping, in which up to 21% of responses flip between biased and unbiased states after quantization, despite showing no change in aggregate bias scores. These flips are strongly driven by model uncertainty, where the responses with high uncertainty are 3-11x more likely to change than the confident ones. Quantization strength amplifies this effect, with 4-bit quantized models exhibiting 4-6x more behavioral changes than 8-bit quantized models. Critically, these changes create asymmetric impacts across demographic groups, where bias can worsen by up to 18.6% for some groups while improving by 14.1% for others, yielding misleadingly neutral aggregate outcomes. Larger models show no consistent robustness advantage, and group-specific shifts vary unpredictably across model families. Our findings demonstrate that compression fundamentally alters bias patterns, requiring crucial post-quantization evaluation and interventions to ensure reliability in practice.

1 Introduction

Post-training quantization is increasingly used to reduce LLM resource demands, but its downstream effects on social bias remain poorly understood and inconsistently evaluated. This paper introduces a unified benchmark and identifies uncertainty-driven response flips and asymmetric demographic impacts that aggregate metrics can conceal.

  • Motivation: Post-training quantization makes LLMs more practical in resource-constrained settings but operates without awareness of downstream behavioral changes.This disconnect is especially concerning as quantized models are increasingly deployed in healthcare, law, and other high-stakes domains.
  • Motivation: Quantized models can exhibit unpredictable social-bias shifts, including harmful behaviors reintroduced after alignment.Prior work also reports increased hallucinations and reduced fact recall, extending quantization concerns beyond performance degradation.
  • Problem and contribution: Conflicting conclusions across disjoint models, datasets, and metrics leave practitioners without actionable guidance.The paper addresses this evaluation gap with standardized response extraction, bias comparison, and rigorous pairwise statistical testing.
  • PostTrainingBiasBench: 50 quantized models across 9 closed-ended and 4 open-ended datasets are evaluated through the unified PostTrainingBiasBench framework.The framework standardizes response extraction and bias-metric comparison while applying rigorous pairwise statistical testing.
  • Masked bias flipping: Up to 21% of responses flip between biased and unbiased states after quantization while aggregate metrics remain unchanged.The paper terms this quantization induced masked bias flipping and reports strong correlations with model uncertainty and quantization strength, but not model size.
  • Asymmetric social group impacts: -14% to +18.6% changes across social groups can occur within the same model despite neutral aggregate metrics.Sub-group analysis therefore reveals dramatically different post-quantization outcomes across demographic groups.

2 Background

Prior work frames social bias as disparate treatment or outcomes between social groups and distinguishes representational from allocation harms. Post-training quantization improves deployment efficiency but can reduce capabilities, while prior studies report inconsistent effects on bias across models and datasets.

  • Social bias: Social bias involves disparate treatment or outcomes between social groups, including representational and allocation harms.Representational harms include marginalizing beliefs such as stereotyping and toxicity, whereas allocation harms involve inequalities in treatment and opportunities.
  • Social bias: Intrinsic bias measures in embedding space correlate poorly with downstream task biases, motivating broader evaluation approaches.Early work measured gender bias in text embeddings, but subsequent studies reported poor correlation with downstream task biases.
  • Post-training quantization: Post-training quantization approximates model parameters with fewer bits module by module to improve efficiency in resource-poor deployment settings.This efficiency comes with capability trade-offs, including worsened fact recall and increased hallucinations.
  • Prior quantization findings: Prior quantization studies found mixed and dataset-dependent bias effects, including increased bias on DiscrimEval, DT-Stereotyping, and age bias on BBQ.Other reported outcomes included minimal effects on CrowS-Pairs, no effects on Adult and RealToxicityPrompts, and unclear results on decoder-only models.

3 A Unified Framework for Measuring Changes in Social Bias

The framework unifies social-bias evaluation across 13 datasets, three capabilities, standardized response selection, and complementary individual-response and aggregate metrics. It also distinguishes response and bias flipping from aggregate behavior flipping while accounting for methodological and detection limitations.

  • Limitations: Methodological inconsistencies in datasets, response extraction, and parsing can produce conflicting social-bias findings across studies.The framework also cautions that LLaMA Guard precision is near 0.5 on BiasLens-GenWhy and CEB-Continuation, limiting interpretation of bias-flip estimates.
  • Benchmark capabilities: 13 datasets are organized into three capabilities: bias identification, equal outcomes under informative context, and preference for refusal or uncertainty under biased prompts.The evaluation includes both closed-ended and unconstrained-generation settings, with datasets covering toxicity, decision fairness, refusal or uncertainty, and multi-turn conversations.
  • Response selection: Closed-ended responses are selected using the highest geometric mean of token probabilities, equivalent to the lowest perplexity or highest length-normalized log-likelihood.This avoids context-independent next-token token biases and parsing difficulties associated with generated responses.
  • Change measurement: Individual response changes are classified as response flipping, while flips between biased and unbiased responses are termed bias flipping.The framework separately defines behavior flipping as a change in aggregate bias measures.
  • Change measurement: Aggregate bias metrics are re-normalized to 0–1, with higher values indicating more bias, and are computed by social axis when available or across the dataset otherwise.Each dataset retains its own bias measure while the common scale facilitates comparison.

4 Results

Quantization changes model behavior in ways that uncertainty, quantization strength, question structure, dataset context, and social group mediate, while aggregate metrics often remain unchanged. These hidden and asymmetric shifts undermine standard bias evaluations and show no consistent robustness advantage for larger models.

  • Uncertainty and behavioral changes: 21% of high-uncertainty BBQ responses flip after quantization, compared with less than 2% for most datasets among low-uncertainty responses.High uncertainty is defined as entropy > 0.66, while low uncertainty is entropy < 0.33.
  • Aggregate metrics and response flipping: 13-14% of IAT and BBQ responses flip despite stable aggregate metrics, while FMT10K reaches 21% under the same condition.These hidden changes are invisible to standard evaluation methodology.
  • Quantization strength: 12% of RTN W4A16 responses change behavior across datasets, versus 2% for RTN W8A16.Stronger quantization also produces 2-3x larger variance in uncertainty changes, with entropy shifts reaching 0.25 points in Credit, StereoSet, and BBQ.
  • Model scale and rankings: Qwen 2 7B has a 2% behavior-flipping rate, compared with 7% for LLaMA 3.1 8B and 9% for Ministral 8B, showing no monotonic size relationship.Within the Qwen 2.5 family, flipping patterns are erratic rather than consistently improving with scale.
  • Question-level vulnerability: 50% of responses flip for some questions, whereas most questions show flipping below 25%, revealing strongly heterogeneous question-level vulnerability.The distribution is heavily right-skewed across datasets.
  • Social-group asymmetry: Quantized Qwen 2.5 14B variants shift biased responses by -10.3% for short individuals and +7% for male individuals.Across all models, the corresponding changes are -1.1% and +1.6%, respectively.

5 Conclusion

Post-training quantization changes model bias in ways stable aggregate metrics can miss, especially for uncertain predictions and demographic subgroups. The findings support mandatory post-quantization, subgroup-focused evaluation and uncertainty-aware safety checks.

  • Key findings: 21% of high-uncertainty predictions experience response flipping, with 3-11× greater susceptibility than confident predictions.The evaluation covered 50 quantized models across 13 bias datasets.
  • Key findings: Up to 33 percentage points separate demographic-group impacts within the same model despite deceptively stable aggregate metrics.Stable aggregates can conceal deterioration for some groups offset by improvements for others.
  • Implications: Uncertainty’s strong correlation with bias changes suggests confidence calibration can prescreen quantization safety.The paper presents this as first evidence in the context of compression.
  • Implications: Zero-centered effect-size distributions do not establish safety because impacts vary unpredictably across datasets and model families.Symmetry indicates effects are not uniformly negative, while coarse aggregates remain unreliable for deployment decisions.
  • Recommendations: Post-quantization assessment should be mandatory, prioritize subgroup metrics and high-uncertainty predictions, and favor 8-bit over 4-bit quantization.The recommendations also call for task-specific rather than benchmark-only evaluations and uncertainty-based flags for vulnerable predictions.

6 Limitations

The study’s conclusions are constrained by English-only, often hypothetical benchmarks and by validation evidence that some reported flipping rates are upper bounds. However, high negative predictive value supports the stability claims in the affected datasets.

  • English-only, hypothetical benchmarks may limit generality to real-world conversational deployment, multilingual settings, and intersectional analyses.
  • 40–55% PPV indicates that LLaMA Guard 3 8B overcalls bias changes in some datasets, making reported flipping rates upper bounds there.
  • 88% NPV supports the stability claims despite overcalling in some datasets.

A Technical Appendices and Supplementary Material · A.3 Comparison to Previous Studies

Table S1 compares the study with previous studies using explicit quantization and model-format labels while restricting the comparison to social-bias datasets.

  • A.3 Comparison to Previous Studies: Table S1 provides a comparison to past studies.The table is explicitly titled “Comparison to past studies.”
  • A.3 Comparison to Previous Studies: IT denotes instruction fine-tuned models.The caption defines IT as instruction fine-tuned models.
  • A.3 Comparison to Previous Studies: W4 denotes 4-bit weight quantization.The caption defines W4 as 4-bit weight quantization.
  • A.3 Comparison to Previous Studies: A8 denotes 8-bit activation quantization.The caption defines A8 as 8-bit activation quantization.
  • A.3 Comparison to Previous Studies: Unspecified A8 means that activations are not quantized.The caption states that activations are not quantized when A8 is not specified.
  • A.3 Comparison to Previous Studies: The comparison excludes datasets unrelated to social bias.The caption limits the listed datasets to those relevant to social bias.

A.4 Dataset Details

The study uses repackaged bias datasets with aggregate metrics, while applying targeted dataset subsets and task-specific grouping to make diverse evaluation feasible. CEB responses are aggregated by social axis across stereotyping and toxicity bias types.

  • The datasets and aggregate metrics are summarized in Table S3, and the repackaged datasets are provided in the associated GitHub repository.
  • CEB covers stereotyping and toxicity bias, with Recognition, Continuation, and Conversation combining questions from both dataset types.CEB-Recognition-T specifically assesses toxicity recognition; aggregate metrics group responses by social axis across stereotyping and toxicity.
  • The evaluation uses subsets of BBQ and FairMT10K to cover diverse datasets feasibly, selecting ambiguous-context BBQ questions and FairMT10K’s challenging Interference Misinformation task.BBQ ambiguous contexts were selected because models show greater bias than in disambiguated contexts; FairMT10K combines stereotyping and toxicity assessment in the Interference Misinformation task.

A.4.1 Creating the IAT Dataset

The IAT dataset pairs contrasting social groups with stereotypically associated words and prompts an LLM to assign groups or form intuitive word pairings. Randomized combinations yield 13,858 unique questions, with aggregate bias defined separately.

  • Dataset construction: 8 to 16 words or short phrases are split between two contrasting social groups, such as Young vs. Old, Takuya vs. Karim, and abled vs. disabled.The LLM assigns a social group to each descriptive text, and bias is defined from assignment frequencies.
  • Dataset construction: The IAT prompting task asks the model to connect words from two sets with hyphens into two natural or intuitive word pairs.The two resulting pairs are separated by a semicolon.
  • Dataset construction: 13,858 unique questions result from all valid social-group-pair combinations, with keyword and group orders randomly shuffled to mitigate ordering bias.An aggregate bias metric is defined separately in Section A.5.

A.4.2 Extracting Social Groups in Datasets BiasLens and FMT10K

Because the publicly released BiasLens and FMT10K data do not provide social-group labels, the study uses GPT-4o to extract targeted groups and their social axes. Regexes and rules then merge extracted groups, with only the final FMT10K turn processed and annotations released in the associated repository.

  • Motivation: The publicly released data omit the social groups targeted by BiasLens-Choices, BiasLens-GenWhy, and FMT10K questions.This prevents direct analysis of social-group-specific behavioral changes from the original files.
  • GPT-4o extraction: GPT-4o-2024-08-06 extracts each prompt’s targeted group and assigns it to one or more social axes.The specified axes are age, gender, sexual orientation, disability, appearance, nationality, race, religion, and socioeconomic status.
  • Post-processing: Simple regexes and rules identify and merge social groups in the GPT-4o text predictions.This post-processing step standardizes the extracted group labels before group-specific analyses.
  • FMT10K and release: For FMT10K, only the last prompt in each five-turn conversation is passed to GPT-4o, and extracted groups and axes are provided in the associated GitHub repository.The repository contains the resulting social-group annotations in the dataset files.

A.5 Aggregate Bias Metrics … A.13 Additional Figures & Tables

The supplementary sections define aggregate bias metrics, document evaluation and quantization procedures, and provide additional evidence that quantization-driven behavioral changes are uncertainty-dependent, asymmetric across groups, and textually disruptive. They also illustrate concrete bias flips and report implementation, model, and tuning details.

  • A.5 Aggregate Bias Metrics: Ambiguous BBQ, StereoSet, and IAT metrics quantify bias through unknown-response selection, stereotypical versus anti-stereotypical balance, and language-modeling capacity.BBQ is unbiased when responses are unknown or stereotypical and anti-stereotypical choices are equally frequent; StereoSet uses a 0–1 bias scale with lower values better; IAT classifies responses by biased-option probability.
  • A.6 Bias Detection in Open-Ended Generations: Paired LLaMA Guard evaluation improves negative predictive value from 0.7 to 0.88 when identifying bias changes between matched pre- and post-quantization responses.Unpaired performance is PPV = 0.86 and NPV = 0.70, with NPV of 0.5-0.6 on challenging datasets.
  • A.11 Preference Tuning Experiment: Preference tuning uses Qwen 2.5 0.5B Instruct as the reference model and SimPO with LoRA over 5 epochs, batch size 4, and learning rate 5 × 10−5.The BBQ tuning and evaluation set contains 5322 questions, with response-flipping rates ranging from 1.5% for old to 12.3% for male groups under RTN W4A16.
  • A.12 Examples of Bias Flipping in Generated Text: Examples show quantization can change opinions from unbiased to biased or biased to unbiased, introduce demographic associations, and alter responses across Qwen and LLaMA models.Examples include SmoothQuant RTN W4A16, AWQ W4A16, RTN W4A16, and GPTQ W4A16 across BiasLens-GenWhy and CEB datasets.
  • A.13 Additional Figures & Tables: Shannon entropy ≥0.66 and lower average token probability are associated with more response flipping, while 4-bit quantization causes greater closed-ended changes than 8-bit quantization.The supplementary figures identify uncertainty and quantization bit width as key correlates of behavioral change.
  • A.13 Additional Figures & Tables: Across quantizations, bias flipping is nearly balanced overall but becomes more asymmetric for particular social groups, with confidence intervals reported for BBQ, FairMT10K, and BiasLens-GenWhy.The supplementary tables identify group-specific differences between unbiased-to-biased and biased-to-unbiased flipping.
  • A.13 Additional Figures & Tables: Quantized generations change length and structure unpredictably, retain only around 30-50% of sequential content, and usually diverge from the original response before 25% of its words.LanguageTool error counts remain mostly similar, while greedy decoding often differs almost immediately in BiasLens-GenWhy and CEB-Conversation; RTN W8A16 preserves responses longer in some datasets.
Loading 2602.06181v1…