PaperlayerView exact revision on arXiv

Source-linked AI summary

Debias-SparseGPT: Bias-Aware Pruning for Large Language Models

Irina Proskurina, Guillaume Metzler, Antoine Gourru, Julien Velcin

arXiv:2609.02496v1cs.CL

TL;DR

Pruning can amplify fairness disparities in generative LLMs despite preserving aggregate accuracy. Debias-SparseGPT adds a paired-input, second-order debiasing term to pruning, and across models and sparsity settings it reduces bias while preserving perplexity and downstream accuracy. Under 2:4 sparsity, richer calibration data further improves fairness and performance, although broader safety coverage and theoretical calibration guarantees remain open.

  • Problem

    Compression can compromise fairness in generative LLMs, and prior methods had not addressed pruning-induced bias during compression.

  • Method

    Debias-SparseGPT extends SparseGPT with a paired-input term that produces a bias-aware Hessian for pruning-mask selection and second-order weight reconstruction.

  • Results

    Debias-SparseGPT consistently improves fairness benchmarks while preserving perplexity and downstream task accuracy across nine LLM families and evaluated sparsity levels.

  • Takeaways & Limitations

    The method achieves a better fairness-performance trade-off while preserving the computational efficiency of sparse pruning.

  • Takeaways & Limitations

    The evaluation primarily covers representational bias, while broader safety evaluation and theoretical bounds for calibration-set effects remain future work.

Abstract

from arXiv · show

Model compression techniques such as pruning and quantization facilitate the efficient deployment and acceleration of Large Language Models (LLMs). However, recent studies show that weight sparsification methods, such as SparseGPT, can amplify existing biases in models, with outputs varying significantly depending on persona cues in the prompt. In this paper, we introduce Debias-SparseGPT, a post-training pruning method incorporating representational debiasing using a second-order term defined over demographically contrasting inputs. We perform empirical validation of our method over a wide range of generative LLMs. Across models and sparsity regimes (25%, 50%, and structured 2:4 sparsity), Debias-SparseGPT consistently reduces pruning-induced bias compared to SparseGPT while preserving model perplexity and zero-shot accuracy. Under the most restrictive 2:4 structured sparsity pattern, which most aggressively degrades model quality, augmenting the calibration set with long-context, content-rich examples further improves both downstream performance and fairness. Overall, Debias-SparseGPT advances the bias-performance trade-off while preserving the computational efficiency of sparse models.

1 Introduction

LLM compression improves deployment efficiency, but pruning can compromise fairness even when aggregate accuracy is preserved. Debias-SparseGPT is introduced to mitigate pruning-induced bias while retaining SparseGPT’s computational advantages and model performance.

  • Compression techniques improve model throughput, reduce inference latency, and lower energy consumption and operational cost.
  • Pruning removes redundant weights to create lightweight models, while SparseGPT uses second-order reconstruction for a strong accuracy-sparsity trade-off.
  • Compression can preserve aggregate accuracy yet compromise fairness, measured through disparate results on matched prompts differing in sensitive attributes.
  • Prior work illustrates prediction differences after pruning, motivating methods that address compression-induced bias in generative LLMs.
  • Debias-SparseGPT modifies pruning-mask construction and second-order reconstruction with a fairness objective while preserving SparseGPT’s computational efficiency.

2 Background

Prior work establishes pruning methods, fairness concerns, and evaluation practices for compressed LLMs. The paper measures representational bias through unsupported demographic predictions and motivates Debias-SparseGPT to preserve abstention performance under compression.

  • 2.1 Related Work: Post-training pruning methods rank weights using second-order saliency, magnitude, or activation-based criteria under a target sparsity constraint.
  • 2.1 Related Work: SparseGPT extends second-order pruning to generative LLMs through layer-wise pruning that uses calibration data to approximate reconstruction Hessians.
  • 2.1 Related Work: Compression research has largely emphasized perplexity, while pruning-related demographic disparities, toxicity, bias, and fairness have received more limited attention.
  • 2.1 Related Work: Debias-SparseGPT is presented as a pruning method that directly incorporates a fairness objective to mitigate pruning-induced bias.
  • 2.2 Measuring Biases in Compressed LLMs: UnQover and BBQ assess whether models predict Unknown or Not stated for contextual questions involving sensitive attributes, where demographic-specific answers without evidence indicate bias.
  • 2.2 Measuring Biases in Compressed LLMs: Fairness is measured by accuracy on Not stated answers, and UnQover error 1 − A_unk increases under sparsification as predictions shift toward specific group labels.
  • 2.2 Measuring Biases in Compressed LLMs: Debias-SparseGPT aims to preserve abstention performance by minimizing the sparsification-induced shift ∆A_unk and reducing stereotype-related errors.

3 Debias-SparseGPT

Debias-SparseGPT modifies SparseGPT with a paired-input objective that preserves layer outputs while discouraging changes in representational differences. Its bias-aware Hessian guides both pruning-mask selection and second-order reconstruction without changing SparseGPT’s computational efficiency.

  • Objective: Debias-SparseGPT formulates pruning over paired inputs X0 and X1, adding a penalty on their input difference ∆X to discourage disparity amplification.The paired inputs can represent pro-stereotypical and anti-stereotypical texts.
  • Algorithm: For target sparsity s, the algorithm retains the (1 −s) fraction of parameters with the largest second-order saliency values, then sets masked weights to zero.Pruning is performed block-wise, with compensation applied after each block.
  • Algorithm: The implementation processes column blocks using a Cholesky factorization of the inverse Hessian, local error buffers, second-order block compensation, and lazy batched tail updates.These steps follow SparseGPT while incorporating the bias-aware Hessian.
  • Objective: The bias-aware Hessian combines reconstruction and paired-difference information, influencing both saliency-based mask construction and retained-weight correction.The additional paired-input term therefore affects which parameters are pruned and how the remaining matrix is reconstructed.
  • Difference from SparseGPT: Debias-SparseGPT preserves SparseGPT’s computational complexity and efficiency while replacing its standard Hessian with one augmented by paired pro- and anti-stereotypical input differences.The method is implemented with inputs including layer weights, paired inputs, target sparsity, and block size.

4 Experimental Settings

The experiments evaluate Debias-SparseGPT across nine instruction-tuned and base LLMs using fairness, language-modeling, and zero-shot benchmarks. Comparisons use common calibration data and multiple sparsity regimes against magnitude pruning, Wanda, SparseGPT, and dense models.

  • Models: Nine LLMs are evaluated: seven instruction-tuned models and two base models spanning LLaMA, Vicuna, Qwen, Mistral, Aya, Phi, Gemma, and DeepSeek families.The evaluated models include both generative instruction-tuned systems and pretrained base models.
  • Evaluation: Fairness is measured with UnQover, BBQ, and CrowS-Pairs, while perplexity is measured on WikiText-2 and zero-shot performance on MMLU and HellaSwag.UnQover and BBQ use Unknown/Not stated accuracy; CrowS-Pairs uses likelihood differences between stereotypical and anti-stereotypical continuations.
  • Baselines and calibration: Four baselines are compared: magnitude pruning, Wanda, SparseGPT, and the original dense models.The setup uses the same calibration data across the three pruning methods.
  • Baselines and calibration: 4212 paired pro- and anti-stereotypical sentences from StereoSet’s development set serve as calibration data.The calibration examples are used to compare pruning methods under matched conditions.
  • Sparsity regimes: The pruning setup covers unstructured sparsity and semi-structured N:M sparsity, including 1:4 comparisons and target sparsity levels defined by the retained-weight constraint.For semi-structured pruning, the method removes the lowest-saliency weights within each M-weight block.

5 Results

Debias-SparseGPT generally improves fairness-benchmark performance and the fairness-performance trade-off over SparseGPT while maintaining comparable language-modeling and downstream-task accuracy. Broader calibration data further improves results under aggressive 2:4 sparsity, while computational efficiency is preserved.

  • Generative bias benchmarks: Debias-SparseGPT consistently outperforms SparseGPT on UnQover and BBQ across evaluated models.For LLaMA, UnQover rises from 35.60% to 60.46%, while BBQ accuracy reaches 70.70% versus 76.40% for the dense model.
  • Fairness-performance trade-off: Debias-SparseGPT achieves the lowest DTO across all model families, indicating a more favorable fairness-performance trade-off than dense and pruned baselines.DTO decreases to 0.399 for LLaMA, 0.291 for Qwen, and 0.674 for Vicuna.
  • General performance: 59.76% versus 59.11% MMLU accuracy on LLaMA and 67.73% versus 67.35% on Qwen show comparable downstream performance to SparseGPT after pruning.Perplexity on Wiki-2 follows the same pattern relative to dense models.
  • Sparsity regimes: 25%, 50%, and 2:4 sparsity evaluations show that Debias-SparseGPT outperforms SparseGPT with comparable MMLU performance and lower DTO across sparsity patterns.The largest DTO decrease reported under 1:4 semi-structured sparsity is from 0.311 to 0.291.
  • Calibration data: 24.9 to 47.3 UnQover and 48.16 to 54.17 MMLU result from adding UltraChat calibration under 2:4 sparsity, with BBQ increasing from 66.70 to 78.60.The augmented calibration set contains 256 UltraChat examples totaling approximately 15k tokens.
  • Efficiency: O(d^2 + nd), or O(d^2) for the evaluated transformer projections, is the preserved layer-wise memory complexity of Debias-SparseGPT pruning.The bias-aware term is accumulated directly into the Hessian, preserving SparseGPT’s memory complexity; throughput increases from 27.54 to 73.05 tokens/s in the reported Qwen evaluation.

6 Conclusion

Debias-SparseGPT extends SparseGPT with a paired-input, bias-aware objective for pruning LLMs. Across diverse models and sparsity settings, it improves fairness-benchmark performance and the fairness-performance trade-off without affecting perplexity or downstream accuracy.

  • Conclusion: Debias-SparseGPT targets the disparate impact of pruning on model performance in causal question answering.The approach is presented as a model-compression method for reducing this impact.
  • Conclusion: A paired-input term creates a bias-aware Hessian that influences both pruning-mask selection and second-order reconstruction.The method is applied to nine LLMs from diverse model families, including instruction-tuned models.
  • Conclusion: Across evaluated model families, Debias-SparseGPT improves UnQover and BBQ under semi-structured and unstructured sparsity while preserving perplexity and downstream-task accuracy.The authors characterize this as the best fairness-performance trade-off across the evaluated model families.
  • Conclusion: Input-difference terms in the compression objective produce beneficial corrections in weight-pruning decisions for unstructured and semi-structured pruning.The objective may also be applicable to quantization or distillation in future work.

Limitations

The paper identifies scope and evaluation limitations involving language coverage, safety dimensions, sparsity-pattern analysis, and aggressive structured sparsity. It also notes unresolved calibration-theory questions and higher computational costs for training-based alternatives.

  • Scope: Experiments use English calibration data and benchmarks, so generalization to multilingual settings is not established.The paper notes that multilingual calibration data may further improve compressed-model performance.
  • Safety evaluation: The main evaluation focuses on representational bias rather than exhaustively covering other safety dimensions.Additional analyses found no increase in unsafe-response rate relative to SparseGPT in the evaluated setting, but broader coverage remains future work.
  • Sparsity analysis: The paper does not comprehensively analyze the learned sparsity patterns, providing only an initial Appendix G analysis.A more detailed investigation of pruning patterns is left for future work.
  • Structured sparsity: Under 2:4 structured sparsity, model perplexity degrades substantially, suggesting that highly constrained patterns may require richer calibration data.UltraChat experiments indicate that longer and more diverse calibration texts can improve performance; theoretical bounds for calibration sensitivity remain unresolved.
  • Computational scope: Training-based improvements may enhance compressed models but add a separate training stage and substantially more computation than post-training pruning.The paper therefore distinguishes these alternatives from the computational profile of post-training methods.

Ethical Considerations

The paper describes artifact licensing, intended implementation, and risks associated with releasing compressed models and code. It also emphasizes evaluating compressed models beyond downstream performance, including bias, uncertainty, and broader safety risks.

  • Artifacts and licensing: Experiments use nine publicly available pretrained LLMs and six datasets without private or personally identifiable information.Model usage complies with the corresponding license terms, including requirements for derivative works.
  • Intended use: Debias-SparseGPT is integrated into llm-compressor and can be combined with quantization and other compression techniques.The implementation is intended for broad language and multimodal model support and is planned for Apache-2.0 release upon acceptance.
  • Potential risks: Releasing compressed models and code may enable misuse involving stereotyped, biased, or toxic content targeting specific communities.The paper also reports that pruning can affect performance where dense models have higher predictive entropy, potentially enabling confidently stated incorrect outputs.
  • Risk assessment: The authors argue that compressed models should be evaluated for group bias, uncertainty, and broader safety risks alongside downstream performance.This framing connects reproducible artifact release with expanded model-risk assessment.

C Experimental Settings

The experiments evaluate nine LLMs using stereotype benchmarks, general-performance benchmarks, perplexity, and efficiency measures under shared calibration data and multiple pruning configurations. The implementation preserves SparseGPT’s layer-wise memory complexity.

  • Models: The study evaluates nine base and instruction-tuned LLMs, with model parameter counts and multilingual support listed in Table 7.License types and provider-specific usage conditions are documented in Table 8.
  • Bias benchmarks: Stereotype evaluation uses BBQ, UnQover, and CrowS-Pairs, covering target-group questions and stereotype–anti-stereotype sentence pairs.Table 9 provides an overview of the stereotype benchmarks.
  • Metrics: UnQover and BBQ use accuracy on Unknown or Not determined answers, while CrowS-Pairs measures stereotypical-likelihood preference with an ideal score of 50%.Candidate answers are scored using next-token logits.
  • Calibration and pruning: Main pruning experiments use 4,212 paired StereoSet examples as shared calibration data, with 256 UltraChat examples additionally used for 2:4 sparsity.The configurations include 25%, 50%, and semi-structured 2:4 sparsity settings.
  • Implementation and efficiency: Debias-SparseGPT preserves SparseGPT’s layer-wise memory complexity by accumulating the bias-aware term directly into the Hessian.Efficiency evaluation also estimates carbon footprint per 1M generated tokens using a stated carbon-intensity assumption.
  • General performance: Pruned models are evaluated using WikiText-2 perplexity and zero-shot MMLU and HellaSwag performance.The evaluation setup uses the LM Evaluation Harness for the multiple-choice benchmarks.

D Extended Results

Extended evaluations show that Debias-SparseGPT improves UnQover accuracy across model families, categories, and sparsity regimes while generally preserving perplexity and downstream performance. Under restrictive 2:4 sparsity, calibration augmentation further improves compressed-model results.

  • Results Across Additional Model Families: Across six additional model families at 1:4 sparsity, Debias-SparseGPT achieves higher UnQover accuracy and lower DTO than SparseGPT with comparable perplexity and downstream performance.The largest reported gains are Phi-4-Mini from 43.36% to 48.51% and Qwen3-8B from 61.84% to 65.00%.
  • Category-Level UnQover Evaluation: For both LLaMA-3.1-8B and Qwen-2.5-7B, Debias-SparseGPT yields higher UnQover accuracy than SparseGPT across all four evaluated categories.LLaMA’s largest gains are religion, 29.2% to 58.5%, and race, 35.5% to 62.3%; Qwen’s are gender, 54.0% to 60.8%, and religion, 90.4% to 93.4%.
  • Predictive Uncertainty: Qwen exhibits higher confidence and lower predictive entropy than LLaMA on UnQover, while LLaMA shows larger changes after pruning.The results are consistent with lower-confidence examples being more susceptible to compression-induced changes.
  • Results Across Sparsity Regimes: Under 25%, 50%, 1:4, and 2:4 sparsity, Debias-SparseGPT generally preserves SparseGPT downstream performance while improving UnQover accuracy.For LLaMA under 1:4 sparsity, UnQover increases from 35.60% to 60.46%; performance degrades more strongly under 2:4 sparsity.
  • Calibration Under 2:4 Sparsity: 256 UltraChat examples raise Debias-SparseGPT’s Qwen performance under 2:4 sparsity to 54.17% MMLU and 47.26% UnQover, from 48.16% and 24.94% with StereoSet alone.DTO decreases from 0.645 to 0.494.

E Calibration Data Ablation Experiments

Calibration-data ablations show that paired-data choice and context-rich examples materially affect the fairness-performance trade-off under aggressive 2:4 sparsity. Moderate UltraChat augmentation improves results, while larger additions do not consistently improve the trade-off.

  • Choice of Calibration Data: Debias-SparseGPT’s objective uses paired stereotypical and anti-stereotypical StereoSet sentences to construct its bias-aware Hessian.These minimal contrastive pairs are used for calibration.
  • UltraChat Ablation: 256 UltraChat examples produce 13.37 perplexity, 54.17% MMLU accuracy, 47.26% UnQover accuracy, and 0.494 DTO.The 64-example setting instead has the highest average UnQover accuracy, 50.32%, and lowest DTO, 0.486.
  • UltraChat Ablation: Adding 1024 UltraChat examples worsens the reported trade-off relative to 256 examples, with perplexity increasing to 13.65, MMLU decreasing to 51.89%, and DTO increasing to 0.501.UnQover accuracy decreases to 48.04%.
  • StereoSet and CrowS-Pairs Calibration Ablation: Full StereoSet calibration outperforms CrowS-Pairs across all reported metrics: perplexity is 13.37 versus 14.68, MMLU is 54.17% versus 53.09%, and UnQover is 47.26% versus 39.80%.DTO is 0.494 for full StereoSet and 0.540 for CrowS-Pairs.
  • StereoSet and CrowS-Pairs Calibration Ablation: Religion-specific StereoSet calibration achieves the strongest fairness-performance trade-off among the evaluated paired calibration sets.It yields the highest average UnQover accuracy and lowest DTO among category-specific StereoSet subsets.

F Safety Evaluation

Safety evaluations indicate that Debias-SparseGPT does not introduce an adverse safety trade-off relative to SparseGPT in the tested 1:4 setting. Additional pruning analysis shows that its altered decisions are concentrated primarily in attention output projections.

  • Evaluation Setup: Safety testing uses deterministic greedy decoding at temperature zero and compares both pruning methods under 1:4 semi-structured sparsity.Unsafe responses are classified by the Guard model.
  • Toxicity and Safety Evaluation: Debias-SparseGPT yields lower unsafe-response rates than SparseGPT across all evaluated model families and both RealToxicityPrompts and HarmBench.The largest reported HarmBench reduction is Vicuna-1.5-7B, from 0.510 to 0.425.
  • Toxicity and Safety Evaluation: For LLaMA-3.1-8B on HarmBench, the unsafe-response rate decreases from 0.265 with SparseGPT to 0.190 with Debias-SparseGPT.Lower unsafe-response rates are better.
  • Toxicity and Safety Evaluation: The reported safety results associate the bias-aware pruning objective with improved performance relative to SparseGPT without an adverse safety trade-off in the evaluated settings.This is the authors’ stated interpretation of the toxicity and HarmBench comparisons.
  • Pruning-Pattern Analysis: Among the 20 matrices with the largest 2:4 pattern disagreement, 16 are attention output projections, indicating non-uniform effects concentrated primarily in self_attn.o_proj.The comparison uses Qwen-2.5-7B-IT compressed with the same StereoSet and UltraChat calibration data.
Loading 2609.02496v1…