PaperlayerView exact revision on arXiv

Source-linked AI summary

Fairness Pruning: Locating Demographic Bias in GLU-MLP Layers via Differential Activations

Pere Martra, Eugenio Martínez Cámara, Alfonso Ureña López

arXiv:2607.28319v1cs.CLcs.CYcs.LG

TL;DR

Demographic bias is encoded in LLM representations, but evidence on locating and selectively intervening on its underlying circuits remains limited. Fairness Pruning uses differential activations to identify demographic-responsive neurons, and zeroing them changes bias responses while retaining general capabilities.

  • Problem

    Demographic biases are structurally encoded in pretrained LLM weights, motivating methods to localize and selectively target their representations.

  • Method

    Fairness Pruning uses minimally contrastive prompts and differential activation magnitudes to identify demographic-responsive neurons for inference-time zeroing.

  • Results

    Zeroing identified neurons localizes demographic bias signals and produces non-monotonic, sign-inverting aggregate effects while retaining 99.49% of reasoning and general knowledge capabilities.

  • Takeaways & Limitations

    Demographic attribute processing and general capabilities operate on dissociable circuits, supporting future directional activation steering beyond blind zeroing.

  • Takeaways & Limitations

    Evaluation uses explicit demographic attributes in BBQ and EsBBQ, so the observed destabilization may not generalize to indirect biases or open-ended generation.

Abstract

from arXiv · show

This work presents Fairness Pruning, a lightweight structural intervention method designed for the management and future mitigation of demographic bias in large language models (LLMs). As a foundational empirical validation of this method, this work focuses on causal bias localization. Using minimally contrastive prompt pairs and inference-time activation capture, the method identifies neurons that react differentially when processing demographic attributes in GLU architectures, evaluating the signal at the down_proj input. Empirical evaluation was conducted on models of up to 3 billion parameters (Llama-3.2 family and Salamandra-2B), combining standardized benchmark evaluation with qualitative text generation experiments. Results demonstrate that zeroing the identified neurons alters how the model responds to associated demographic variables. However, rather than producing flat mitigation, the intervention causes bidirectional bias destabilization: because BiasScore is unsigned, candidate sets mix neurons that push toward and against the stereotype, and the net effect on aggregate bias depends on which sign dominates. The intervention is extremely surgical: zeroing at most 40 neurons in Llama-3.2-1B (less than 0.031% of total MLP width) achieves a mean retention of 99.49% in reasoning and general knowledge capabilities. These findings empirically confirm that demographic bias processing and model capabilities operate on dissociable circuits, establishing the methodological foundations for transitioning from blind zeroing toward directional behavior modulation.

I. INTRODUCTION · II. RELATED WORK · A. Demographic Bias in LLMs and Mitigation Strategies

The paper frames demographic bias as a structurally encoded problem requiring targeted, affordable localization rather than broad intervention. It reviews existing measurement and mitigation approaches, motivating differential activations as a surgical alternative to SAE-based or weight-wide methods.

  • I. INTRODUCTION: LLMs reproduce and sometimes amplify demographic biases because pretraining captures historical inequalities present in their data.These biases are encoded in learned representations and weights rather than arising only from isolated errors.
  • I. INTRODUCTION: Deployed in hiring, healthcare, education, and finance, structurally encoded bias can become a vector for discrimination at scale.The paper emphasizes that biased model behavior has consequences beyond an abstract technical property.
  • I. INTRODUCTION: Existing responses act on data, weights, or representations, but none distinguishes bias-encoding components from those sustaining general capabilities.The reviewed lifecycle phases are pretraining-data filtering or rebalancing, fine-tuning or alignment, and postprocessing representation modification.
  • I. INTRODUCTION: Sparse Autoencoders can isolate interpretable concepts precisely, but their additional training requirements make them computationally expensive for resource-constrained research.The proposed method addresses this gap by locating demographic-sensitive neurons without additional training and at consumer-hardware-compatible cost.
  • I. INTRODUCTION: The paper hypothesizes that demographic bias is concentrated in identifiable neurons, enabling selective intervention without affecting general capabilities.This hypothesis builds on observed pruning dynamics and prior work on functional specialization in MLP and GLU-based architectures.
  • I. INTRODUCTION: Differential activations from minimally contrastive prompt pairs locate neurons responding differently to a demographic attribute without additional training.The method is implemented in OptiPFair and uses public English and Spanish contrastive-prompt datasets aligned with BBQ and EsBBQ.
  • A. Demographic Bias in LLMs and Mitigation Strategies: BBQ separates stereotypical tendency from general comprehension through ambiguous and disambiguated multiple-choice scenarios and serves as the de facto English bias-evaluation standard.The passage also notes limitations associated with its multiple-choice form.
  • A. Demographic Bias in LLMs and Mitigation Strategies: INLP removes protected-attribute directions from representations, but modern generative applicability is limited and iterative projections may remove correlated non-target information without scope guarantees.This limitation motivates structural methods that can target specific internal components more directly.

B. Pruning and Mechanistic Interpretability

Prior pruning work primarily treats structured sparsity as compression, while mechanistic interpretability motivates locating specialized circuits that may process demographic attributes. Evidence from stereotypical-relation studies shows that bias can concentrate in a few causally relevant neurons whose removal reduces stereotyped continuations.

  • Structured pruning: 50% of weights in GPTscale models can be pruned with minimal degradation using second-order approximations, establishing structured pruning primarily as compression.Wanda combines weight magnitude with activation statistics, while LLM-Pruner extends pruning to neuron groups.
  • Width-pruning specialization: Factual-knowledge capabilities degrade monotonically as expansion ratio decreases, whereas algorithmic capabilities and instruction-following are preserved under width pruning guided by PPM.PPM denotes Peak-to-Peak Magnitude, capturing each neuron’s full weight dynamic range.
  • Neuron specialization: Specialization of factual knowledge and instruction processing makes neuron specialization for demographic attributes plausible within the model’s internal representations.This hypothesis follows from the reported capability dichotomy under width pruning.
  • Mechanistic interpretability: Mechanistic interpretability frames transformer behavior through circuits and attention subgraphs, while Sparse Autoencoders address superposition and polysemantic neuron representations.The cited framework supports causal investigation of internal model processing, including previously opaque MLP layers.
  • Causal bias localization: 1–3 neurons per relation concentrate bias in BERT stereotypical relations, and zeroing them consistently reduces confidence in producing those stereotyped continuations.Voria et al. identify causally contributing neurons using integrated gradients on biased triples from CrowS-Pairs.

III. FAIRNESS PRUNING BASED ON DIFFERENTIAL ACTIVATION … C. Detection of Biased Neurons

Fairness Pruning identifies demographic-sensitive neurons through contrastive activation analysis in GLU-based MLPs, then combines bias sensitivity with structural importance to select neurons for zeroing. The pipeline uses controlled prompt pairs, neuron-level scoring, selective intervention, and evaluation of bias and capabilities.

  • III. FAIRNESS PRUNING BASED ON DIFFERENTIAL ACTIVATION: Fairness Pruning uses controlled prompt pairs, neuron-level activation analysis, bias and structural scoring, and selective zeroing to test causal demographic-bias processing.The method is designed for GLU-based MLP architectures.
  • A. Overview of the Pipeline: The pipeline computes BiasScore from differential activations and FairnessPruningScore from combined bias sensitivity and structural importance before evaluating bias benchmarks and general capabilities.Selected neurons are zeroed out for intervention and subsequent evaluation.
  • B. Prompt Pair Datasets: The datasets contrast prompts differing in one demographic attribute and cover Age, Gender, PhysicalAppearance, RaceEthnicity, and Religion.The English and Spanish datasets were published on Hugging Face as fairness-pruning-pairs-en2 and fairness-pruning-pairs-es3.
  • B. Prompt Pair Datasets: 14 attribute pairs and 70 prompt pairs comprise the English dataset, while 20 attribute pairs and 100 prompt pairs comprise the Spanish dataset.Attribute_1 denotes the majority or nonstereotyped attribute, whereas attribute_2 denotes the attribute historically associated with negative bias.
  • B. Prompt Pair Datasets: Each prompt pair must tokenize to exactly the same number of tokens for position-by-position activation comparison, with runtime validation required for Salamandra-2B.Candidate pairs were prevalidated with the Llama-3.2-1B tokenizer.
  • B. Prompt Pair Datasets: Five templates cover occupational, institutional, healthcare, social, and educational contexts, with incomplete sentences eliciting potentially biased continuations.For Spanish Gender pairs, validation uses the full noun phrase to respect grammatical gender agreement.
  • C. Detection of Biased Neurons: At the down_proj input, BiasScore is the absolute difference between per-sequence mean activations for paired prompts, averaged across category prompt pairs; higher values indicate differential demographic responses.This location captures the integrated SiLU(gate_proj(x)) × up_proj(x) signal in SwiGLU architectures.
  • C. Detection of Biased Neurons: FairnessPruningScore combines normalized bias sensitivity with structural importance, using PPM dynamic weight magnitude; exploratory experiments set α = 0.8.Structural importance is intended to reduce capability degradation during more aggressive neuron removal.

D. Pruning Strategy

The pruning strategy selectively zeroes BiasScore-identified neurons across all three GLU-MLP projections to test whether they causally mediate responses to demographic attributes. Because the architecture remains unchanged, the intervention supports direct evaluation and production-inference compatibility.

  • Neuron zeroing: Each selected neuron is zeroed symmetrically by setting its gate_proj.weight row, up_proj.weight row, and down_proj.weight column to zero.This prevents the neuron from contributing to the residual stream through any defining projection.
  • Causal validation: The intervention tests causal relevance: measurable disruption of responses to demographic attributes validates the localization of the identified neurons.The objective is verification rather than directing bias toward a specific value.
  • Evaluation compatibility: The model architecture is unchanged, enabling direct lm_eval evaluation and compatibility with production inference engines without additional adaptations.The pruning operates on parameters rather than requiring architectural modifications.

E. Evaluation Framework · IV. EXPERIMENTAL SETUP

The evaluation measures pruning effects independently on general capabilities and demographic bias, using selected benchmarks, while experiments span three models and controlled intervention settings. BiasScore-based neuron selection, reproducible execution, and public artifacts define the experimental setup.

  • E. Evaluation Framework: General capabilities and demographic bias are measured before and after each pruning intervention to quantify effects on both dimensions.This establishes two independent evaluation axes.
  • E. Evaluation Framework: Capability evaluation uses lm-evaluationharness with benchmarks covering language modeling, reasoning, general knowledge, and instruction following in English and Spanish.The full base-model battery exposed near-random performance on several Llama-3.2-1B benchmarks.
  • E. Evaluation Framework: Intervened models are evaluated on WikiText, MMLU, ARC-Challenge, and English and Spanish HellaSwag, selected where base performance exceeds chance.Reported baselines include MMLU (32.0%, 5-shot, +7pp above chance), ARC-Challenge (37.2% acc norm, +12pp above chance), and English HellaSwag (64.2% acc norm, +39pp above chance).
  • E. Evaluation Framework: BBQ and EsBBQ measure four per-category metrics, with ambiguous-context bias score primary because it captures stereotype fallback when information is insufficient.The metrics are ambiguous-context accuracy, disambiguated-context accuracy, ambiguous-context bias score, and disambiguated-context bias score.
  • IV. EXPERIMENTAL SETUP: Experiments use Llama-3.2-1B as the primary development model, Llama-3.2-3B as a scaling reference, and Salamandra-2B for cross-lingual validation.The setup tests whether localization patterns and zeroing behavior remain consistent with increased parameters and across languages.
  • IV. EXPERIMENTAL SETUP: Candidate neurons are selected by BiasScore over down_proj_input activations, while FairnessPruningScore is omitted because interventions zero at most 40 neurons per layer.The intervention regime is described as at most 40 neurons out of thousands per layer.
  • IV. EXPERIMENTAL SETUP: All experiments run on Google Colab Pro with an NVIDIA L4 GPU in bfloat16, fixed seed 42, greedy decoding, pinned dependencies, and publicly released code, datasets, and results.Greedy decoding uses do_sample=False, and reproducibility is supported by set_seed(42) and exact dependency versions.

V. RESULTS · A. Baseline Capabilities and Bias

The results establish baseline capability and demographic-bias profiles for the unmodified models before analyzing localization and intervention effects. Baselines show model-, language-, and category-specific performance and bias patterns, including both stereotypical and anti-stereotypical tendencies.

  • V. RESULTS: The results are organized around baseline capabilities and bias, neural-space bias localization, and subsequent neuron-zeroing experiments.Baseline measurements provide the reference for quantifying intervention effects.
  • A. Baseline Capabilities and Bias: The control battery contains five benchmarks, while eleven tasks were evaluated overall; near-chance Llama-3.2-1B performance excludes five tasks from retention analysis.The excluded tasks are GSM8K, IFEval, ARC-ES, Belebele, and TruthfulQA MC2.
  • A. Baseline Capabilities and Bias: Llama-3.2-3B is consistently superior across evaluated dimensions, while Salamandra-2B displays asymmetric language-dependent behavior.The supplied passage attributes the 3B profile to its larger parameter count, but the Salamandra-2B description is truncated.
  • A. Baseline Capabilities and Bias: 25.1% MMLU places Salamandra-2B at chance level, whereas HellaSwag-ES exceeds Llama-3.2-1B by 4.87pp.The passage links this contrast to stronger Spanish competence in commonsense reasoning and heavier Spanish pretraining emphasis.
  • A. Baseline Capabilities and Bias: Religion and Age have Llama-1B bias scores of 5.33% and 7.50%, while RaceEthnicity is negative for Llama-1B (−1.66%) and Salamandra-2B (−0.52%).Positive values indicate alignment with the historical stereotype; negative values indicate anti-stereotypical inclination.
  • A. Baseline Capabilities and Bias: Across BBQ, accuracy is lower in ambiguous contexts and substantially higher in disambiguated contexts.The BBQ bias score uses positive values for historical-stereotype alignment and negative values for anti-stereotypical inclination.
  • A. Baseline Capabilities and Bias: 71.0% is Llama-3.2-3B’s disambiguated EsBBQ accuracy, compared with 16.1% in ambiguous context; Spanish Religion bias_ambig is negative in all three models.The Religion values are −3.24% for Llama-1B, −5.56% for Llama-3B, and −9.72% for Salamandra-2B.

B. Bias Localization: Depth, Neural Structure, and Circuit Specificity … 3) Category Overlap and Circuit Specificity:

Bias signals concentrate in final-layer down_proj_input activations, while candidate neurons remain spatially scattered and largely category-specific. Cross-language transfer is moderate, despite consistent depth localization across English and Spanish.

  • 1) Depth Localization:: Across all 30 model-language-category combinations, down_proj_input reaches its highest mean bias signal in the model’s final layer.The maxima are L15 for Llama-3.2-1B, L27 for Llama-3.2-3B, and L23 for Salamandra-2B.
  • 1) Depth Localization:: Gender records a down_proj_input mean 3–11 times lower than the dominant category and compresses least under SwiGLU filtering.Gender contributes 12 distinct layers in Llama-3.2-1B English and 18 in Llama-3.2-3B English, versus 6–10 for other categories.
  • 2) Spatial Structure Within Layers:: Candidate neurons in final layers are scattered across the full intermediate dimension rather than forming contiguous blocks.This distribution rules out block-based pruning strategies based on index ranges.
  • 3) Category Overlap and Circuit Specificity:: At Top-1%, pairwise category overlap in Llama models ranges from 0.204–0.324, leaving 66%–88% of candidate neurons unique to each category.Reported medians are 0.302 for Llama-3.2-1B English, 0.249 for Llama-3.2-1B Spanish, 0.293 for Llama-3.2-3B English, and 0.280 for Llama-3.2-3B Spanish.
  • 3) Category Overlap and Circuit Specificity:: Gender pairs average 0.157 versus 0.266 for non-Gender pairs in Llama-3.2-3B English at Top-0.1%.Across Llama models and thresholds, Gender-pair mean Jaccard values are 0.02–0.11 points below non-Gender pairs.
  • 3) Category Overlap and Circuit Specificity:: At Top-0.1%, the N-way intersection contains 13–42 neurons, with 69%–100% in the final layer; at Top-1%, it grows to 239–489 neurons for Llama and 356–404 for Salamandra.At Top-1%, 95%–99% of shared neurons remain concentrated in deeper layers.
  • 3) Category Overlap and Circuit Specificity:: Salamandra-2B has Top-1% median Jaccard values of 0.332 in English and 0.345 in Spanish, versus 0.249–0.302 in Llama.At Top-0.1%, its 0.344–0.550 range is attributed to 85%–98% of candidates concentrating in L23 across categories.
  • 3) Category Overlap and Circuit Specificity:: At Top-1%, EN-ES Jaccard ranges from 0.130–0.200 in Llama-3.2-1B and 0.130–0.187 in Llama-3.2-3B, with Religion highest at 0.200 and 0.187.Final-layer peak localization is consistent across all three models in both languages, while concrete neuron transfer remains moderate.

4) Cross-Lingual Consistency: · C. Intervention Effects: Zeroing Results on Bias and Capabilities · 1) Qualitative Analysis of Generated Text:

Cross-lingual circuit transfer varies by model and category, while zeroing localized neurons produces causal but unstable demographic-behavior changes rather than uniformly reducing bias. Qualitative examples and aggregate English results show bidirectional effects with limited capability-separation implications.

  • 4) Cross-Lingual Consistency:: 0.130 is the Top-1% EN-ES Jaccard for Llama-3.2-1B and Llama-3.2-3B, showing signal amplitude and cross-lingual transferability can dissociate.A category may trigger strong differential activations in one language without the same neurons responding in the other.
  • 4) Cross-Lingual Consistency:: 0.187–0.218 is Salamandra-2B’s Top-1% cross-lingual range across categories, with no category clearly more or less transferable.The passage relates this uniformity to more balanced bilingual pretraining and more symmetrically distributed demographic representations.
  • C. Intervention Effects: Zeroing Results on Bias and Capabilities: Zeroing localized neurons tests whether demographic-bias circuits are causal and whether bias can be manipulated without damaging unrelated capabilities.The intervention targets neurons whose activity changes when demographic attributes vary in prompts.
  • C. Intervention Effects: Zeroing Results on Bias and Capabilities: The intervention is evaluated through generated-text examples, BBQ and EsBBQ measurements, and standard capability benchmarks.The qualitative analysis is illustrative rather than representative, while later sections provide systematic evaluation.
  • 1) Qualitative Analysis of Generated Text:: Five examples across two models and two languages illustrate bias reduction, transformation, and amplification after zeroing selected neurons.The examples include Llama-3.2-1B experiments in English and Salamandra-2B experiments in Spanish.
  • 1) Qualitative Analysis of Generated Text:: Zeroing the same neurons can reduce bias for one prompt while amplifying it for another, or transform physical violence into institutional discrimination.One example reports explicit racial hostility after neurons that reduced bias in another example were zeroed.
  • 1) Qualitative Analysis of Generated Text:: The qualitative data confirm bidirectional destabilization rather than systematic aggregate-bias reduction after zeroing the highest-BiasScore neurons.The passage presents this as the predicted consequence of treating those neurons as regulators of demographic expression.
  • 1) Qualitative Analysis of Generated Text:: 0.80pp and 0.50pp are the English BBQ variation ranges for ambiguous and disambiguated contexts, over baselines of 1.70% and 1.75%.No experiment lowers both metrics simultaneously, and six of eight cases show opposite movement between contexts.

2) BBQ/EsBBQ Results:

BBQ/EsBBQ results show that neuron zeroing produces category-level and cross-category bias shifts rather than systematic aggregate improvement. The strongest causal effect is a 5.5pp Religion shift with sign inversion in Llama-3B, while Spanish results show destabilization around an anti-stereotypical baseline.

  • English BBQ: In Llama-1B, Religion Top-5 lowers bias_ambig from 5.33% to 3.67% (−1.66pp) but raises bias_disambig from 3.74% to 4.63% (+0.89pp).The opposing movements show why aggregated category scores can mask target-category changes.
  • English BBQ: In Llama-3B, zeroing 20 neurons reduces Religion bias_ambig from 6.00% to 1.50% (−4.50pp) and shifts bias_disambig from +3.84% to −1.67%.The disambiguated score reverses from stereotypical to anti-stereotypical bias, producing a 5.5pp benchmark behavior change.
  • Spanish EsBBQ: In Spanish EsBBQ, Religion bias_ambig moves from −3.24% to −5.56% (−2.32pp) under Religion Top-5, deepening an existing anti-stereotypical tendency.A more negative score after zeroing does not indicate bias reduction when the baseline is already anti-stereotypical.
  • Spanish EsBBQ: Salamandra-2B Religion bias_ambig oscillates between −14.81% for Age Top-5 and −7.87% for Age Top-20.Age and Gender interventions also shift Religion measurements, indicating cross-category effects associated with circuit overlap.
  • Synthesis: Aggregate scores do not improve systematically, but category-level movements confirm causal relevance under the unsigned neuron-selection criterion.The Religion Top-20 experiment in Llama-3B provides the clearest example through a 4.5pp shift and sign reversal.

3) General Capabilities Retention:

Zeroing 5–40 differentially activated neurons preserves Llama-3.2-1B general capabilities, with all 20 task retentions between 97.93% and 101.34% and a 99.49% mean. Results on Llama-3.2-3B show the same pattern, supporting the use of BiasScore-based selection as a conservative intervention.

  • Llama-3.2-1B: 0.031% of total intermediate positions were zeroed in the most aggressive Llama-3.2-1B experiment, removing 40 of 131,072 neurons.The experiments zeroed between 5 and 40 neurons across 16 layers of 8,192 neurons each.
  • Llama-3.2-1B: 99.49% mean retention was measured across 20 general-capability results for Llama-3.2-1B, whose individual retentions ranged from 97.93% to 101.34%.Values above 100% are interpreted as statistical evaluation noise rather than capability gains.
  • Per-task retention: 99.78%–100.03% English and 99.37%–100.19% Spanish HellaSwag retention remained practically unchanged across all experiments.MMLU showed the widest variation, including Religion·Top-5 at 101.34%, which reflects statistical evaluation noise.
  • Llama-3.2-3B: 97.57% WikiText retention was the lowest observed value in the Llama-3B experiments, corresponding to 9.78 vs 9.54 and a 0.24 perplexity-point degradation.The RaceEthnicity·Top-20 intervention zeroed 20 neurons out of 229,376 total positions; MMLU was 57.86% vs 57.83% baseline.
  • Interpretation: BiasScore-only selection was sufficiently conservative because dispersed neurons sensitive to demographic attributes did not constitute a bottleneck for general capabilities.The passage attributes this result to selecting neurons for differential sensitivity rather than structural importance.

VI. DISCUSSION, LIMITATIONS & CONCLUSIONS … C. The Dataset as a Design Variable

The experiments indicate that BiasScore identifies regulatory neurons whose effects on demographic expression are bidirectional and whose removal leaves general capabilities largely intact. The method’s resolution and future direction depend on designing contrastive datasets that capture bias direction and target increasingly specific behaviors.

  • A. The Regulatory Role of Neurons in Bias Modulation: BiasScore-identified neurons act as regulators of demographic expression rather than stereotype repositories, with bidirectional destabilization in six of eight Llama-3.2-1B experiments.The Religion·Top-20 experiment in Llama-3.2-3B also showed complete sign inversion in the disambiguated context.
  • A. The Regulatory Role of Neurons in Bias Modulation: BiasScore measures differential-response magnitude but not direction, making zeroing effects on final behavior inherently unpredictable.A high-scoring neuron may amplify or suppress bias, but the unsigned score does not distinguish these roles.
  • A. The Regulatory Role of Neurons in Bias Modulation: For L15:166, individual zeroing (n=1) changed BBQ by ∆+0.12pp in ambiguous bias and ∆−0.07pp in disambiguated bias, while 5 of 9 qualitative generations matched the base model.L15:166 was the most dominant neuron in the grid, with a 32% gap over the second Religion candidate, yet its removal produced little variation.
  • A. The Regulatory Role of Neurons in Bias Modulation: Zeroing at most 40 neurons, or 0.031% of total MLP width in the most aggressive case, preserved an average 99.49% retention across the full benchmark battery.The retained metrics included language modeling, general knowledge, and reasoning, supporting dissociation between demographic processing and general capabilities.
  • B. Efficiency of the Proposed Method vs. Brute Force: Contrastive activation localization is computationally more targeted than fine-tuning, which updates all model weights and can degrade unintended capabilities or cause catastrophic forgetting.The method is presented as an alternative to brute-force interventions that do not distinguish bias-related weights from capability-supporting weights.
  • C. The Dataset as a Design Variable: The contrastive prompt-pair dataset determines the method’s localization resolution and scope, while symmetric pairs yield an honest but directionless absolute-difference BiasScore.This design limits separation of neurons that amplify bias from those that suppress it.
  • C. The Dataset as a Design Variable: An asymmetric corpus with known biased and neutral prompts would enable signed scores, distinguish amplifying from suppressing neurons, and replace zeroing with selective per-neuron multiplicative scaling.The proposed transition would turn the method into targeted activation steering rather than circuit removal.
  • C. The Dataset as a Design Variable: Narrow, context-specific datasets could localize neurons responding to precise behaviors, enabling interventions on specific biases instead of broad demographic categories.Broad datasets produce neurons responsive to general demographic categories, whereas employment-discrimination pairs could target a concrete scenario.

D. Study Limitations · E. Conclusions · F. Future Work

The study’s conclusions are bounded by benchmark, architectural, tokenization, and intervention-scale limitations. Its results support localized demographic-bias circuits, dissociable from general capabilities, and motivate signed, directional steering plus broader behavioral and architectural analyses.

  • D. Study Limitations: BBQ and EsBBQ evaluate explicit demographic attributes in multiple-choice formats, leaving indirect signals such as names, cultural behaviors, and occupations outside the study’s coverage.The passages state that these benchmarks do not capture biases emerging from indirect signals and do not guarantee identical destabilization dynamics beyond their design.
  • D. Study Limitations: The architectural analysis covers models under 4 billion parameters with SwiGLU MLP blocks and excludes Transformer attention mechanisms.Evaluated models include Llama-3.2-1B, Llama-3.2-3B, and Salamandra-2B.
  • D. Study Limitations: Exact tokenization parity in contrastive pairs restricts viable attributes, particularly in highly morphologically inflected languages such as Spanish.The limitation arises from the localization method’s requirement that contrastive pairs tokenize identically.
  • D. Study Limitations: 40 neurons was the maximum intervention scale, leaving larger-scale network behavior for future investigation.The zeroing experiments used conservative parameters and did not explore larger intervention scales.
  • E. Conclusions: The results confirm that demographic bias is robustly localized in the final layer through specific, separable circuits for each demographic category.This finding supports the hypothesis that demographic bias is concentrated in identifiable neurons rather than diffusely distributed throughout the network.
  • E. Conclusions: Zeroing produces non-monotonic, sign-inverting aggregate bias effects as intervention scale grows because the unsigned BiasScore captures differential-response magnitude but not direction.The passage contrasts scalar metrics such as BBQ with the qualitative richness of open generation.
  • E. Conclusions: 99.49% average retention in reasoning and general knowledge shows that demographic-attribute processing and general capabilities operate on dissociable circuits.The structural intervention is described as highly localized while preserving this average capability retention.
  • F. Future Work: Future work will use asymmetric corpora for signed BiasScore estimation, hyper-specific datasets for individual behavior localization, and attention-head analysis through a separate pipeline.Signed scores would support directional activation steering, while attention analysis is treated as an independent research direction rather than an incremental extension.
Loading 2607.28319v1…