PaperlayerView exact revision on arXiv

Source-linked AI summary

Reference-Based Bias Detection in LLMs via Relative Representations of Hidden States

Marek Jeliński, Jan Dubiński, Maciej Chrabaszcz, Sebastian Cygert

arXiv:2609.10060v1cs.AI

TL;DR

Output-based bias audits can be costly and may miss internal changes that do not yet appear in generated text. The paper uses relative hidden-state representations to compare an audited model with a reference model in a shared space, where ΔB measures shifts in target-group associations. Across three model families and three benchmarks, ΔB covaried with output-level bias in 15 of 18 settings and reached |r| = 0.84 under full fine-tuning, while remaining a complement to output-based auditing.

  • Problem

    Existing output-based bias audits use costly curated benchmarks or judge models and may miss internal changes that do not appear in generated text.

  • Method

    Relative representations encode sentences through similarities to fixed anchors, placing audited and reference models in a shared space where ΔB measures target-group shifts toward positive or negative attributes.

  • Results

    15 of 18 settings showed ΔB covarying with output-level bias, reaching |r| = 0.84 under full fine-tuning; thresholding detected increased-bias checkpoints with ROC AUCs of 0.65–0.99.

  • Takeaways & Limitations

    The method provides a usable, robust comparison space for auditing related model variants and is intended to complement rather than replace output-based evaluation.

  • Takeaways & Limitations

    The measure depends on a meaningful reference model, covers English coarse single-axis groups in three 4B–8B decoder-only models, and weakens under parameter-efficient adaptation, especially for Gemma.

Abstract

from arXiv · show

Existing bias auditing methods typically rely on model outputs, requiring costly benchmarks or judge models and potentially missing internal shifts that never appear in generated text. We propose a reference-based method that audits bias in hidden-state representations across related model variants, for example before and after fine-tuning. Because fine-tuning reshapes representation geometry, absolute hidden states are not directly comparable, so we encode each sentence by its similarities to a fixed set of anchor sentences, yielding relative representations in a shared comparison space. There we measure how target groups shift in their association with positive and negative attributes, a quantity we call the Representational Bias Shift $ΔB$. Across three model families and the WildGuardMix, DecodingTrust and ToxiGen benchmarks, $ΔB$ correlates with output-level bias change in 15 of the 18 settings we test, reaching $|r| = 0.84$ ($p < 0.001$) under full fine-tuning and becoming more model-dependent under parameter-efficient adaptation. Thresholding $ΔB$ detects checkpoints whose bias increased with ROC AUC between $0.65$ and $0.99$, and on WildGuardMix and DecodingTrust it separates them better than a SEAT-based baseline for all three families. $ΔB$ is also stable under changes to the anchor set, attribute sets and target templates. Our method requires no task-specific evaluation data and audits a model in about three minutes, using $3$-$50\times$ less compute than the output-level benchmarks considered here. We view it as complementary to output-based auditing rather than a replacement for it.

1 Introduction

The paper addresses the limits of output-based bias auditing by comparing hidden-state changes between related model variants in a shared relative-representation space. Its Representational Bias Shift ΔB tracks output-level bias change in most tested settings and remains robust to several design choices.

  • Motivation: Output-based audits can be costly, difficult to scale, vulnerable to judge bias, and unable to reveal internal changes absent from generations.The paper motivates hidden-state auditing as a complementary way to examine shifts that may precede visible behavioral changes.
  • Method: Relative representations encode sentences by similarities to fixed anchor sentences, placing audited and reference models in a shared comparison space despite reshaped hidden-state geometry.The method then compares target-group associations with positive and negative attribute sets relative to the reference model.
  • Results: ΔB was stable across anchor sets, attribute sets, and target templates, and on WildGuardMix and DecodingTrust it outperformed a SEAT-based baseline across all three model families.The reported robustness comes from ablations and benchmark comparisons.
  • Results: 15 of 18 settings showed ΔB covarying with output-level bias, reaching |r| = 0.84, while thresholding detected increased-bias checkpoints with ROC AUC between 0.65 and 0.99.These evaluations span three model families, three output-level benchmarks, and two fine-tuning regimes.

2 Related Work

The related work situates the paper within research on LLM bias, representation-space association tests, and the contested relationship between intrinsic and downstream bias measures. Prior findings report both weak cross-task correlations and cases where apparent debiasing leaves bias recoverable.

  • Bias in LLMs: LLM bias research covers social-group harms as well as political, geographic, and cultural distortions.The literature defines bias as systematic behavioral distortions involving group preferences, stereotypes, or unfounded assumptions.
  • Representation-based measurement: Representation-space bias measurement developed from WEAT and gender-direction analyses to sentence-level methods such as SEAT.These approaches measure associations directly in representations rather than relying only on generated outputs.
  • Intrinsic versus extrinsic bias: Prior work disputes whether representation-level and downstream-task metrics reliably track one another across tasks and languages.Other studies report that debiasing metrics can hide bias while leaving it recoverable, motivating comparative auditing across model variants.

3 Method

The method measures bias in hidden-state representations by comparing target-group associations with positive and negative attributes, using relative representations to align fine-tuned and reference models. It defines ΔB as the change in mean bias between an audited model and an unmodified reference model.

  • Bias measurement: Bias is estimated from target sentences’ associations with positive and negative attribute sets, with positive or negative values indicating the direction of association.The method uses mean associations across target, positive-attribute, and negative-attribute sentence sets.
  • Absolute versus relative representations: Absolute-embedding bias scores can reflect fine-tuning-induced geometric artefacts because fine-tuning reshapes latent spaces across model variants.This motivates using a comparison space rather than directly comparing raw embeddings.
  • Absolute versus relative representations: Relative representations encode sentences through similarities to shared anchor sentences, making representations comparable across model variants without fitting a cross-model map.The same anchor sentences are encoded separately by each model, so corresponding coordinates retain the same meaning.
  • Reference-model comparison: The Representational Bias Shift ΔB compares mean bias in an audited fine-tuned model with mean bias in an unmodified reference model.Each target group receives its own ΔB, and the method interprets a negative ΔB as movement toward negative attributes, or increased bias.
  • Reference-model comparison: ΔB is treated as a proxy whose validity depends on its empirical relationship to output-level bias, because representational differences alone do not establish discriminatory behaviour.The method therefore frames representation-based auditing as an empirical comparison with behavioural bias rather than a self-sufficient behavioural claim.

4 Results

Across three model families and multiple bias benchmarks, ∆B tracks fine-tuning-induced output-level bias and can identify more harmful checkpoints. The signal is generally stronger under full fine-tuning, remains useful under LoRA, and is robust to several design choices while requiring substantially less compute than output evaluation.

  • Experimental setup: The study compares fine-tuned models with their base models using shared sentence anchors and paired external-bias and ∆B measurements.The evaluation includes full and LoRA fine-tuning, three model families, and WildGuardMix, DecodingTrust, and ToxiGen.
  • WildGuardMix: Full fine-tuning produces significant negative correlations between ∆B and external bias change across all three model families.As checkpoints become more harmful, they move lower in ∆B and farther along the merge spectrum in the same ordering as the external benchmark.
  • WildGuardMix: Thresholding ∆B reaches ROC AUC 0.93 for Mistral, 0.89 for Llama, and 0.78 for Gemma under full fine-tuning.Under LoRA, detection remains 0.78 for Mistral and 0.92 for Llama, while Gemma’s correlation disappears with r = −0.04.
  • DecodingTrust: On DecodingTrust, Mistral and Llama reach r = −0.82 and −0.84, respectively, with Llama detection at ROC AUC 0.91.The pattern extends from harmfulness detection to stereotypical bias, while Gemma is weaker and becomes nonsignificant under LoRA.
  • ToxiGen: On ToxiGen, Llama reaches r = −0.62 and ROC AUC 0.91, while Mistral is nonsignificant under full fine-tuning because toxicity saturates.Under LoRA, agreement is significant for all three families, with detection between 0.69 and 0.86.
  • Robustness and efficiency: ∆B remains stable across attribute sets and target templates, while neutral anchors reach ROC AUC 0.892 at 1k anchors.Relative representations also outperform comparison baselines across threshold sweeps, and auditing takes roughly 3 minutes versus 9–156 minutes for the output benchmarks.

5 Discussion

The paper presents ∆B as a lightweight tool for auditing representation shifts across model versions and fine-tuning side effects. It is intended for detection and monitoring, not mitigation, and a small ∆B does not establish output safety.

  • Contribution: The method detects fine-tuning-induced shifts in hidden-state representations and supports auditing side effects across model versions.The audited model need not originate from the reference model, enabling comparisons across independently trained checkpoints.
  • Scope: ∆B is deliberately a detection and auditing tool rather than a mitigation method.Using it directly as a training objective is less straightforward because minimizing ∆B need not reduce output bias.
  • Caveat: A small ∆B should be interpreted as no detected representational shift, not as evidence that the model is safe.The method is dual-use because its cheap, differentiable signal can itself be optimized against.

6 Conclusions

The method tracks external bias changes across benchmarks and provides a robust comparison space for related model variants, while remaining complementary to output-based evaluation.

  • ΔB tracks external bias changes across all three benchmarks.
  • On WildGuardMix and DecodingTrust, ΔB separates increased-bias checkpoints better than a SEAT-based baseline.
  • Relative representations enable auditing when related models’ hidden spaces are not aligned.
  • The method is robust to anchor choice and template variation but requires a meaningful reference model and weakens under parameter-efficient adaptation, especially for Gemma.
  • The authors position the approach as complementary to, rather than a replacement for, output-based bias evaluation.

Limitations

The method has limitations involving representation-to-behaviour reliability, contextual sensitivity, reference dependence, and evaluation scope.

  • The method may be less reliable when representations depend strongly on prompt design and token position.
  • Because ΔB is relative, it cannot certify either model as unbiased or audit a checkpoint in isolation.
  • The evaluation uses English templates, coarse single-axis groups, three decoder-only models, and synthetically induced harmfulness, leaving other languages, intersectional groups, un named harms, larger-scale models, and naturally occurring fine-tuning outside scope.

Ethical Considerations

The study deliberately induces harmful behaviour to evaluate the auditing method, while acknowledging release, dual-use, and coverage considerations.

  • The study fine-tunes models on a synthetically harmful WildGuardMix split and merges checkpoints into a graded harmful-behaviour spectrum.
  • The authors release the auditing code and sentence sets but not the harmful checkpoints.
  • The method is dual-use because its cheap, differentiable signal can be optimised against, and keeping ΔB small need not reduce output bias.
  • WildGuardMix prompts lack target-group labels, so the study maps them to nine topics consolidated from DecodingTrust’s 24 groups for comparable aggregation.

C Fine-Tuning Implementation Details

The implementation evaluates three instruction-tuned models under LoRA and full fine-tuning, constructs graded checkpoints and benchmark scores, and tests robustness across stimulus and pooling choices.

  • Models and training: The experiments use Llama 3.1-8B-Instruct, Mistral-7B-Instruct-v0.3, and Gemma 3-4B-IT.
  • Datasets: Two dataset variants contain 8k examples each: unharmful-only data and an even split of harmful and synthetic examples.
  • Model merging: Seven checkpoints per model and training regime are created by linearly merging unharmful and synthetically harmful checkpoints across five interpolation ratios and two endpoints.
  • Training regimes: Each model is fine-tuned with either LoRA or full fine-tuning using shared three-epoch training settings.
  • Evaluation: WildGuardMix uses five sampled responses per prompt, while ToxiGen samples five continuations and scores the first with TOXIGEN_ROBERTA.
  • Evaluation: DecodingTrust scores stereotype agreement using its stereotype evaluation pipeline.
  • Robustness analyses: Robustness experiments vary attribute wording, target wording, token pooling, and independent fine-tuning runs.
  • Pooling strategy: Relative representations outperform SEAT under every pooling scheme, with RR ranging from 0.882 to 0.964 and SEAT from 0.776 ± 0.013 to 0.791.Mean pooling gives the strongest RR score, while SEAT remains comparatively flat across pooling choices.

E.5 Stability Across Fine-Tuning Runs

Across three random seeds, ΔB remains tightly clustered for Llama fine-tuning on both unharmful and synth datasets, indicating stability with respect to training randomness.

  • The stability analysis covers Llama fine-tuned on the unharmful and synth datasets, with each point representing a social group.
  • Standard deviation is 0.003 across three random seeds for both the unharmful and synth datasets.The experiments compute ΔB across social groups for each fine-tuning run.
  • ΔB values remain tightly clustered across runs, suggesting the metric captures systematic fine-tuning effects rather than incidental training noise.The runs vary initialization, data ordering, and optimization dynamics through different random seeds.

G Method Comparison Scatter Plots

The scatter-plot comparison shows that relative representations provide the strongest directional relationship between ΔB and external bias-score change among the evaluated methods.

  • RR forms the tightest and best-separated relationship between ΔB and ΔBias Score, while Procrustes-SEAT shows no relationship.The comparison includes RR, SEAT, Procrustes-SEAT, and CKA drift on Llama.
  • CKA drift correlates in magnitude but lacks a directional bias-valence axis.Its ROC AUC and correlation are therefore treated as undirected measures in the comparison.

H Fine-tuning Results

The fine-tuning results evaluate ΔB against output-level bias changes across model families, benchmarks, and full or LoRA adaptation settings.

  • Full fine-tuning: Fully fine-tuned models are evaluated against WildGuardMix and DecodingTrust using correlations between ΔBias Score and ΔB plus ROC AUC from thresholding ΔB.Figures 7–10 cover all three model families and provide detailed Llama results.
  • Thresholding ΔB is used to classify harmful and unharmful models or checkpoints across the reported benchmark settings.The corresponding figures show ROC AUC curves or scores for the classifier.
  • LoRA fine-tuning: LoRA results retain the full-fine-tuning relationship, but the signal is noisier and more model-dependent, particularly for Gemma.Relative representations still provide useful discrimination for Mistral and Llama.
  • ToxiGen: ToxiGen relates ΔB to changes in generated toxicity toward individual demographic groups under both full and LoRA fine-tuning.The benchmark is scored at the same demographic granularity at which ΔB is defined.
Loading 2609.10060v1…