Source-linked AI summary
Tracing Stereotypes from Representation to Output in Multilingual LLMs
Ariun-Erdene Tumurchuluun, Yusser Al Ghussin, Pinzhen Chen, Josef van Genabith, Koel Dutta Chowdhury
TL;DR
Multilingual LLMs exhibit stereotype-related behavior that varies across languages, but output evaluations do not reveal where stereotype information is represented or how it influences predictions. This study compares probing, attribution patching, sparse autoencoders, and feature ablation across three models, finding that decodability, output influence, and cross-lingual intervention effects diverge.
Problem
Output-level evaluations document multilingual stereotype variation but do not establish where stereotype-related information is represented or how strongly it influences predictions.
Method
The study compares linear probing, attribution patching, sparse autoencoders, and feature ablation in Llama-3.1-8B, Qwen3-8B, and Gemma-2-9B.
Results
Probe performance peaks 36–53% of model depth earlier than attribution across all three models, while selected sparse features are category-linked but show varying lexical alignment and ablation effects.
Takeaways & Limitations
Decodability, output influence, and cross-lingual ablation effects need to be measured separately because they capture distinct aspects of multilingual stereotype-related behavior.
Takeaways & Limitations
The study is limited to four high-resource languages, 8B–9B models, and SAE suites whose differing configurations confound comparisons and limit universal interpretation of ablation effects.
Abstract
from arXiv · showhide
Multilingual LLMs show stereotype-related behavior that varies across languages, but behavioral scores do not show where the relevant information is represented or how it affects the output. To investigate these internal mechanisms, we compare linear probing, attribution patching, sparse autoencoders (SAEs) and feature ablation in Llama-3.1-8B, Qwen3-8B, and Gemma-2-9B. Probe performance peaks substantially earlier than attribution in all three models, with a separation of 36-53% of model depth. Retained Llama-Scope features often match the social category on which they were selected and form recurring semantic families, but their lexical alignment and ablation effects vary across SAE suites. Only 6-18% of evaluated residual-stream features have language-agnostic effects under our criterion, and none are category-agnostic. Language-agnostic features have larger mean ablation effects in Llama-Scope, but this pattern does not repeat in the other SAE suites. Decodability, output influence, and cross-lingual ablation effects therefore need to be measured separately.
1 Introduction
Multilingual stereotype behavior varies across languages, but output behavior alone cannot reveal whether differences arise from representation, output influence, or both. The study separates these mechanisms and finds distinct depth profiles, category-linked sparse features, and limited cross-lingual transfer.
- Behavioral stereotype differences vary in magnitude and direction across languages, even for parallel prompts and identical social categories.
- Output-level evaluations provide only a partial view because models can retain biased associations despite appearing unbiased under explicit evaluation.
- The study compares layer-wise probing, attribution patching, sparse autoencoders, and feature ablation across three multilingual models, four languages, and six social categories.
- 36–53% of model depth separates peak probe decodability from peak attribution influence across all three models.
- Among 139 glossed Llama-Scope features, 123 match their selected social category and form recurring semantic families, while ablation effects can reduce or increase measured bias.
- 6–18% of evaluated residual-stream features are language-agnostic under the study’s criterion, none are category-agnostic, and larger effects in Llama-Scope do not replicate consistently across suites.
- Decodability, estimated output influence, and cross-lingual or cross-category intervention effects must therefore be measured separately.
2 Background and Related Work
Prior work primarily evaluates multilingual stereotype bias through model outputs, while interpretability methods probe different properties of internal computation. This study addresses whether decodability, causal influence, and intervention effects diverge across languages and model depth.
- Behavioral resources evaluate multilingual stereotypes through ambiguous or disambiguated questions, parallel cross-lingual items, culturally situated stereotypes, and stereotype associations across languages and groups.
- Linear probes measure linearly accessible information, attribution patching estimates internal influence on outputs, and SAEs decompose dense activations into sparse features.
- Multilingual studies report language-specific, language-agnostic, and late language-transition representations, but their behavioral effects across languages remain unresolved.
- Probe accuracy does not establish that decoded information is used for the task, motivating intervention-based interpretability evaluations.
- The study compares probe and attribution profiles, then tests selected SAE features by ablation across languages, social categories, models, and SAE suites.
3 Experimental Setup
The experiments trace stereotype-related information from linear decodability to output influence and feature-level intervention. They use three comparable multilingual models, multilingual benchmark data, sparse features, lexical anchors, and held-out ablations.
- The pipeline traces stereotype-related information from linear decodability to output influence and finally to feature-level intervention.
- Layer-wise logistic-regression probes locate accessible representations, while attribution patching measures their estimated influence across model depth.
- Lexical anchors and available feature glosses relate selected sparse features to social categories, and individual features are ablated on SHADES across languages and categories.
- The study uses Llama-3.1-8B, Gemma-2-9B, and Qwen3-8B, controlling model size while covering three independently developed families.
- MBBQ supplies parallel English, Spanish, Dutch, and Turkish items across six social categories, while held-out SHADES interventions cover English, Spanish, and Dutch.
- A Llama-3.1-8B example shows language-dependent behavior: Dutch selects the stereotype-consistent answer even after disambiguating evidence identifies the grandson.
4 Where Is Stereotype Information Decodable?
Stereotype-related information becomes most linearly decodable in middle layers rather than near the output. Across multilingual models, probe peaks precede late attribution peaks, separating accessibility from estimated output influence.
- Disambiguated MBBQ items form contrastive factual-versus-stereotype-consistent completion pairs for layer-wise probing.
- 36–53% of model depth separates peak probe decodability from peak estimated output influence across the three models.
- All three multilingual models show mid-layer probe peaks: Llama peaks at layer 15, while Qwen and Gemma peak at layers 21–23.
- Llama reaches peak macro-F1 0.845, compared with 0.894 for Qwen and 0.899 for Gemma.
- Permuted-label probes remain at chance, and stronger regularization lowers performance without moving the peak.
- The English-centric OLMo-7B reference has weaker decodability, peaking at macro-F1 0.606 and concentrating more strongly on English.
5 Where Does Stereotype Information Influence the Output?
The study separates where stereotype-related information is decodable from where it most strongly influences model outputs. Across three models, attribution influence peaks substantially later than probe decodability, with model-dependent late-layer profiles.
- The analysis compares complementary clean–corrupted contrasts to test evidence injection and stereotype routing.Evidence injection varies factual evidence, while stereotype routing contrasts counter-stereotypical and pro-stereotypical targets under disambiguated contexts.
- Attribution scores are aggregated by layer, category, and language using a single trapezoidal AUC, while comparisons focus on maximal decodability and attribution.
- Peak decodability precedes peak output influence by 36–53% of model depth across all three models.The comparison concerns peak locations, not the first layers where either signal appears.
- The probe-to-attribution peak separation is 53% for Llama, 48% for Gemma, and 36% for Qwen.
- Attribution magnitude peaks near the output, whereas probe performance peaks in middle layers.Maximal attribution occurs at the final layer in Llama, Qwen, and Gemma.
6 What Do the Selected SAE Features Represent?
The selected SAE features often align with their target social categories and form recurring semantic families, but lexical alignment varies substantially across SAE suites. Feature selection uses contrastive completion differences, while suite comparisons remain confounded by differences in training and dictionary properties.
- The four SAE suites differ in training data, sparsity, dictionary configuration, and language coverage, so suite differences cannot be attributed to training language alone.
- Contrastive selection compares stereotype-consistent and control completions while holding context and demographic entities fixed.The contrast deemphasizes features responding similarly to both completions, including features encoding shared demographic content.
- Lexical anchors are obtained by matching each feature direction to the most similar row of the model’s unembedding matrix.The maximum cosine similarity is used as the feature’s anchor cosine and compared with a model-specific null threshold.
- 123 of 139 glossed Llama-Scope features, or 88.5%, match the social category used for selection.These features also form recurring semantic families, including disability-status and gender-identity subfamilies.
- Lexical alignment varies across SAE suites: 46.9% of 1,741 Llama-Scope features versus 8.8% of 1,841 Llama-Multi features exceed chance.The corresponding Gemma-Scope and Qwen-Multi rates are 41.7% of 2,445 and 0.8% of 738 features, respectively.
- Anchoring in Llama-Scope rises from 4.8% at layer 11 to 74.7% at layer 31 and exceeds 50% by layer 20.
7 What Are the Effects and Transferability of Feature Ablation?
Feature ablations reveal heterogeneous, bidirectional effects that vary across SAE suites, languages, and social categories. Cross-language stability is a minority pattern and does not reliably predict stronger intervention effects.
- Transfer across languages and categories: LSS and CSS quantify cross-language and cross-category variation, but low specificity can also reflect consistently small intervention effects.Therefore, specificity scores should be interpreted together with the underlying Δbias values.
- Intervention effects: Feature ablations can either reduce or increase measured bias, so differential activation does not determine a feature’s causal direction.Negative intervention effects indicate reduced measured bias, while positive effects indicate increases.
- Intervention effects: Layer depth and anchor cosine do not reliably predict signed intervention effects, so feature removal effects must be measured rather than inferred.Depth weakly tracks effect magnitude but not signed effect; anchor cosine is at best weakly predictive.
- Intervention effects: Llama-Scope has the largest mean ablation effect, while Gemma-Scope is smaller and Qwen-Multi and Llama-Multi are approximately zero on average.The reported means are −0.055, −0.016, −0.0008, and +0.0002, respectively.
- Transfer across languages and categories: Only 6–18% of evaluated residual-stream features are language-agnostic, whereas none are category-agnostic under the LSS/CSS criteria.In Llama-Scope, 18.3% are language-agnostic and all evaluated features are category-dependent.
- Transfer across languages and categories: Language-agnostic features have larger effects in Llama-Scope, but this advantage is absent or reversed in the other SAE suites.In Llama-Scope, they reduce measured bias by 46–79% across evaluation languages versus 32–55% for language-dependent features.
8 Discussion and Conclusion
The study separates stereotype-related decodability, estimated output influence, and feature-level intervention across multilingual LLMs. These properties are related but distinct, so localization should precede rather than substitute for intervention analysis.
- Discussion and Conclusion: Probe performance peaks substantially earlier than attribution across Llama, Qwen, and Gemma, with peaks separated by 36–53% of model depth.Decodability peaks in middle layers, whereas estimated output influence peaks near the output.
- Discussion and Conclusion: Selected Llama-Scope features often match their identified social category, but their ablation effects vary considerably in magnitude and direction.Semantic relatedness does not guarantee a fixed behavioral effect after removal.
- Discussion and Conclusion: Only 6–18% of evaluated residual-stream features have language-agnostic intervention effects, while none are category-agnostic under the study’s criterion.The larger language-agnostic effects observed in Llama-Scope do not repeat across the other SAE suites.
- Discussion and Conclusion: Decodability, output influence, and cross-lingual ablation effects should therefore be measured separately rather than treated as interchangeable properties.The paper frames localization as a starting point for intervention analysis.
Limitations
The study’s conclusions are bounded by its language, model-size, SAE, pathway, and bias-category coverage. These constraints limit how broadly its observed separations and transfer patterns can be generalized.
- Scope: The analysis covers four high-resource languages, while SHADES interventions use only three because Turkish is absent from that benchmark.Whether the findings hold in typologically diverse or lower-resource languages remains unexamined.
- Scope: Only 8B–9B parameter models are evaluated, limiting assessment of how the patterns scale with model size.The paper does not establish whether the decodability–causality separation persists at other scales.
- Measurement: SAE suites differ in dictionary size, sparsity, training data, and language coverage, confounding comparisons of SAE quality and multilingual training.The strongest ablation effects occur in Llama-Scope, suggesting partial rather than universal replication.
- Measurement: Individual feature ablation measures causal relevance but does not account for redundancy in bias pathways.The study also leaves intersectional biases such as age × gender unaddressed.
Ethics Statement
The paper analyzes sensitive social-stereotype representations using public benchmarks and released model weights, framing the work as bias diagnosis and mitigation rather than characterization of groups. The appendix organizes detailed results by the pipeline’s stages.
- Ethics Statement: The study covers six sensitive categories and uses only publicly available MBBQ and SHADES benchmarks plus released model weights.No human data was collected.
- Ethics Statement: The authors frame the analysis strictly as a diagnostic tool for detecting and mitigating bias, not as a characterization of groups.This scope is stated in the ethics discussion for the sensitive categories studied.
- Ethics Statement: AI assistants supported coding and paper editing, while the authors conducted the scientific claims, experimental design, and analysis.The statement assigns scientific responsibility to the authors.
- Appendix Overview: The appendix follows the pipeline’s four stages in the same order as the main text.Table 5 maps the detailed results across those stages.
- Appendix Overview: The probing figures report layer-wise macro-F1 across categories, models, and languages, with control bands, chance markers, and probe-norm diagnostics.The figures also show regularization sensitivity and causal intervention pairings.
- Appendix Overview: SAE comparisons span residual-stream and other activation streams, with differences in dictionary width, sparsity, coverage, and training-language composition.These design differences complicate direct comparison across SAE suites.
G Ablation
The ablation analysis separates language and category specificity across SAE features and examines how these groups transfer across languages and categories. Layer depth correlates more strongly with agnostic than specific features.
- Language Specificity Score and Category Specificity Score distributions classify SAE features using a 0.5 threshold.Figure 11 distinguishes agnostic (<0.5) from specific (≥0.5) features for each SAE suite.
- Language-transfer matrices compare mean bias change across discovery and evaluation languages within each LSS group.Rows encode discovery language, columns encode evaluation language, and colour scales are model-specific.
- Category-transfer matrices compare mean bias change across mBBQ discovery categories and evaluation languages within each LSS group.Rows encode discovery category and columns encode evaluation language.
- Layer depth correlates more with agnostic features (r = −0.190) than specific features (r = −0.116).