PaperlayerView exact revision on arXiv

Source-linked AI summary

Aligned but Flattened: Analyzing the Trade-off between Cultural Alignment and Diversity in LLMs

Jingshen Zhang, Shaoyang Xu, Wenxuan Zhang

arXiv:2609.00565v1cs.SIcs.CL

TL;DR

Existing alignment-focused evaluation may overlook whether LLMs preserve cultural diversity. The paper introduces a joint alignment–diversity framework and benchmarks six LLMs on World Values Survey data, finding that fine-tuning consistently improves alignment while producing severe cultural flattening. A mechanistic analysis further examines restricted activation spaces, while the authors limit their empirical scope primarily to supervised fine-tuning and static evaluation.

  • Problem

    Alignment scores alone provide an incomplete picture of cultural fidelity because models can follow dominant response patterns without preserving cultural diversity.

  • Method

    The paper jointly evaluates cultural alignment and diversity, fine-tuning six LLMs on culture-specific World Values Survey data and comparing behavioral shifts with human responses.

  • Results

    Fine-tuning consistently improves cultural alignment but sharply reduces diversity, with models collapsing toward dominant or conservative response patterns.

  • Takeaways & Limitations

    Alignment gains can coincide with cultural flattening and risks of marginalizing under-represented cultural perspectives.

  • Takeaways & Limitations

    The empirical analysis is restricted to supervised fine-tuning and static benchmarks, leaving preference-based methods and dynamic open-ended evaluation untested.

Abstract

from arXiv · show

Cultural fine-tuning has become the de facto paradigm for building culture-aware large language models (LLMs), yet existing optimization exclusively for alignment scores provides an incomplete portrait of cultural fidelity by systematically obscuring inherent cultural diversity. This unidimensional evaluation lens prompts a fundamental question: do models genuinely perceive distinct cultural nuances, or do they merely memorize dominant cultural values? To address this, we propose a synergistic evaluation framework that jointly formalizes cultural alignment and diversity. Through extensive benchmarking of six mainstream LLMs on the World Values Survey, this framework uncovers a systematic and critical trade-off: the pursuit of cultural alignment consistently incurs an acute expense of diversity, leading to severe "cultural flattening." Investigating this behavioral shift, we demonstrate that these superficial alignment gains stem from models artificially anchoring to dominant majorities, converging onto a monolithic response pattern that wipes out the heterogeneous distributions inherent to human groups. Crucially, our mechanistic analysis suggests that this diversity collapse is not merely a behavioral anomaly but more likely a structural consequence of the low-rank bias inherent in neural network optimization. Therefore, our findings expose the limitations of current post-training paradigms and call for a shift toward alignment objectives that preserve cross-cultural pluralism.

1 Introduction

The paper argues that alignment scores alone inadequately capture cultural fidelity because they can obscure diversity, and introduces a joint framework to measure both. Across six LLMs, cultural fine-tuning improves alignment while sharply reducing behavioral diversity, with low-rank optimization offered as a mechanistic account.

  • Motivation: Alignment-only evaluation can miss whether models preserve variation within and across cultural groups.A model may track dominant response patterns while achieving higher aggregate similarity scores.
  • Findings: Across six LLMs, fine-tuning consistently increases cultural alignment while reducing behavioral diversity.
  • Findings: Categorical questions show collapse toward one or two dominant response modes, while ordinal questions show retreat toward a conservative middle option.Both patterns make culturally differentiated personas more homogeneous.
  • Framework: The framework jointly measures whether model responses match human values and preserve cross-cultural variation.
  • Mechanism: The mechanistic analysis links cultural flattening to fine-tuning compressing diverse cultural values into a low-rank subspace.The analysis associates this restricted activation space with reduced preservation of heterogeneous cultural profiles.

2 Related Work

Related work studies cultural alignment and cultural diversity as important but largely separate concerns. This paper shifts the reference point for diversity from human responses to differences among model behaviors, enabling joint analysis of their interaction.

  • Cultural Alignment: Prior work explores cultural awareness through in-context learning, supervised fine-tuning, and reinforcement-learning preference optimization.
  • Positioning: The paper formalizes diversity as discrepancies across model behaviors rather than discrepancies from human responses.This reframing enables systematic analysis of the alignment–diversity interplay.
  • Cultural Diversity: Recent studies also examine whether deployed models reproduce diverse human conceptual and behavioral distributions.

3 Measuring the Alignment–Diversity Trade–off

The paper defines cultural alignment as fidelity to a target group and cultural diversity as behavioral distinction across groups. Both objectives use a shared similarity-based framework, while diversity must remain grounded in human behavioral distributions.

  • Evaluation Setup: The evaluation conditions models on culture-specific personas and compares their survey responses with human reference populations.
  • Alignment: Cultural Alignment measures how fully a model reproduces the behavioral tendencies of its target sociodemographic group.A(M) compares model responses M(q|p, c) with human references H(q|p, c) through similarity S(·, ·).
  • Diversity: Cultural Diversity measures whether responses remain behaviorally distinguishable across different cultural subgroups.The intended behavior preserves localized variation rather than collapsing into homogeneous outputs.
  • Interpretation: Higher diversity is not automatically more realistic because meaningful variation must remain grounded in human behavioral distributions.Excessive reductions in diversity signal cultural flattening.
  • Unified Measurement: Alignment and diversity are complementary objectives derived from the same similarity function, providing a common basis for comparing fidelity and differentiation.The framework instantiates the shared similarity function using Soft Accuracy.

4 Empirical Evidence of the Alignment–Diversity Trade-off

Across six LLMs evaluated on World Values Survey data, fine-tuning consistently improves cultural alignment while sharply reducing cultural diversity. The resulting convergence favors dominant or centralized response patterns and can marginalize under-represented cohorts.

  • Evaluation setup: Fine-tuning six LLMs on World Values Survey data consistently increased alignment while narrowing behavioral diversity across models.The evaluation used held-out WVS-7 data, six mainstream LLMs, persona prompts, and two supervised fine-tuning variants.
  • Trade-off overview: Alignment gains ranged from 1.7 for Gemma-2 to 11.9 for Mistral, while all fine-tuned diversity scores fell below 11.0 versus the human reference of 37.9.Vanilla baselines remained above 17.5 on diversity, so fine-tuning pushed scores below the lower bound of the base models.
  • Behavioral dynamics: Fine-tuned models often improved alignment by anchoring to dominant response modes rather than preserving heterogeneous cultural behavior.For one seven-option religious-attendance item, outputs concentrated on Option G, the human plurality at 39%, while Qwen-3 SFT-E amplified minority Option A, endorsed by 6% of humans.
  • Majority anchoring: Majority Choice Rate rose above the human reference of 49.98%, while Choice Pool Size collapsed toward 1.0 under SFT and SFT-E.These diagnostics indicate stronger plurality-consensus following alongside severe behavioral homogenization.
  • Bias and risk: Fine-tuning reduced Qwen alignment for a Nigerian persona from 51.52 to 37.59, a relative decrease of 27.04%, illustrating marginalization risk for under-represented cohorts.The authors report that cohort-specific degradation occurred across all model families and warrants attention when the same cohort is systematically affected.

5 Mechanistic Analysis

The mechanistic analysis tests whether the alignment–diversity trade-off reflects a low-rank representational bottleneck. Probing FFN activation patterns and rank-space geometry, it finds compact, largely decoupled representations after cultural fine-tuning.

  • 5.1 Hypothesis and Probing Method: The analysis hypothesizes that low-rank simplicity bias confines fine-tuning updates to a narrow representational subspace.The authors frame this as an interpretive hypothesis rather than a rigorously causally validated derivation.
  • 5.1 Hypothesis and Probing Method: FFN neuron activation patterns serve as a measurable proxy for the effective low-rank structure of cultural representations.For each question and layer, neurons are selected using a threshold based on the K-th largest activation, with K = 100.
  • 5.1 Hypothesis and Probing Method: The cardinality of the aggregated activated-neuron set estimates the breadth of the model’s effective cultural representational space.A smaller |NM| indicates repeated reliance on a narrower set of FFN neurons.
  • 5.2 Empirical Analysis: Figure 4 compares country-level cultural representations before and after fine-tuning using activated-neuron-set overlap and shared rank space.The left panel uses a shared 3D MDS projection, while the right panel shows proportions of shared activation ranks.
  • 5.2 Empirical Analysis: Less than 10% of the W/O activation space is retained across most evaluation settings after fine-tuning.Comparable retention ratios under SFT and SFT-E indicate that this representational shift is largely insensitive to the training corpus.
  • 5.2 Empirical Analysis: For most models, the incremental space occupies less than 50% of W/O dimensions, reaching 25% in Qwen-2.5.Together with the high geometric decoupling between W/O and SFT spaces, this supports a restrictive rank bottleneck interpretation.

6 Conclusion

The paper formalizes cultural alignment and diversity jointly and finds that cultural fine-tuning improves alignment while substantially reducing diversity. Its mechanistic analysis links this pattern to compact, largely decoupled activation subspaces and motivates multicultural alignment that preserves pluralistic representation.

  • 6 Conclusion: The unified framework reveals a systematic alignment–diversity trade-off across LLMs, with improved alignment accompanied by substantial diversity loss.Fine-tuning pushes models toward dominant majority preferences and monolithic response patterns, producing cultural flattening.
  • 6 Conclusion: Mechanistic evidence indicates that cultural fine-tuning operates within a compact, largely decoupled activation subspace.The paper presents this as a structural perspective on the observed reduction in diversity.
  • 6 Conclusion: The findings underscore the need for multicultural alignment strategies that improve cultural fidelity while preserving pluralistic representation.

Limitations

The paper’s limitations concern its restricted alignment-method coverage, static evaluation setting, and lack of a concrete mitigation strategy. It also notes that AI was used only to aid and polish academic writing.

  • Limitations: The empirical analysis evaluates only supervised fine-tuning, leaving preference- and reinforcement-based methods unexamined.The authors cite challenges in constructing reliable culturally grounded preference annotations and reward signals.
  • Limitations: The diagnostic evaluation uses static benchmarks that cannot fully capture context-dependent and evolving cultural values.Future work is proposed in open-ended, interactive environments and for deployment-time effects on representations.
  • Limitations: The work does not develop a concrete strategy to mitigate cultural flattening.The authors suggest investigating culture-aware knowledge distillation in future work.
  • Limitations: AI assistance was limited to academic writing and did not contribute to core research processes.

B Detailed Description and Statistical Analysis of the WVS

The evaluation uses WVS-7 country and respondent data spanning diverse value topics and demographic groups. Persona prompting injects demographic context, conditions behavioral responses, and enforces a single categorical output for deterministic extraction.

  • B.1 Detailed Description of WVS: The survey covers social, cultural, material, governmental, ethical, and economic value domains across 259 questions.Listed topics include attitudes toward social groups, institutions, employment risks, political systems, and religious law.
  • B.1 Detailed Description of WVS: Table 3 maps ISO three-letter country abbreviations to full country names for cross-referencing regional cohorts.
  • B.1 Detailed Description of WVS: WVS-7 data were collected between 2017 and 2021, creating a temporal limitation despite the dataset’s breadth and recency.Sampling covers gender, age, and social class to mitigate representation bias.
  • C Prompt Design: Persona prompting injects demographic profiles such as region, country, age, education, and social class to target localized cultural priors.The prompt also applies behavioral conditioning and restricts outputs to one categorical choice without commentary.

D.1 Country-Level Distribution of Training Samples

The training data spans fourteen target countries and uses cleaned cultural-response data in SFT and SFT-E configurations. The two configurations have nearly identical option-count and answer-label distributions, reducing concerns that later alignment–diversity patterns arise from label-position artifacts.

  • Country-Level Distribution of Training Samples: Training data covers fourteen target countries and includes samples for both SFT and SFT-E configurations after filtering null, empty, and ambiguous entries.The cleaning process removed entries labeled "nonknown" before training.
  • Answer-Option Composition: 96.57% of SFT and 96.54% of SFT-E samples come from questions with two, three, or four answer options.The option-count proportions differ by at most 0.31 percentage points between configurations.
  • Answer-Label Distribution and Balance: Answer labels remain broadly distributed across valid options rather than collapsing onto a single answer position.For four-option questions, each answer represents approximately 21%–29% of samples.
  • Answer-Label Distribution and Balance: Excluding the United States and Australia does not introduce a systematic label shift between SFT and SFT-E.The two configurations have highly similar label distributions despite this exclusion.

E Detailed SFT Implementation

The study fine-tunes backbone models on downstream cultural instruction datasets using SFTTrainer, one epoch, and a linear learning-rate schedule. Evaluation uses greedy decoding with temperature τ = 0, alongside distributed GPU implementations for training and diagnostics.

  • Training Protocol: Backbone models are fine-tuned with SFTTrainer for one epoch using a linear learning-rate scheduler on two NVIDIA A100 GPUs.The protocol targets downstream cultural instruction datasets.
  • Evaluation and Hardware Configuration: Evaluation generation uses greedy decoding with temperature τ = 0 to eliminate downstream stochasticity and ensure strict reproducibility.Training and diagnostic evaluations use a distributed configuration of four NVIDIA Tesla P100 GPUs.

F Alignment–Diversity Correlations Across Tuning Conditions

Across tuning conditions, the alignment–diversity trade-off is most pronounced during the behavioral transition caused by fine-tuning rather than as a static within-condition relationship. The paper connects this flattening to restricted representational capacity, interference among cultural signals, and reduced sensitivity to marginalized cohorts.

  • Alignment–Diversity Correlations: Jointly analyzing W/O and fine-tuned states reveals stronger negative alignment–diversity correlations than comparisons within either tuning condition.Within SFT, SFT-E, and their pooled cohort, the reported relationship is not significant; the joint transition highlights alignment gains alongside diversity losses.
  • Minority-Cohort Sensitivity: Fine-tuning can improve overall alignment while reducing sensitivity to the lowest-aligned Nigerian cohorts, including after SFT-E for Qwen and Llama models.The reported performance drop concerns marginalized demographic groups and alignment granularity.
  • Low-Rank Simplicity Bias: Low-rank optimization restricts SFT updates to a narrow manifold, forcing the model to prioritize limited representational dimensions.This rank bottleneck is presented as a mechanistic basis for declining cultural diversity.
  • Polysemantic Competition and Interference: Polysemantic competition lets dominant alignment signals interfere with nuanced marginalized cultural signals in the restricted representational subspace.The paper states that this interference prevents simultaneous representation of the full spectrum of cultural heterogeneity.
  • Rank Starvation and Geometric Collapse: Rank starvation and geometric collapse squeeze cultural variability toward class means, allowing alignment to cultural averages while reducing heterogeneous representation.The paper characterizes flattening as a resource-allocation consequence under rank constraints, not merely a data-frequency issue.
Loading 2609.00565v1…