PaperlayerView exact revision on arXiv

Source-linked AI summary

Aligned but Stereotypical? How System Prompts Shape Demographic Bias in LLM-Based Text-to-Image Models

NaHyeon Park, Na Min An, Kunhee Kim, Soyeon Yoon, Jiahao Huo, Hyunjung Shim

arXiv:2512.04981v2cs.CVcs.LG

TL;DR

LLM-based text conditioning can introduce implicit demographic assumptions even as it improves prompt understanding and alignment. The paper benchmarks demographic skew, examines system prompts as a bias pathway, and proposes FAIRPRO, which reduces bias while maintaining user intent and image quality.

  • Problem

    The paper investigates whether LLM-based text conditioning introduces demographic bias when user prompts do not specify demographic attributes.

    Source: §1 · pp. 1–2

  • Method

    The paper evaluates prompt complexity and system-prompt effects, then proposes FAIRPRO, which adaptively generates fairness-aware system prompts while preserving user intent.

    Source: §1 · p. 2

  • Results

    Across eight recent T2I models, LLM-based systems generally show stronger demographic skew than non-LLM baselines; FAIRPRO reduces bias while maintaining user intent and image quality.

    Source: §1 · p. 2, §6 · p. 8

  • Takeaways & Limitations

    Fairness audits should include the language-conditioning stage, where demographic bias may emerge before image generation.

    Source: §6 · p. 8

    FAIRPRO mitigates biased output distributions at test time but does not remove biases encoded in the underlying generative model.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Text-to-image (T2I) systems increasingly rely on Large Language Model (LLM)-based text conditioning to interpret and expand user prompts. While this improves prompt understanding and text-image alignment, we find that it can also introduce implicit demographic assumptions, even when demographic attributes are unspecified. To systematically investigate this behavior across varying levels of prompt ambiguity and complexity, we construct a comprehensive benchmark covering diverse prompt settings. Evaluations on eight recent T2I models show that LLM-based systems consistently exhibit stronger demographic skew than non-LLM-based baselines. We further analyze system prompts, a component unique to LLM-based T2I systems that guides prompt interpretation and expansion. Our analyses show that these instructions strongly influence text embeddings, which subsequently leads to biased image generations. Motivated by these findings, we propose FairPro, a training-free debiasing framework that adaptively generates fairness-aware instructions while preserving user intent. Experiments demonstrate that FairPro substantially reduces demographic disparities while maintaining prompt fidelity.

1 Introduction

LLM-based text conditioning can improve prompt understanding while introducing demographic assumptions when prompts leave attributes unspecified. The paper benchmarks demographic skew, traces a role for system prompts, and proposes FAIRPRO to reduce disparities while preserving user intent.

1 Introduction

  • Across eight recent T2I models, LLM-based systems generally produce more demographically skewed images than non-LLM baselines across prompt-complexity levels.As prompts grow more complex, demographic bias and text–image alignment both increase.Source: Figure 1 · p. 1, §1 · p. 2
  • Underspecified human-related prompts can elicit stereotypical portrayals that risk disadvantaging underrepresented groups.The paper identifies LLM-based text conditioning as an underexplored bias-propagation pathway and instruction-conditioned processing as a means of encoding and propagating latent demographic priors.Source: §1 · p. 2
  • System prompts shift neutral prompts toward demographic associations in text representations, which are strongly correlated with the demographics in generated images.These instructions guide how LLM-based systems interpret and transform user inputs before image synthesis.Source: §1 · p. 2
  • FAIRPRO adaptively replaces fixed system prompts with fairness-aware instructions intended to reduce demographic disparities while preserving user intent.The framework dynamically detects prompt-specific demographic assumptions and generates context-aware instructions rather than overriding the original prompt.Source: §1 · p. 2
  • COMPBIAS is a large-scale benchmark spanning multiple levels of prompt complexity for analyzing demographic skew.The benchmark contains 1,024 prompts.Source: §1 · p. 2

2 Preliminary Analyses

The section defines demographic bias as skew in attributes left unspecified by a prompt and introduces COMPBIAS to test how that skew varies with prompt complexity. Across the analyses, LLM-based systems show stronger bias, while system prompts shape demographic associations in text representations that are reflected in generated images.

2.1 Problem Setting

  • Demographic bias is skew in human-image attributes not explicitly specified by the prompt; explicitly specified attributes are excluded from evaluation.The analysis covers gender, age, ethnicity, and appearance.Source: §2.1 · p. 3

2.2 Proposed COMPBIAS Dataset

  • COMPBIAS contains 1,024 prompts across four complexity levels, from neutral occupations through explicit attributes and context to LLM-expanded descriptions.The benchmark provides a controlled framework for the paper’s main analyses.Source: §2.2 · p. 3

2.3 Research Questions and Analyses

  • Qwen-Image has the highest bias score among the evaluated models, and bias increases when prompts add an explicit demographic attribute while other attributes remain unspecified.The section reports that this increase can extend to correlated attributes not specified by the prompt.Source: §2.3 · p. 4
  • LLM-based models show higher demographic bias than non-LLM baselines at every prompt-complexity level, alongside higher text–image alignment.The authors describe this as a bias–alignment tension.Source: §2.3 · p. 4
  • Removing default system prompts weakens gender associations in text embeddings while retaining comparable cosine similarity to the original prompt embeddings.The reported male/female embedding similarities are 0.712/0.682 with the system prompt and 0.705/0.692 without it.Source: §2.3 · p. 4, Figure 2 · p. 4
  • Decoded demographic cues are often mirrored in generated images, with 64% agreement across occupational prompts.The analyses also report fewer demographic terms in decoded text after system-prompt removal.Source: §2.3 · p. 5, Figure 4 · p. 5

3 Our FAIRPRO

FAIRPRO replaces fixed system prompts with prompt-specific fairness-aware instructions, aiming to reduce demographic skew without overriding user intent. Its qualitative and overall results describe reduced bias while preserving prompt semantics, image quality, and alignment.

3 Our FAIRPRO

  • Removing the default system prompt can reduce demographic bias but may weaken prompt enrichment and instruction following.FAIRPRO is proposed as an alternative to simply removing that prompt.Source: §3 · p. 5
  • FAIRPRO dynamically generates an input-specific fairness-aware system prompt while preserving the LLM-based prompt-interpretation process.The framework uses LLM self-reasoning to identify plausible stereotypical completions and construct a revised instruction that preserves user intent.Source: §3 · p. 5, Figure 5 · p. 5
  • FAIRPRO preserves explicitly specified attributes and promotes diversity only among attributes left unspecified.For example, it keeps gender specified in “A female data engineer” while allowing variation in age, ethnicity, and appearance.Source: §3 · p. 5
  • FAIRPRO reduces demographic bias while maintaining prompt semantics, image quality, and strong alignment across models and prompt-complexity levels.The supplied qualitative results also describe greater diversity in unspecified attributes without overriding explicit demographic constraints.Source: Figure 6 · p. 6, Figure 7 · p. 6

4 Experiments

FAIRPRO reduces demographic bias across prompt complexities and evaluated LLM-based T2I models while preserving prompt fidelity and image quality. It also improves visual diversity, generalizes to TIBET, and outperforms static or user-prompt interventions in the reported comparisons.

4 Experiments

  • FAIRPRO consistently reduces demographic bias across prompt-complexity levels and LLM-based models without sacrificing prompt fidelity or image quality.The reported results include general alignment and GenEval quality evaluations.Source: §4 · p. 6, Table 1 · p. 6
  • FAIRPRO produces more diverse images than default and no-system-prompt baselines, including broader variation in unspecified attributes.The diversity evaluation covers prompt-complexity levels and reports semantic and perceptual diversity measures.Source: Table 2 · p. 6, §4 · p. 7
  • FAIRPRO attains the lowest mean bias scores, with consistent reductions for gender, ethnicity, and appearance and a smaller positive reduction for age.Its bias reduction is larger and more stable than system-prompt removal in the reported comparison.Source: §4 · p. 7, Table 3 · p. 7
  • On TIBET, FAIRPRO achieves the lowest mean bias scores for SANA1.5 and Qwen-Image, with gains across demographic attributes.The results support generalization beyond the primary benchmark.Source: §4 · p. 7, Table 4 · p. 7
  • FAIRPRO achieves the best overall bias–alignment trade-off among evaluated variants; fixed fairness prompts, removing adaptation, and removing self-auditing weaken results.The two-call variant does not consistently improve over the final single-call design.Source: §4 · p. 7, Table 5 · p. 7
  • User-prompt rewriting yields limited, unreliable bias reduction, whereas FAIRPRO adapts the system prompt before prompt interpretation.The comparison favors system-prompt adaptation for mitigating demographic skew.Source: §4 · p. 8, Table 6 · p. 8

5 Related Work

Recent T2I systems use LLMs or VLMs and embedded system prompts to elaborate user prompts, while prior work studies demographic bias and interventions across the generation pipeline. This paper focuses on adapting system-level instructions as a less-explored intervention point.

5.1 LLM-Based T2I Models

  • Recent LLM-based T2I systems use LLMs or VLMs to interpret user prompts and produce richer textual conditioning than earlier CLIP- or T5-based approaches.Embedded system prompts can guide elaboration of attributes such as color, quantity, composition, and spatial relations.Source: §5.1 · p. 8

5.2 Demographic Bias in T2I Models

  • Prior T2I bias work studies stereotypes, open-set bias, multiattribute interactions, and bias amplification, alongside interventions at text, vision-language, language, image, and data levels.The cited studies include Luccioni et al. (2023), Chinchure et al. (2024), D’Incà et al. (2024), and Shukla et al. (2025).Source: §5.2 · p. 8
  • This work instead adapts system-level instructions at test time, targeting prompt interpretation while preserving prompt-enrichment capability.It distinguishes this intervention from modifying training data, model parameters, generated images, or only user prompts.Source: §5.2 · p. 8

6 Discussion & Conclusion

The paper finds stronger demographic skew in LLM-based T2I models and identifies system prompts as a possible source of implicit demographic assumptions. FAIRPRO adapts those prompts and reduces bias while maintaining user intent and image quality.

6 Discussion & Conclusion

  • LLM-based T2I models show stronger demographic skew than conventional systems, with bias increasing as prompts grow more complex.Representation-level analyses suggest system-level language conditioning can shift textual representations guiding image synthesis.Source: §6 · p. 8
  • FAIRPRO generates input-adaptive, fairness-aware system prompts and consistently reduces demographic bias while maintaining user intent and image quality.The conclusion calls for fairness audits of the language-conditioning stage as well as generated images.Source: §6 · p. 8

Limitations

The study’s limitations include test-time mitigation that leaves underlying model biases intact, discrete demographic categories, and a uniform fairness reference. Its analysis also spans architecturally different conditioning pathways, using decoded text as a diagnostic rather than an exact intermediate prompt for every model.

Limitations

  • Test-time mitigation does not modify model parameters and therefore does not remove biases encoded in the underlying generative model.The authors suggest exploring fairness-aware auditing or conditioning in training pipelines for more fundamental mitigation.Source: Limitations · p. 9
  • The study uses three gender and seven ethnicity categories, which cannot fully represent the diversity and fluidity of real-world identities.The authors recommend extending annotation to finer-grained and intersectional social attributes.Source: Limitations · p. 9
  • A uniform demographic distribution serves as a simple fairness reference, but appropriate fair distributions remain context-dependent and unresolved.Source: Limitations · p. 9

A Model-Specific Forms of LLM/VLM-Mediated Decoded Texts

  • LLM/VLM-mediated models use different conditioning mechanisms, so decoded text is a diagnostic probe rather than necessarily the exact intermediate prompt consumed by each model.The language or vision-language component may act as a prompt enhancer, semantic text encoder, or hidden-state provider.Source: A Model-Specific Forms of… · p. 12

B COMPBIAS Dataset

The benchmark dataset is publicly available under CC-BY 4.0, with representative prompt examples provided across four linguistic complexity levels.

B COMPBIAS Dataset

C.1 Evaluation Using Different Judges

The appendix checks evaluation consistency, illustrates demographic attributes introduced during prompt expansion, tests a different age grouping, and details embedding-based bias analysis.

C.1 Evaluation Using Different Judges

  • 82% agreement with Llama-3.2 and 91% with InternVL3, alongside GPT-4o cross-validation, support consistency of the evaluation findings.Source: §C.1 · p. 12

C.2 Examples of Injecting Demographic Stereotypes

  • Qwen2.5-7B-Instruct expansions of neutral prompts sometimes add gender or age attributes, illustrating demographic assumptions introduced by prompt expansion.Source: §C.2 · p. 12

C.3 Evaluation on Different Age Categories

  • The overall bias trend remains stable when age is grouped as young, middle-aged, and older adults.Source: §C.3 · p. 12

D.1 Text Embedding Analysis

  • Gender bias is measured by the absolute expected difference between occupation-embedding cosine associations with male and female concept embeddings.The gender concept embeddings are means of gender-related term embeddings.Source: §D.1 · p. 12
  • Text-to-image conditioning can use intermediate language or multimodal representations without explicitly rewriting the prompt.Source: Table 7 · p. 13

D.2 Decoded Text Analysis

Decoded text analysis examines demographic words associated with system prompts; accompanying material describes a token-probability probe, reports bias-evaluation comparisons, and gives FAIRPRO inference overhead.

D.2 Decoded Text Analysis

  • Decoded outputs are examined for age- and ethnicity-related words, and the findings suggest system prompts can induce assumptions that may propagate into generated images.Source: §D.2 · p. 13

D.3 Model Decision Analysis

  • The model-decision probe contrasts man- and woman-referencing prompt templates and scores gender preference by differences in first-token probabilities.Positive values indicate preference for the male-labeled option; negative values indicate preference for the female-labeled option, with absolute bias averaged across occupations.Source: §D.3 · pp. 13, 15
  • FAIRPRO generally achieves the lowest bias scores in the InternVL3 evaluation, while the GPT-4o evaluation also reports effective bias mitigation.Source: Table 9 · p. 14, Table 10 · p. 14

E Implementation Details of FAIRPRO

  • System-prompt generation takes 1.58s with Qwen2.5-VL and 1.63s with Gemma2, for total inference overheads of 1.05× and 1.23×, respectively.Image generation remains approximately 35s for Qwen-Image and 7s for SANA1.5.Source: E Implementation Details of FAIRPRO · p. 15

F Ablation Study Details

The ablation materials enumerate prompt and FAIRPRO variants, provide examples across prompt-complexity levels, and document the system and meta instructions.

F Ablation Study Details

  • The ablation compares default and absent system prompts, fixed fairness instructions, variants without a user prompt or chain-of-thought, and one-call versus two-call FAIRPRO.The two-call variant separates stereotype identification from revised-system-prompt generation.Source: F Ablation Study Details · p. 15
  • The prompt tables provide examples for neutral, attribute, attribute-plus-context, and expanded prompts, including examples with injected demographic stereotypes.Source: Table 14 · p. 16, Table 15 · p. 17, Table 16 · p. 18
  • The appendix documents model-specific default system prompts and FAIRPRO meta instructions, whose instructions are highly similar across models apart from the final output constraint.Source: Table 19 · p. 21, Table 20 · p. 22
Loading 2512.04981v2…