Source-linked AI summary
Fair Diffusion: Instructing Text-to-Image Generation Models on Fairness
Felix Friedrich, Manuel Brack, Lukas Struppek, Dominik Hintersdorf, Patrick Schramowski, Sasha Luccioni, Kristian Kersting
TL;DR
The paper examines bias in generative diffusion models and presents Fair Diffusion, an inference-time method for instructing image generation toward fairness. It demonstrates control over generated-image bias, including arbitrary proportions for gender and race, without data filtering or additional training.
Problem
Stable Diffusion and its underlying representations exhibit severe gender and racial biases that can be mirrored in generated images.
Method
Fair Diffusion uses the model’s textual interface and user-provided fair instructions at deployment to steer generated outcomes without changing the input prompt.
Results
The method shifts bias in generated images in any direction, yielding arbitrary proportions for attributes such as gender and race.
Takeaways & Limitations
Fair Diffusion provides an immediately deployable way to increase fairness in the outcomes of current image-generation models.
Takeaways & Limitations
The fairness definition covers one binary attribute and assumes attributes are known, definable, measurable, and separable; multiple interacting non-binary attributes are harder to satisfy simultaneously.
Abstract
from arXiv · showhide
Generative AI models have recently achieved astonishing results in quality and are consequently employed in a fast-growing number of applications. However, since they are highly data-driven, relying on billion-sized datasets randomly scraped from the internet, they also suffer from degenerated and biased human behavior, as we demonstrate. In fact, they may even reinforce such biases. To not only uncover but also combat these undesired effects, we present a novel strategy, called Fair Diffusion, to attenuate biases after the deployment of generative text-to-image models. Specifically, we demonstrate shifting a bias, based on human instructions, in any direction yielding arbitrarily new proportions for, e.g., identity groups. As our empirical evaluation demonstrates, this introduced control enables instructing generative image models on fairness, with no data filtering and additional training required.
Fair Diffusion
Fair Diffusion is a deployment-stage strategy that uses textual guidance to steer pretrained text-to-image models toward user-defined fairness outcomes. It supports flexible attribute proportions without modifying the input prompt or retraining the model.
- Fair Diffusion instructs a pretrained diffusion model on fairness during deployment using textual guidance.
- Fair guidance extends classifier-free guidance with an additional term that jointly conditions generation on the prompt and fairness instructions.The instructions are encoded as vectors, scaled, and applied in a chosen direction during generation.
- Users can randomly choose guidance directions from a desired distribution, enabling arbitrary proportions for attribute expressions.Equal probabilities yield statistical parity for a binary attribute, while other distributions can implement different fairness definitions.
- A lookup table identifies biased concepts and associates them with user-designed instructions that promote fair image generation.The lookup table can be set up by any user and could support API integration.
- The paper evaluates fairness as statistical parity in generated outputs, while noting that the definition is limited for multiple interacting attributes.The approach assumes attributes are known, definable, measurable, and separable.
Experiments
Experiments identify gender-occupation biases across LAION-5B, CLIP, and Stable Diffusion, with Fair Diffusion shifting generated outcomes toward the defined fair boundary. Qualitative results show changed gender appearance while preserving occupation-related features.
- Bias inspection: LAION-5B contains stereotypical gender-occupation biases, with lower female-appearing rates in science and engineering and higher rates in arts and caregiving.
- Bias inspection: CLIP associates male-appearing images more closely with career, science, and engineering, while skin-color attributes amplify these gender-occupation associations.The reported examples have high effect sizes and are highly significant at p ≤0.05.
- Mirrored unfairness: 56% of evaluated occupations show amplified LAION-5B gender bias in Stable Diffusion outputs, while 22% are reflected and 22% mitigated.The paper also finds biases in all inspected pipeline components and no clear overall mirroring tendency.
- Bias mitigation: Fair Diffusion shifts gender proportions inside the fair boundary across exemplary occupations and achieves fairness on average for the model outcome.This holds whether Stable Diffusion amplified, reflected, or mitigated the corresponding LAION-5B bias.
- Qualitative results: Qualitative generations alter typical gender appearance while retaining occupation-related features and largely preserving overall image composition.
Discussion
The discussion presents Fair Diffusion as a deployment-stage method for steering biased image-generation outcomes toward user-specified fairness proportions, while emphasizing limits in fairness definitions, evaluation components, and user interaction.
- Practical implications: Fair Diffusion can be immediately deployed to mitigate biases in current image-generation models, but broader debiasing of pipeline components may still be necessary.The authors note that evaluated LAION-5B images remain affected by bias and that Fair Diffusion does not remove all unfairness.
- Deployment-stage mitigation: Fair Diffusion enables users to mitigate biases in generated outcomes at deployment without making the underlying model entirely fair.The strategy controls unfair models to increase fairness in outputs.
- Flexible fairness targets: Fair Diffusion supports flexible outcome proportions by adjusting the guidance probability, including non-uniform targets such as 70/30 for binary attributes.This flexibility can represent occupation proportions or other fairness notions.
- Evaluation limitations: The evaluation is constrained by binary gender labeling, biased or imperfect automated classifiers, and web-crawled data that does not fully represent reality.The study reports fragile behavior for non-binary fairness instructions and limitations from CLIP and FairFace.
- Beyond text-guided fairness: The method remains compatible with multiple guidance interfaces and could extend beyond text through multilingual, visual, or multimodal encodings.The paper discusses AltCLIP, Textual Inversion, and MultiFusion as possible interfaces.
- User interaction: User control creates a dual-use risk because malicious instructions could amplify biased outcomes and reduce diversity.The paper therefore calls for mechanisms to detect malicious interaction.
Conclusion
The paper introduces Fair Diffusion as a deployment-stage method for steering biased generative models toward user-specified fairness outcomes. Its evaluation finds biases in Stable Diffusion’s data and representation, while showing that generated-image proportions can be shifted in arbitrary directions.
- Fair Diffusion instructs generative text-to-image models on fairness at deployment without requiring additional training.The method uses the model’s textual interface and steering approaches to control fairness.
- Severe gender and racial biases appear in LAION-5B and CLIP, and these biases are mirrored in the downstream diffusion model.
- Fair Diffusion shifts generated-image bias in any direction, yielding arbitrary proportions for attributes such as gender and race.
- The authors identify disentangling model components and comparing diffusion models as directions for future fairness research.
- The method can mitigate, preserve, or amplify fairness-related distributions according to the chosen steering direction.
Methods
The paper audits bias sources across diffusion-model components and introduces Fair Diffusion as a deployment-stage mitigation strategy using user instructions. Its evaluation examines dataset, representation, and generated-outcome bias while emphasizing constraints of binary gender measurement and classifier dependence.
- Bias inspection: The study inspects bias in training data, learned representations, and generated outcomes using Stable Diffusion and associated evaluation procedures.Generated-image evaluation creates a synthetic dataset, measures attribute correlations, and assesses fairness.
- Motivation: Large-scale pre-training and downstream data improve performance but also increase the risk of introducing biases into models.
- Limitations of prior approaches: Data pre-processing alone is inadequate because filtered models can retain bias, lose generalization ability, and require datasets tailored to different fairness definitions.
- Fair Diffusion: Fair Diffusion operates at deployment and uses additional textual instructions to increase or decrease concepts, including combinations of opposing directions.The approach builds on classifier-free guidance and can combine multiple concepts arbitrarily.
- Fair Diffusion: Unlike simple prompt engineering, the selected Sega-based implementation edits dedicated features while preserving the broader image composition.The method also avoids requiring language understanding to place gender-related words in prompts.
- Evaluation limitations: The evaluation is limited by classifiers such as FairFace that provide only binary-valued gender labels, although gender is non-binary.
A Further Applications and Results with Fair Diffusion
Further applications show that Fair Diffusion can alter gender and appearance-related attributes while retaining recognizable content. The examples also expose preliminary behavior and risks, including uneven handling of sexuality-related terms.
- Further applications: Fair Diffusion changes firefighters’ gender appearance while preserving overall image composition and keeping the occupation identifiable.
- Further applications: The method generates images with instructed racial or ethnic appearances and changes multiple visual features, including hair, lips, nose, cheeks, and chin.
- Further applications: Default Stable Diffusion tends toward heteronormative depictions and younger people, whereas Fair Diffusion can generate homosexual couples and people of different ages.The authors describe these images as illustrative of the method’s potential.
- Further applications: The terms “homosexual” and “gay” often produce male-homosexual couples, while adding “lesbian” can generate female couples.The authors characterize these findings as preliminary and suggestive of future research directions.
B A more detailed Inspection of CLIP Biases
The CLIP inspection finds gender-occupation, intersectionality, and broader cultural biases in image representations. These biases may contribute to the shift between LAION-5B and Stable Diffusion.
- Intersectionality: Skin-color attributes amplify gender-occupation bias, producing intersectional patterns of compounded advantage or disadvantage.
- Cultural bias: CLIP also exhibits broader cultural biases by associating Arab-Muslim and African-American appearances with unpleasantness.
- Implications: The authors identify CLIP as an inherent source of bias that can help explain bias shifts between LAION-5B and Stable Diffusion.
C Different Setups for Fair Diffusion
Additional experiments show that Fair Diffusion’s effectiveness depends on instruction choice, classifier behavior, and editing parameters. Sega preserves composition better than the compared alternatives, but several setup and measurement limitations remain.
- Different setups: Performance is worse for alternative edit instructions than for the instructions used in the main text.The authors cannot clearly attribute this loss entirely to Fair Diffusion.
- Evaluation limitations: Measured female-appearance rates depend on FairFace, which may classify less clearly identifiable generated people incorrectly and with high uncertainty.
- Different setups: Stable Diffusion has limited understanding of concepts such as “non-binary” and “gender,” making them difficult to use for steering fairer outcomes.
- Different setups: Distinctive instructions are recommended because similar concepts may interfere, while positive guidance alone is insufficient relative to combined positive and negative guidance.
- Editing techniques: Only Sega preserves image composition while editing gender appearance in isolation; the other compared methods alter composition and struggle with the target edit.
D Further Experimental Details
The experiments examine fairness guidance across occupations, identities, heteronormativity, age, and image-editing settings. They also document prompt ambiguity and show that guidance outcomes depend on concept selection and formulation.
- Experimental challenges: “Face of a dishwasher” is ambiguous and mainly produces dishwashing machines rather than occupation-related portraits.The authors suggest evaluating prompts beyond “A photo of the face of a occ” in future work.
- Data organization: The occupation dataset is divided into four hand-selected subgroups: Science, Arts, Engineering, and Caregiving.Each subgroup contains between 50K and 120K images.
- Visual examples: Fair Diffusion changes typical gender appearance for firefighter images while retaining residual occupation-related features.The example uses -“male person” + “female person” guidance.
- Guidance comparisons: Positive guidance alone increases fairness, but negative guidance improves performance relative to the positive-only setup.The comparison covers female-appearing rates for six occupations and the per-group median across 150 occupations.
- Guidance comparisons: Positive and negative guidance can interfere when they use similar concepts, so distinct guidance concepts are required for optimization.The tested configurations remain closer to the fair boundary than SD and LAION, but perform worse than the setup used in Fig. 4.