PaperlayerView exact revision on arXiv

Source-linked AI summary

SocialCounterfactuals: Probing and Mitigating Intersectional Social Biases in Vision-Language Models with Counterfactual Examples

Phillip Howard, Avinash Madasu, Tiep Le, Gustavo Lujan Moreno, Anahita Bhiwandiwalla, Vasudev Lal

arXiv:2312.00825v2cs.CVcs.AI

TL;DR

Prior VLM bias studies have mostly probed social attributes individually, with intersectional combinations limited by the availability of suitable image-text pairs. This paper generates and filters counterfactual image-text pairs to probe and mitigate intersectional bias. Evaluations find intersectional retrieval biases across six VLMs, and training on the dataset reduces skewness.

  • Problem

    Prior VLM bias studies have primarily examined single social attributes, in part because exhaustive image-text pairs for combinations of attributes are difficult to collect.

    Source: §1 · p. 1

  • Method

    The authors use diffusion models with cross-attention control to generate and filter similar counterfactual image-text pairs that vary in intersectional social attributes.

    Source: §1 · pp. 1–2

  • Results

    All six evaluated VLMs exhibit significant intersectional retrieval bias, and training on SocialCounterfactuals reduces skewness, with ALIP showing up to a 0.426 absolute reduction in MaxSkew@K.

    Source: §1 · p. 2, §5.2 · p. 6

  • Takeaways & Limitations

    The evaluations show SocialCounterfactuals can be used to uncover intersectional bias and as a resource for mitigating retrieval skewness in VLMs.

    Source: §1 · p. 2, §6 · p. 8

    The analysis uses binary perceived gender and selected races, physical characteristics, and occupations; its results cannot be assumed to generalize to omitted social and demographic terms.

    Source: §9 · p. 16

Abstract

from arXiv · show

While vision-language models (VLMs) have achieved remarkable performance improvements recently, there is growing evidence that these models also posses harmful biases with respect to social attributes such as gender and race. Prior studies have primarily focused on probing such bias attributes individually while ignoring biases associated with intersections between social attributes. This could be due to the difficulty of collecting an exhaustive set of image-text pairs for various combinations of social attributes. To address this challenge, we employ text-to-image diffusion models to produce counterfactual examples for probing intersectional social biases at scale. Our approach utilizes Stable Diffusion with cross attention control to produce sets of counterfactual image-text pairs that are highly similar in their depiction of a subject (e.g., a given occupation) while differing only in their depiction of intersectional social attributes (e.g., race & gender). Through our over-generate-then-filter methodology, we produce SocialCounterfactuals, a high-quality dataset containing 171k image-text pairs for probing intersectional biases related to gender, race, and physical characteristics. We conduct extensive experiments to demonstrate the usefulness of our generated dataset for probing and mitigating intersectional social biases in state-of-the-art VLMs.

1. Introduction

Prior VLM bias probes largely examined one social attribute at a time, in part because exhaustive real image-text coverage of attribute combinations is difficult. The paper uses diffusion-generated counterfactual pairs to probe and mitigate intersectional bias, and reports findings from evaluations of six VLMs and training experiments.

1. Introduction

  • Social biases can arise from underrepresented social groups in image-text datasets and appear as associations between attributes and subjects such as occupations.For example, a neutral attorney prompt may retrieve a disproportionate number of images depicting one race.Source: §1 · p. 1
  • Prior VLM bias studies used existing image-text pairs and mostly tested one social attribute at a time, leaving intersectional combinations underexplored.The text attributes this focus in part to the difficulty of collecting exhaustive image-text examples across attribute combinations.Source: §1 · p. 1
  • Variation among real-image depictions can make it difficult to attribute disproportionate retrievals to social attributes alone.Other image differences may also contribute to the retrieval results.Source: §1 · p. 2
  • Stable Diffusion with cross-attention control generates highly similar pairs depicting a common subject while varying intersectional social attributes.The method generates candidate images and applies three filtering stages to retain high-quality counterfactuals.Source: §1 · p. 2
  • 171k+ counterfactual image-text pairs form SocialCounterfactuals, covering intersectional biases related to race, gender, and physical characteristics.The authors report that evaluations of six VLMs reveal intersectional biases and that training experiments reduce retrieval skewness.Source: §1 · p. 2

2. Related Work

Prior work has probed bias in VLMs using collected and annotated images, while mitigation approaches have addressed biases in several ways. The cited work generally considers single social attributes rather than intersectional bias.

2.1. Probing bias in pre-trained models

  • Prior work has studied social bias in language models and proposed approaches for bias detection and mitigation in vision-only models.Some cited studies address intersectionality in bias evaluations.Source: §2.1 · p. 2
  • Existing VLM bias datasets rely on sourced or annotated images, are smaller than SocialCounterfactuals, and focus on single attributes rather than intersectional biases.Examples include VLStereoSet, VisoGender, and MultiModal Bias.Source: §2.1 · p. 2

2.2. Mitigating bias in pre-trained VLMs

  • Mitigation approaches for pre-trained VLMs include prompt-learning adversarial methods, fair sampling, contrastive learning, residual representations, projection matrices, and synthetic contrast sets.Smith et al. [48] used synthetic contrast sets to produce 7,946 image-text pairs for gender bias.Source: §2.2 · p. 2
  • The cited prior mitigation works focus on a single social attribute at a time, unlike this paper’s focus on intersectional bias.The examples include debiasing for gender as a single attribute.Source: §2.2 · p. 3

3. Generating SocialCounterfactuals

SocialCounterfactuals is built by generating captions and counterfactual image sets, then filtering the generations. The resulting dataset contains 170,832 image-text pairs across three intersectional attribute segments.

3.1. Terminology

  • The paper treats gender and race as perceived attributes and notes limits in the categories represented.It does not make claims about gender identification or assignment, and its race list is not representative of all races.Source: §3.1 · p. 3

3.2. Constructing captions for probing social biases

  • The authors construct counterfactual caption sets by combining prefixes, subjects, and pairs of social attributes.They use occupations as subjects and attributes for gender, race, and physical characteristics; the resulting construction uses 260 occupations and produces 54,080 captions grouped into 3,120 sets.Source: §3.2 · p. 3

3.3. Counterfactual image generation

  • Prompt-to-Prompt cross-attention control helps separate image generations retain shared details while varying with the prompt’s attribute changes.The authors over-generate 100 image pairs while sampling the shared-attention parameter p from 0.1 to 0.9, then filter candidates.Source: §3.3 · p. 3

3.4. Three-Stage Filtering

  • CLIP similarity filtering checks both caption-to-image similarity and within-set image similarity, while NSFW and attribute-detectability filters further screen generations.The attribute-detectability filter uses thresholds based on how many images in a set have the targeted attribute discernible by CLIP scores.Source: §3.4 · p. 4
  • 97.5% of sampled images accurately depicted their captions after attribute-detectability filtering, up from 90.8% before that stage.The quality estimate comes from a random sample of 100 race-gender counterfactual sets.Source: §3.4 · p. 4
  • 13,824 counterfactual sets containing 170,832 image-text pairs remain after filtering, across three segments defined by attribute-type pairs.Table 1 reports the retained counts for the dataset segments.Source: §3.4 · p. 4, Table 1 · p. 4

4. Probing Intersectional Biases

The paper measures retrieval skew for attribute pairs using neutral prompts and reports intersectional and marginal patterns across VLMs. All six evaluated models show skew, with its extent varying across attribute combinations, occupations, and races.

4.1. Evaluation Metrics

  • MaxSkew@K aggregates pairwise Skew@K by taking the maximum across the attribute pairs under consideration.Skew@K compares the observed proportion of retrieved images with an attribute pair to its desired proportion.Source: §4.1 · p. 5

4.2. Results

  • All six evaluated VLMs show significant retrieval skew, with Race-Gender biases tending to be greatest and Physical Characteristics-Gender biases tending to be lower.CLIP and FLAVA have the lowest overall MaxSkew@K among the six models.Source: §4.2 · p. 5, Figure 3 · p. 6
  • Occupation-specific extremes vary across models: FLAVA and LaCLIP peak on Race-Gender skew for Makeup Artist, while LaCLIP and OpenCLIP peak on Physical Characteristics-Race skew for Barber.CLIP and FLAVA also have multiple occupations with zero skew across segments, whereas SLIP has none.Source: §4.2 · p. 5
  • Marginal gender skew varies across races; CLIP’s Indian-subject gender skew is nearly 2x its Latino-subject skew.CLIP and FLAVA show greatest gender bias for Indian subjects, while SLIP and ALIP show greatest bias for Middle Eastern subjects.Source: §4.2 · p. 5, Figure 4 · p. 6
  • Doctor retrieval patterns show that similar gender biases can align with different races across models, and CLIP retrieves no Black female doctors.SLIP, OpenCLIP, LaCLIP, and ALIP favor male images, but their race-specific patterns differ; CLIP retrieves a slightly higher proportion of women than men.Source: §4.2 · pp. 5–6

5. Mitigating Intersectional Biases

Training on SocialCounterfactuals reduces intersectional retrieval skew across the tested models and bias types, with effects that generalize to occupations withheld from training. Debiased CLIP also reduces skew on two real-image datasets, while showing small task-performance losses on some evaluations.

5.1. Training Experiment Setting

  • For each dataset segment, the experiment holds out 20% of occupation subjects for testing and fine-tunes ALIP, CLIP, and FLAVA on the remaining counterfactual sets.MaxSkew@K is measured on the withheld occupations.Source: §5.1 · p. 6

5.2. Results for Intersectional Social Biases

  • Training reduces MaxSkew@K across all three intersectional bias types; ALIP improves by up to 0.426, and CLIP reduces Physical Char./Race skew by 0.327.The strongest pretrained intersectional biases benefit most, and debiasing effects generalize to occupations withheld from training.Source: §5.2 · p. 6, Table 2 · p. 7

5.3. Analysis of Race-Gender Debiased CLIP Model

  • Figure 5 reports the proportions of Doctor images retrieved at k = 12 using neutral prompts.The accompanying analysis separates race and gender marginals from intersectional race-gender retrievals.Source: §5.3 · p. 6, Figure 5 · p. 7
  • With no existing real-image resource for the intersectional biases studied, the analysis evaluates debiased CLIP on PATA and VisoGender.PATA covers race, age, and gender labels across scenes; VisoGender benchmarks gender bias in occupational settings.Source: §5.3 · p. 7
  • On real-image VisoGender and PATA, debiased CLIP reduces skewness by 15% and 12%, respectively; both show lower MaxSkew@K than SocialCounterfactuals.The authors interpret the lower skew on these single-attribute datasets as evidence that SocialCounterfactuals reveals more retrieval skew.Source: §5.3 · p. 7, Table 3 · p. 7
  • Debiased CLIP matches or exceeds pretrained CLIP in all three Flickr30K text-retrieval settings, with a minor degradation in image retrieval.These comparisons assess CLIP debiased for race-gender intersectional bias.Source: §5.3 · p. 8, Table 4 · p. 8
  • Debiased CLIP has minor accuracy reductions on zero-shot image-recognition datasets, illustrating a fairness–performance trade-off.The training process was not tuned to balance task-specific performance against debiasing; task-focused tuning or mixing real data may change that balance.Source: §5.3 · p. 8, Table 5 · p. 8

6. Conclusion

The paper presents a method for generating counterfactual examples to probe and mitigate intersectional bias in VLMs, and reports that SocialCounterfactuals helps reduce retrieval skew. Its scope is limited by possible latent biases in the templates and methods and by incomplete coverage of social categories.

6. Conclusion

  • The work presents an automatic method for generating counterfactual examples to probe and mitigate intersectional VLM bias, and reports significant retrieval biases across six pretrained VLMs.SocialCounterfactuals covers gender, race, and physical characteristics across occupations; training experiments show it can reduce retrieval skew.Source: §6 · p. 8
  • Future work could extend the approach to additional attributes and subjects, and explore training strategies that balance bias mitigation with task-specific performance.The authors also suggest investigating whether SocialCounterfactuals can help reduce bias in generative text-to-image diffusion models.Source: §6 · p. 8
  • The authors identify possible latent biases in their templates and methods, and limit interpretation of social attributes and associations to perceived categories and their experiments.The study uses binary gender and does not exhaustively cover races, physical characteristics, or occupations because of data limitations.Source: §6 · p. 8

7. Additional Analysis of SocialCounterfactuals

The analysis examines generation failures, occupation-specific depiction patterns, and limits encountered when extending SocialCounterfactuals beyond occupations. Filtering improves image-generation accuracy, but some social-attribute combinations and subject types remain difficult to depict reliably.

7.1. Examples of Counterfactual Sets

  • The authors show additional counterfactual-set examples in Figures 14–16.Source: §7.1 · p. 12

7.2. Error Analysis

  • 90.8% of 1,200 sampled images correctly depicted occupation, gender, and race; gender was unidentifiable in 5.2%, and female and male generation failures occurred in 2.2% and 0.8%.The gender-identification failures typically involved subjects looking backward or facing the wrong direction.Source: §7.2 · p. 12, Table 6 · p. 12
  • 12% of Middle Eastern female subjects failed to generate, while no generation failures were observed for Middle Eastern male subjects.Across White, Black, Indian, and Asian races, female and male generation-failure rates were approximately the same.Source: §7.2 · p. 12, Table 8 · p. 13
  • 97.5% overall accuracy after CLIP Attribute Detectability Filtering, up from 90.8%; good-image rates were 99% for male and 96% for female images.The section also reports 90.8% correctly generated images before this final filtering stage.Source: §7.2 · p. 12

7.3. Analysis of Occupation Detectability

  • Occupation depictions sometimes blur similar jobs, switch football codes across race groups, and encode appearance or gender patterns such as glasses, body size, and missing male midwives or maids.Hairdresser and barber images sometimes bind the described race to the client rather than the worker; the authors say cross-occupation similarity should not significantly affect bias estimates because skew is measured separately by occupation.Source: §7.3 · pp. 12, 14

7.4. Generating counterfactuals for other subjects and social attributes

  • Personality-trait images were omitted because Stable Diffusion struggled to depict the traits, degrading the quality of the social-attribute depictions.Source: §7.4 · p. 14
  • Religion counterfactuals were excluded because some terms produced nearly identical images and discernible differences were primarily conveyed through race.The authors leave investigation of intersectional religion biases to future work.Source: §7.4 · p. 14

8. Dataset Generation & Experiment Details

This section reports compute and training settings, describes the filtering pipeline and its effects, and gives details of caption and dataset construction.

8.1. Details of compute infrastructure used

  • The authors used up to 256 Intel Gaudi-2 AI accelerators on an Intel 3rd Generation Xeon cluster to create the counterfactual data.Source: §8.1 · p. 14

8.2. Details of training experiments

  • CLIP was fine-tuned for one epoch with batch size 32 and learning rates of 7e-6, 5e-6, and 9e-6 for the three listed attribute pairs; FLAVA and ALIP used the same setting.Source: §8.2 · p. 14

8.3. Details of dataset filtering

  • 312,000 original counterfactual sets (5,408,000 images) were reduced to 21,359 after CLIP similarity filtering, 21,116 after NSFW filtering, and 13,824 in the final dataset.The NSFW stage reduced the set count by 1%.Source: §8.3 · p. 14, Table 11 · p. 16
  • NSFW filtering discarded approximately 0.9% of race-gender sets, 1.4% of physical-characteristic-gender sets, and 2.7% of physical-characteristic-race sets.Source: §8.3 · p. 15
  • CLIP attribute-detectability filtering uses thresholds calibrated from manual labels to remove sets with too few images whose targeted attributes CLIP can discern.The process first labels 100 randomly sampled sets, then compares CLIP image-text scores for prompts such as “a female person” and “a male person.”Source: §8.3 · p. 15

8.4. Details of Caption Construction

  • Caption construction combined four prefixes, occupations, and 100 over-generations per set; the study began with 260 occupations and retained 158 after filtering.Physical-characteristic bias used the keywords skinny, obese, young, old, and tattooed.Source: §8.4 · p. 15

9. Additional Discussion of Limitations and Ethical Considerations

The authors bound their findings to perceived attributes and the categories their methods can depict, and caution that the study does not cover all groups or contexts. They also warn that debiasing is not guaranteed to eliminate harm.

Limitations

  • The study uses binary gender categories and does not cover all races, physical characteristics, or occupations; findings do not generalize to omitted terms or other languages.The authors state that labels were limited to those Stable Diffusion could depict and that the study was conducted in English.Source: §9 · p. 16
  • Templates and methodologies may contain latent biases that contribute to the implicit biases exhibited by VLMs.Source: §9 · p. 16

Ethical Considerations

  • The selected categories may harbor stereotypes and miss intersectional characteristics, so they should not be treated as representative of entire groups.The authors also note trade-offs in constructing lists of social attributes.Source: §9 · p. 16
  • Bias-reduction strategies do not guarantee bias elimination and may still harm or stigmatize subsets of individuals; further development is advised before sensitive applications.Source: §9 · p. 16

10. Additional Analysis and Results

The additional analyses find residual intersectional and marginal gender bias across evaluated models, and a higher MaxSkew score for three-attribute than two-attribute intersections. They also report image-quality metric advantages over two real-image datasets and largely similar bias patterns after filtering.

10.4. Additional details and results for bias probing experiments

  • After NSFW and attribute-detectability filtering, intersectional MaxSkew distributions and marginal gender skew largely retain the previously reported trends.The appendix also plots retrieval proportions for multiple occupations, including Doctor and Software Developer.Source: Figure 6 · p. 17, §10.4 · p. 18, Figure 7 · p. 18

10.1. NDKL and Bias@K results

  • CLIP and FLAVA generally have the lowest NDKL scores, but all models remain above the ideal value of 0; ALIP and FLAVA show the least marginal gender bias overall.CLIP shows some bias toward female images, while LACLIP, OpenCLIP, and SLIP show strong bias toward male images.Source: §10.1 · p. 17, Table 14 · p. 17, Table 15 · p. 17
  • Tables 16 and 17 report NDKL and Bias@K for pre-trained and debiased CLIP, ALIP, and FLAVA, evaluated on occupations withheld from training; Bias@K measures gender bias only.The tables estimate results by withholding counterfactual sets for 20% of occupations.Source: §10.1 · p. 17, Table 16 · p. 18, Table 17 · p. 18

10.2. Preliminary evaluations using counterfactuals with three intersectional attributes

  • 0.693 is CLIP's MaxSkew@60 for three-attribute intersections, above the 0.567–0.59 range for two-attribute pairs on the same images.The three-attribute set contains 16k images across 266 counterfactual sets; the authors describe broader intersectional evaluation as a direction for future study.Source: §10.2 · p. 17

10.3. FID and IS evaluation of SocialCounterfactuals and other datasets

  • SocialCounterfactuals outperforms VisoGender and PATA across all reported Inception Score and FID metrics relative to ImageNet and LAION-2B.Source: §10.3 · p. 17

10.4. Additional details and results for bias probing experiments

Loading 2312.00825v2…