PaperlayerView exact revision on arXiv

Source-linked AI summary

Understanding and Evaluating Racial Biases in Image Captioning

Dora Zhao, Angelina Wang, Olga Russakovsky

arXiv:2106.08503v2cs.CV

TL;DR

Image captioning is useful for visual reasoning and accessibility, but its data and models can contain social biases. This paper manually annotates perceived gender and skin color in COCO, traces bias through captions and models, and finds demographic imbalances, racial language, and lighter-versus-darker caption differences that are more pronounced in newer systems.

  • Problem

    Existing image-captioning bias research has emphasized gender or manual captions, leaving racial and intersectional bias propagation into automated systems less studied.

  • Method

    The authors annotate perceived gender and skin color for 28,315 people in COCO and compare manual and automatically generated captions across demographic groups.

  • Results

    The study finds dataset imbalance, racial terms in manual captions, lighter-skinned performance advantages on CIDEr and BLEU but not SPICE, and persistent word-choice differences after controlling for visual appearance.

  • Takeaways & Limitations

    Bias is present in the dataset and automated captioning models, appears to be increasing in models indicative of the field’s progress, and warrants mitigation efforts.

  • Takeaways & Limitations

    Race is operationalized through perceived phenotypic skin color, a subjective and static proxy that can miss differences captured by other conceptualizations of race.

Abstract

from arXiv · show

Image captioning is an important task for benchmarking visual reasoning and for enabling accessibility for people with vision impairments. However, as in many machine learning settings, social biases can influence image captioning in undesirable ways. In this work, we study bias propagation pathways within image captioning, focusing specifically on the COCO dataset. Prior work has analyzed gender bias in captions using automatically-derived gender labels; here we examine racial and intersectional biases using manual annotations. Our first contribution is in annotating the perceived gender and skin color of 28,315 of the depicted people after obtaining IRB approval. Using these annotations, we compare racial biases present in both manual and automatically-generated image captions. We demonstrate differences in caption performance, sentiment, and word choice between images of lighter versus darker-skinned people. Further, we find the magnitude of these differences to be greater in modern captioning systems compared to older ones, thus leading to concerns that without proper consideration and mitigation these differences will only become increasingly prevalent. Code and data is available at https://princetonvisualai.github.io/imagecaptioning-bias .

1. Introduction

The paper examines how racial, gender, and intersectional biases propagate from COCO images and manual captions into automated image captioning systems. It finds demographic imbalances, racial language, performance differences, and word-choice differences that are especially pronounced in newer models.

  • Image captioning supports visual reasoning and accessibility, but societal biases already affect computer vision systems and can propagate through their pipelines.The study therefore traces bias from images to manual captions and then to generated captions.
  • Manual captions contain racial terms, including slurs; newer transformer-based models learn racial descriptors, whereas older systems do not.The slurs themselves do not yet appear to be learned by the models.
  • Captioning systems perform slightly better on lighter-skinned images according to CIDEr and BLEU, but not SPICE.This pattern is consistent with disparate accuracies reported for other visual recognition systems.
  • Lighter- and darker-skinned people appear in systematically different visual contexts, while caption word choices still differ after controlling for visual appearance.The word-choice differences are particularly apparent in manual captions and modern transformer-based systems.
  • The work establishes a COCO-based foundation for studying bias propagation and releases its data and code for research.The authors emphasize that the annotations support analysis of captioning and other visual recognition tasks.

2. Related Work

Prior research documents dataset and algorithmic bias across machine learning, while racial bias in image captioning has received less attention than gender bias. This work extends that literature by studying racial bias propagation into automated captioning and the role of racial descriptors.

  • Dataset biases include underrepresentation and stereotyped portrayals, and can propagate into trained models.These concerns motivate analysis of bias in widely used benchmarks such as COCO.
  • Bias mitigation research addresses data collection, cleanup, synthetic generation, transparency, and, in extreme cases, dataset withdrawal.The cited literature treats dataset bias as having complex roots, including search engines, collection practices, and real-world disparities.
  • Algorithmic bias work includes studying bias amplification, where social biases in data become amplified in trained models.This paper applies that perspective to the propagation of bias from manual image captions into captioning models.
  • Earlier image-captioning bias research often focused on gender, while racial-bias analysis was largely limited to manual captions.This paper expands racial-bias analysis to automated image captioning systems.
  • Whether generated captions should include race is an open question, because some blind or low-vision people prefer racial descriptors in certain contexts.This preference motivates further study of how automated systems label identities.

3. Crowdsourcing Demographic Annotations

The authors crowdsource perceived gender and skin-color annotations for selected COCO validation-set people, aggregate them to instance- and image-level labels, and analyze demographic distributions. The annotations expose substantial gender, racial, and intersectional underrepresentation, while worker-demographic comparisons show little systematic disagreement.

  • 3.1. Annotation process: The study selects 15,762 COCO 2014 validation images containing 28,315 sufficiently large person instances for demographic annotation.The source dataset contains 40,504 validation images and five human captions per image.
  • 3.1. Annotation process: AMT workers annotate Fitzpatrick skin type and binary gender expression three times per instance, with an unsure option for either label.The interface asks for perceived attributes rather than verified identities.
  • 3.1. Annotation process: The labels are imperfect proxies because they are subjective, discretized social constructs and operationalize race through phenotypic skin color.This formulation can miss differences captured by other conceptualizations of race.
  • 3.1. Annotation process: Quality control combines experienced workers with 57 gold-standard images, requiring correct labeling of five injected examples in each 50-image task.Workers needed more than 1,000 completed tasks and a 98% acceptance rate.
  • 3.2. Gender annotations: Gender annotations identify males as 47.4% of instances and females as 23.7%, with 26.6% unsure and 2.2% lacking consensus.Contradictory labels are most common for children, obscured faces, and people whose features conflict with gender stereotypes.
  • 3.2. Gender annotations: At the image level, male images comprise 47.4% and female images 21.0%, and the distribution closely mirrors instance-level annotations.Image labels are mapped primarily from the largest person bounding box, with an exception for a sufficiently large opposite-gender second person.
  • 3.2. Gender annotations: Manual gender annotations agree with automatically derived labels on 66.3% of 5,413 images and disagree on 1.4%, leaving 32.3% not reliably labelable with one gender.The comparison highlights the difference between manual and caption-derived demographic labels.
  • 3.3. Skin color annotations: Fitzpatrick Types 2 and 1 account for 31.5% and 15.4% of instances, while Types 5 and 6 account for only 1.9% and 1.7%.The authors identify this darker-skinned underrepresentation as representational harm.

4. Experiments

The experiments trace racial bias from images and manual captions into automated captioning, finding disparities in descriptors, performance, visual co-occurrences, and word choice. These differences are especially evident in newer transformer-based systems and persist after visual controls.

  • 4.1. Captions contain racial descriptors: Racial descriptors appear in manual captions more often for Black than white individuals, at 0.54% versus 0.03% of instances.For 26.9% of white individuals described racially, another race is also mentioned.
  • 4.2. Performance differs slightly between lighter and darker images: Att2in, Transformer, AoANet, and Oscar score 2.7 ± 0.7, 3.2 ± 1.2, 1.9 ± 1.6, and 3.0 ± 1.1 higher CIDEr on lighter than darker images.The difference is not noticeable for SPICE, and the CIDEr gap correlates positively with overall model performance at R2 = 0.343.
  • 4.3. Visual appearance differs between lighter and darker images: Darker-skinned people appear more often with outdoor and vehicle objects, whereas lighter-skinned people appear more often with indoor and furniture objects.Sink, potted plant, and toothbrush occur with lighter-skinned people over 13x as often as darker-skinned people, despite a 7x image-count difference.
  • 4.4. Captions describe people differently based on skin tone: Even with visual appearance controlled, captions use different vocabularies for lighter- and darker-skinned individuals, with distinguishability increasing for several newer systems.The classifier reaches AUC 61.8 ± 3.8 on human captions; AoANet rises from 65.9 ± 4.2 for FC to 68.0 ± 1.8.

5. Discussion and Conclusion

The paper studies how racial and intersectional biases appear in COCO images, captions, and captioning models, finding evidence of bias in both the dataset and automated systems. It also proposes mitigation directions while cautioning that undetected bias may remain and may increase with newer systems.

  • Discussion and Conclusion: The study annotates skin color and gender expression to examine racial bias in COCO and its propagation into trained image captioning models.The annotations support analysis of bias across images, manual captions, and automatically generated captions.
  • Discussion and Conclusion: The dataset and automated captioning models contain instances of bias, while the authors caution that experiments failing to detect bias do not establish its absence.The paper reports that observed bias appears to increase in models indicative of the field’s progression.
  • Discussion and Conclusion: The authors propose clearer annotation instructions, removal of captions containing slurs or offensive concepts, and more diverse datasets to improve bias measurement and mitigation.They also identify differences among generated-caption systems, including lower apparent bias for Oscar than for other systems.

A. Comparing collected gender annotations with automatically derived ones

The authors compare manually collected gender annotations with several caption-derived labeling schemes. Expanding gendered vocabulary and changing labeling thresholds improves coverage and accuracy, but all automated methods remain imperfect.

  • A. Comparing collected gender annotations with automatically derived ones: The authors vary person-size filtering, gendered vocabulary size, and the number of captions required to mention gender when deriving image-level labels.The expanded vocabulary includes multiple masculine and feminine terms beyond “man” and “woman.”
  • A. Comparing collected gender annotations with automatically derived ones: Extensions increase both the number of correctly labeled images and the accuracy of labeled images, but all caption-derived gender-labeling methods remain inaccurate or incomplete.The results caution against relying on automated gender derivation.

B. Racial descriptors

The racial-descriptor analysis searches captions for a predefined set of race- and ethnicity-related keywords and compares alternative automatic gender-labeling approaches shown in Figure 7.

  • B. Racial descriptors: The analysis searches lowercased captions for descriptors including “white,” “Black,” “Asian,” “Latino,” “Hispanic,” “Native,” and “Indigenous.”The keyword list also includes “Caucasian,” “African,” “Latina,” and “Latinx.”
  • B. Racial descriptors: Figure 7 compares automatic image-level gender-annotation methods using different gendered-word set sizes and caption-count requirements.The percentage over each bar indicates accuracy, and all methods are described as imperfect and noisy.

C. Caption performance

Captioning performance is evaluated across multiple models and metrics on the COCO 2014 validation set. Newer transformer-based systems outperform older models overall, while performance differences between lighter and darker images are larger for several newer systems.

  • C. Caption performance: Performance differences between lighter and darker images are reported for BLEU, CIDEr, and SPICE on the COCO 2014 validation set.Table 3 reports caption performance for these metrics, multiplied by 100, with 95% confidence intervals across five random-seed models per architecture.
  • C. Caption performance: Oscar has the best performance across all reported metrics, and newer transformer-based models outperform older models across all metrics.The comparison covers four models in the cited analysis.
  • C. Caption performance: Differences for METEOR and ROUGE are greater for Att2in, Transformer, and Oscar, with AoANet showing smaller differences.The authors state that this supports an association between better-performing models and greater performance discrepancies.

D. Vocabulary differences coefficients

This section compares the words associated with lighter and darker images in manual and automatically generated captions, while also reporting caption-performance differences across architectures.

  • Vocabulary differences coefficients: The analysis presents predictive words for lighter and darker images across both manual captions and automatically generated captions.These word-choice comparisons are reported in Table 5.
  • Captioning performance: Caption-performance differences between lighter and darker images are measured with BLEU, METEOR, ROUGE, CIDEr, and SPICE on COCO 2014 validation data.Scores are computed as lighter minus darker and multiplied by 100, with 95% confidence intervals across five models per architecture.
  • Vocabulary differences coefficients: Logistic regression identifies the words most predictive of lighter and darker images in manual and automated captions after controlling for visual content.For automated captions, coefficients are averaged across five models per architecture; lower coefficients are more predictive of lighter images.
Loading 2106.08503v2…