Source-linked AI summary
Women also Snowboard: Overcoming Bias in Captioning Models
Kaylee Burns, Lisa Anne Hendricks, Kate Saenko, Trevor Darrell, Anna Rohrbach
TL;DR
Image captioning models can amplify gender bias and rely on contextual cues when generating gendered words. The paper introduces Equalizer with complementary losses that suppress gender predictions without evidence and encourage confidence when evidence is present. Equalizer reports lower gender-word error, closer agreement with ground-truth gender ratios, and more frequent focus on people than competing approaches.
Problem
Image captioning models can amplify training-data bias and use contextual cues rather than evidence from the person when generating gendered words.
Method
Equalizer combines Appearance Confusion Loss and Confident Loss to produce equal gender probabilities without gender evidence and confident predictions when evidence is present.
Results
Equalizer reports lower gender-word error, closer ground-truth gender ratios under distribution shift, and greater focus on people when predicting gender.
Takeaways & Limitations
The complementary losses encourage captioning models to be cautious without gender information and discriminative when gender evidence is available.
Takeaways & Limitations
Appearance Confusion Loss requires human rationales about appropriate visual gender evidence, and clothing-based cues may be inappropriate for some applications.
Abstract
from arXiv · showhide
Most machine learning methods are known to capture and exploit biases of the training data. While some biases are beneficial for learning, others are harmful. Specifically, image captioning models tend to exaggerate biases present in training data (e.g., if a word is present in 60% of training sentences, it might be predicted in 70% of sentences at test time). This can lead to incorrect captions in domains where unbiased captions are desired, or required, due to over-reliance on the learned prior and image context. In this work we investigate generation of gender-specific caption words (e.g. man, woman) based on the person's appearance or the image context. We introduce a new Equalizer model that ensures equal gender probability when gender evidence is occluded in a scene and confident predictions when gender evidence is present. The resulting model is forced to look at a person rather than use contextual cues to make a gender-specific predictions. The losses that comprise our model, the Appearance Confusion Loss and the Confident Loss, are general, and can be added to any description model in order to mitigate impacts of unwanted bias in a description dataset. Our proposed model has lower error than prior work when describing images with people and mentioning their gender and more closely matches the ground truth ratio of sentences including women to sentences including men. We also show that unlike other approaches, our model is indeed more often looking at people when predicting their gender.
1 Introduction
The paper targets gender bias in image captioning, where models may rely on contextual cues instead of evidence from the described person. It introduces Equalizer, which combines complementary losses to reduce bias, improve gender predictions, and encourage attention to people.
- Motivation: Image captioning models can make gender predictions from contextual objects or locations rather than visual evidence about the described person.The paper highlights scenes where objects such as snowboards or laptops can influence gendered words.
- Motivation: Prior work can be correct for the wrong reason by ignoring the person even when it predicts the gender word accurately.Figure 1 contrasts incorrect predictions caused by attending to a laptop with correct predictions that still fail to attend to the person.
- Equalizer Model: Unlike a prior rebalancing approach, Equalizer does not assume that training and test gender distributions match or require access to the entire test set.The paper evaluates caption generation for images independently of other test images.
- Equalizer Model: Equalizer combines Appearance Confusion Loss and Confident Loss to be cautious without gender evidence and confident when gender evidence is present.The losses are designed to confuse predictions when gender evidence is removed while strengthening predictions when it remains.
- Results: Equalizer produces lower gender-word error, remains effective under shifted gender distributions, and focuses more often on humans than contextual image content.It also generates gender-neutral words such as “person” when it lacks confidence about gender.
2 Related Work
Related work frames captioning bias as amplification arising from dataset co-occurrences, language priors, and reporting conventions. The paper distinguishes its approach from test-set rebalancing and connects it to rationale-based and fairness-oriented methods.
- Unwanted Dataset Bias: Bias amplification can increase correlations learned from captioning data, and balancing one attribute or category may not correct broader co-occurrence imbalance.The paper notes that balancing across all possible co-occurrences is infeasible in large vocabularies.
- Unwanted Dataset Bias: Caption annotations can reflect reporting and cultural biases by specifying gender or roles that cannot be confirmed from the image.A snowboarder may be labeled as a man even when gender evidence is occluded.
- Prior Approaches: Prior work rebalances test-time predictions but requires the entire test set and assumes test-time distributions match training distributions.The paper instead evaluates a setting where the test distribution can differ from the training distribution.
- Fairness: Appearance Confusion Loss relates to distribution-matching methods by making man and woman predictions similar when gender information is unavailable.Its purpose is to enforce a fair prior under removed gender evidence.
- Right for the Right Reasons: The paper evaluates whether models use human-appropriate evidence, motivated by work on right-for-the-right-reasons explanations and dataset-bias discovery.Unlike methods requiring accurate explanations during training, it verifies the model’s visual focus at test time.
3 Equalizer: Overcoming Bias in Description Models
Equalizer combines standard captioning with complementary losses that make gender predictions cautious without evidence and confident when evidence is present. It uses masked images and gender-relevant annotations while remaining compatible with other deep captioning frameworks.
- 3 Equalizer: Overcoming Bias in Description Models: Annotation masks identify image evidence appropriate for gender decisions, and masking produces images in which inappropriate gender information is removed.The masked image is formed as I ⊙ M, where masked pixels should not contribute to the gender decision.
- 3 Equalizer: Overcoming Bias in Description Models: Appearance Confusion Loss encourages equal man-versus-woman probabilities when gender evidence is absent, while preserving ordinary captioning for visible non-gender content.Non-gendered words are trained with standard cross entropy so the model can still describe objects such as dogs and frisbees in masked images.
- 3 Equalizer: Overcoming Bias in Description Models: Confident Loss encourages correct gender-word predictions when gender evidence is present without forcing the model to emit a gendered word.It compares predicted probabilities for woman and man words using a quotient, with ε added for numerical stability.
- 3 Equalizer: Overcoming Bias in Description Models: Equalizer combines Caption Correctness, Appearance Confusion, and Confident losses to produce image-relevant captions with calibrated gender predictions.The loss weights are selected on a validation set, with α = 1, µ = 1, and β = 10 in the reported experiments.
- 3 Equalizer: Overcoming Bias in Description Models: The two novel losses are complementary: the lowest misclassification rate occurs only when Appearance Confusion Loss and Confident Loss are combined.Equalizer can substitute its base captioning framework with any other deep captioning framework.
4 Experiments
Experiments evaluate Equalizer on gender prediction, gender-ratio alignment, human-description agreement, and whether visual explanations focus on people. Across these measures, Equalizer reduces error and distribution sensitivity while improving fairness and person-focused evidence, though clothing-based gender cues may not suit every application.
- Datasets and Metrics: The MSCOCO-Bias dataset labels images as man or woman from caption mentions, while MSCOCO-Balanced provides a 1:1 woman-to-man ratio for testing distribution shifts.MSCOCO-Bias has roughly a 1:3 woman-to-man ratio; the balanced test set randomly selects 500 images from each gender category.
- Datasets and Metrics: Error counts man/woman misclassifications while treating gender-neutral terms as non-errors, rewarding conservative predictions when gender is unclear.The evaluation emphasizes low error because gender-word mistakes are sensitive, and expects gender-neutral outputs when gender evidence is not obvious.
- Error and Bias Robustness: Equalizer consistently improves error over baselines and remains similarly accurate across MSCOCO-Bias and MSCOCO-Balanced, indicating reduced sensitivity to test-time gender-distribution shifts.Its full model also improves on variants using only Appearance Confusion Loss or only Confident Loss.
- Gender Ratio: Equalizer achieves the closest generated woman-to-man caption ratio to ground truth on both datasets, with male-word difference 0.147 versus 0.193 for Baseline-FT.For female words, the reported differences are 0.096 for Equalizer and 0.99 for Baseline-FT; the paper notes that the two losses are complementary.
- Fairness Across Classes: Equalizer has the lowest outcome-type divergence between men and women, with Jensen-Shannon divergence 0.018, while improving error for the minority Women class.The model shows similar Correct, Incorrect, and Other outcome rates across genders, a fairness criterion related to equalized odds.
- Human Agreement and Visual Evidence: As annotator confidence increases, Equalizer surpasses other models in three-class accuracy, and its pointing-game explanations most accurately land within person masks.At low confidence it more often predicts gender-neutral terms; the pointing evaluation uses Grad-CAM and occlusion-based saliency maps on MSCOCO-Balanced.
- Limitations: The model may use clothing such as dresses as gender evidence, which the authors note may be inappropriate for some applications.This limitation motivates caution when applying the approach to protected-class prediction.
A Content
The supplementary material adds quantitative and qualitative analyses, including per-word performance, masked-image behavior, expanded gender-word training, and further examples.
- B Additional analysis: Section B analyzes per-word performance, masked-image behavior, and training with larger sets of gendered words.
- C Qualitative Examples: Section C provides additional qualitative examples comparing the baselines with Equalizer.
B Additional analysis
Additional analyses show that Equalizer reduces gender imbalance on masked images and generally improves performance on images containing gender-biased words, while expanding the gendered-word sets yields no significant MSCOCO gains.
- 1.98 was the Equalizer man-to-woman gender ratio on masked images, closer to the ideal 1.0 than the ablated variants.Equalizer without Appearance Confusion Loss scored 3.45, while Equalizer without Confident Loss scored 2.87.
- Equalizer generally outperformed Baseline-FT on error rate and gender-ratio difference for images containing selected gender-biased words.The analysis used five words biased toward women and five biased toward men, including snowboard, skateboard, umbrella, and kitchen.
- Equalizer without Appearance Confusion Loss had the best error performance, whereas the full Equalizer had the best predicted gender ratio in the biased-word breakdown.
- Training with larger sets of gendered words improved error and ratio on MSCOCO-Bias, but the improvement was not statistically significant on MSCOCO.The main-paper results used only {man} and {woman}; larger sets included additional familial, occupational, and gender terms.
C Qualitative Examples
Qualitative examples show Equalizer predicting neutral “person” when gender evidence is difficult or poses are unusual, while Grad-CAM visualizations compare whether models use appropriate gender evidence.
- Equalizer predicts “person” instead of “woman” or “man” when visible gender evidence is challenging, such as occluded faces or partially visible bodies.The examples include a person whose face is occluded by a giraffe and another showing only hands and arms.
- The Confident Loss can produce neutral words even when Equalizer attends to clear, correct gender evidence but remains uncertain.This accounts for an example where the model predicts “person” despite looking at appropriate gender evidence.
- Figure 6 contrasts success cases with correct gender predictions based on the right evidence against failures involving incorrect gender and wrong evidence.The qualitative comparisons include both baselines and Equalizer, with Grad-CAM maps used for visualization.