PaperlayerView exact revision on arXiv

Source-linked AI summary

Image Representations Learned With Unsupervised Pre-Training Contain Human-like Biases

Ryan Steed, Aylin Caliskan

arXiv:2010.15052v3cs.CYcs.CV

TL;DR

Unsupervised vision models learn from massive web-image datasets, raising the question of whether they also acquire social biases. The paper develops iEAT to measure biased associations in image representations and finds racial, gender, and intersectional biases in ImageNet-pretrained models, with results often matching documented human biases. The findings suggest that stereotypical portrayals of social groups online can be reflected in learned representations and warrant analysis before consequential use.

  • Problem

    The paper asks whether unsupervised computer-vision models learn implicit social biases from massive web-scraped image datasets despite having no labels during training.

  • Method

    The paper develops iEAT, adapting embedding association tests to image representations, and applies it to iGPT and SimCLRv2 pretrained on ImageNet.

  • Results

    The models show racial, gender, and intersectional biases, and 8 tests match documented human biases, including 4 of 5 biases also found in language models.

  • Takeaways & Limitations

    Unsupervised pretraining can embed biases from stereotypical portrayals in web images, so practitioners should analyze pretrained models before consequential use.

  • Takeaways & Limitations

    Google Image Search stimuli can underrepresent minority groups and therefore underestimate bias, while nonsignificant results do not necessarily indicate no bias.

Abstract

from arXiv · show

Recent advances in machine learning leverage massive datasets of unlabeled images from the web to learn general-purpose image representations for tasks from image classification to face recognition. But do unsupervised computer vision models automatically learn implicit patterns and embed social biases that could have harmful downstream effects? We develop a novel method for quantifying biased associations between representations of social concepts and attributes in images. We find that state-of-the-art unsupervised models trained on ImageNet, a popular benchmark image dataset curated from internet images, automatically learn racial, gender, and intersectional biases. We replicate 8 previously documented human biases from social psychology, from the innocuous, as with insects and flowers, to the potentially harmful, as with race and gender. Our results closely match three hypotheses about intersectional bias from social psychology. For the first time in unsupervised computer vision, we also quantify implicit human biases about weight, disabilities, and several ethnicities. When compared with statistical patterns in online image datasets, our findings suggest that machine learning models can automatically learn bias from the way people are stereotypically portrayed on the web.

1 INTRODUCTION

The paper asks whether unsupervised image representations learn human-like social biases from web-scraped image data. It introduces iEAT to quantify such associations and reports racial, gender, and intersectional biases in state-of-the-art models.

  • Motivation: Unsupervised image representations may embed human-like biases even when trained without labels.The paper connects this expectation to stereotypical patterns in web images and analogous findings for language embeddings.
  • Contribution: The paper introduces iEAT, a systematic method for detecting and quantifying social bias in unlabeled-image representations.The method adapts embedding association testing to images.
  • Findings: Two state-of-the-art ImageNet-pretrained models show statistically significant racial, gender, and intersectional biases.The models are iGPT and SimCLRv2.
  • Findings: Across 15 documented tests, 8 machine results match documented human biases, including 4 of 5 biases also found in large language models.The seven nonsignificant tests use IATs with only small samples of picture stimuli.
  • Broader analysis: The paper compares model results with statistical analyses of race and gender in image datasets and presents a case study of sexualization bias in image generation.These analyses examine how stereotypical portrayals on the web relate to learned model associations.

2 RELATED WORK

Prior work established systematic tests for bias in language embeddings and documented social bias in supervised vision systems. This paper extends embedding-association ideas to unsupervised computer vision, where existing visual-dataset methods still rely on human annotations.

  • Research gap: Existing machine-bias tests primarily target language embeddings, while a visual-dataset method still depends on human annotations.The paper positions iEAT as an annotation-free approach for unsupervised image representations.
  • Embedding association tests: Embedding association tests adapt the Implicit Association Test by measuring differential associations between target concepts and evaluative or stereotypical attributes.In language, these tests use representation geometry rather than human response latency.
  • Test scope: Validated tests span innocuous associations, such as flowers versus insects, and potentially harmful associations involving social groups, age, or disability.The same framework can examine both broadly shared stereotypes and socially consequential biases.
  • Connection to NLP: The paper’s approach is closest to contextualized language methods that pool representations, but it pools over images rather than words.This transfers an established representational strategy from NLP to image embeddings.

3 APPROACH

The approach adapts embedding association tests to image representations, replacing verbal stimuli with systematically collected images and evaluating valence, stereotype, and intersectional associations. It uses differential-association statistics, permutation significance tests, and standardized effect sizes.

  • Framework: iEAT applies the WEAT statistical framework to image embeddings, using the same test statistic, effect size, and p-value structure.The method is designed for measuring biased associations in representations rather than human response latency.
  • Replication tests: The replication tests measure associations of two target concepts with pleasant versus unpleasant images or with paired stereotypical attributes.Examples include male versus female with career versus family attributes.
  • Image stimuli: When prior tests use verbal stimuli, the paper maps their concepts to images and collects stimuli from CIFAR-100 or Google Image Search using controls for visual confounds.The procedure uses multiple images, isolated subjects or objects, and balanced representation where applicable.
  • Intersectional tests: The intersectional tests compare White males with White females, Black males, and Black females to examine race, gender, and intersectionality hypotheses.The hypotheses concern emergent intersectional effects and whether racial and gender differences resemble specific subgroup contrasts.
  • Statistics: The test statistic compares differential associations of target sets X and Y with attribute sets A and B, while permutation tests evaluate significance.Effect size d standardizes the separation between the two relative associations.
  • Interpretation: A positive result supports an association between the sampled image sets, not automatically an association between abstract social concepts.Generalization depends on whether the image samples adequately represent the intended categories.

4 COMPUTER VISION MODELS

The study examines two state-of-the-art unsupervised image-representation models, iGPT and SimCLRv2, both pretrained on ImageNet, and extracts high-quality features for evaluating image associations.

  • The study focuses on iGPT and SimCLRv2, two unsupervised models selected for state-of-the-art linear-evaluation performance.Both models were published in summer 2020 and use different architectures.
  • Both models are pretrained on ImageNet 2012, a large web-derived benchmark containing 1.2 million annotated images across 200 object classes.The authors assume ImageNet’s portrayals of people reflect portrayals across the web, while noting that contemporary verification remains future work.
  • iGPT: iGPT autoregressively predicts the next pixel in flattened image sequences, and its extracted embeddings can support linear classification.The model adapts the GPT-2-style next-token prediction objective to cropped images.
  • iGPT: The authors use iGPT embeddings from a middle layer, specifically f_20, because the final autoregressive-prediction layer may be less suitable for object recognition.They also hypothesize that stronger linear-evaluation features may be more likely to encode stereotypical visual associations.
  • SimCLRv2: SimCLRv2 uses a ResNet encoder and learns representations through augmented views of the same image, with the average-pooled encoder output used after pretraining.The method includes cropping, color distortion, and Gaussian blur during augmentation.

5 STIMULI

The authors construct image stimuli to approximate established psychological IAT categories, using representative visual exemplars while addressing the difficulty of depicting abstract social concepts and demographic imbalance.

  • The stimulus sets systematically represent target concepts and attributes while adhering closely to stimuli from validated psychological studies.Images came from original IAT stimuli, CIFAR-100, or Google Image Search.
  • Google Image Search stimuli can underrepresent demographic groups and therefore underestimate stereotype-congruent bias, while intersectional tests require separate equal-sized demographic image sets.The authors specifically collected equal-sized sets for White male, White female, Black male, and Black female categories.
  • Abstract concepts such as pleasantness are represented by searching for images corresponding to verbal IAT exemplars rather than searching directly for the abstract label.Examples include orchids for flowers, centipedes for insects, sunsets for pleasantness, and morgues for unpleasantness.
  • The collection procedure controls potential image confounds by gathering multiple images per verbal exemplar and standardizing selection across visual characteristics.The cited procedure considers factors such as lighting, background, dominant colors, and object placement.
  • Choosing Valence Stimuli: Valence stimuli use a consolidated set of high- and low-valence words from an experimentally validated database, prioritizing words with strong imagery when direct physical depiction is difficult.Human participants rated the source words for pleasantness and imagery in a controlled laboratory setting.

6 EVALUATION

The evaluation tests whether image-representation associations correspond to established human and language-model biases and checks whether the significance procedure produces false positives.

  • The iEAT validation compares image-model results with human IATs and word-embedding WEATs, including a baseline flower–insect versus pleasant–unpleasant association.The flower–insect test is treated as a near-universal human-bias baseline.
  • 8 of 15 replicated human IATs showed significant bias in embeddings from at least one of the two tested models.This result provides the paper’s principal predictive-validity comparison with established human bias measurements.
  • 10.3% of random partitions produced false positives at p < 10^-1, while 1.2% produced statistically significant false positives at p < 10^-2.The random-partition test used 1,000 partitions of the flower–insect stimulus sets.

7 EXPERIMENTS AND RESULTS

Unsupervised image representations from iGPT and SimCLRv2 contain significant racial, gender, intersectional, and other human-like biases, with patterns varying across models and tests. These associations align with documented human biases and may reflect stereotypical or unequal portrayals in ImageNet and other online image datasets.

  • Model Comparison: SimCLRv2 has stronger biases than iGPT but fewer bias types, possibly because its image transformations and greater architectural complexity favor concrete object classification.A linear classifier reaches 88.5% accuracy on CIFAR-100 with iGPT embeddings and 89% with SimCLRv2 embeddings.
  • Racial Biases: Both iGPT and SimCLRv2 exhibit significant racial biases, including Arab-Muslim valence associations and White–Black weapon stereotypes.iGPT also shows a Skin Tone effect size of 1.26 with p<10^-2.
  • Gender Biases: Both models show significant Gender-Career bias, with iGPT effect size 0.62 and SimCLRv2 effect size 0.74.The associations pair male with career attributes and female with family-related attributes.
  • Other Biases: iGPT shows a thin–overweight pleasantness bias with effect size 1.67 and p=10^-4, while several ethnicity, disability, and age tests show no significant bias.The authors note that some null results may reflect fewer than 10 picture stimuli and do not establish bias absence.
  • Intersectional Valence: iGPT’s intersectional valence results most consistently match social-psychology hypotheses, including a White-versus-Black effect size of 1.16 with p<10^-3.The largest differential valence bias compares White females with Black males: effect size 1.46, p<10^-3.
  • Intersectional Stereotypes: Intersectional gender stereotypes are significant but contradictory: White male–White female Gender-Career bias has effect size 0.97, while the White male–Black male career-family association reverses direction with effect size 0.89.The general male-versus-female Gender-Career effect is 0.81, whereas the White male–Black female comparison is insignificant.
  • Potential Data Origins: Unequal representation and stereotypical contexts in ImageNet and other web image datasets could allow unsupervised models to learn social associations without explicit social labels.Examples include unequal race and gender representation and gender-linked differences between home or hotel versus industrial and construction scenes.

8 DISCUSSION

The study shows that unsupervised image representations can learn and embed human-like social biases from web-derived training images, while highlighting important dataset and mitigation limits.

  • Biases can be learned automatically from images and embedded in general-purpose representations.
  • iGPT image completions provide a qualitative case study of how stereotypes may propagate despite limited significant bias in next-pixel prediction features.
  • The findings replicate four of five human biases also found in large language models and suggest similar effects from stereotypical online portrayals.
  • Visual association testing captures biases that may affect humans and machines but cannot be captured in text corpora.
  • ImageNet 2012 may not represent the full population of internet images and was designed for object detection rather than social-attribute recognition.
  • Updated ImageNet person-category data could change the results, while larger-scale pretraining remains costly and bias mitigation remains an open problem.

9 CONCLUSIONS

The paper introduces a principled image-based bias measure and uses it to replicate documented biases while identifying additional social biases in unsupervised image models.

  • The method adapts embedding association tests from language to measure bias in unsupervised image models.
  • The study replicates validated bias tests and documents several social biases, including severe intersectional bias.
  • The results suggest that unsupervised image models learn human biases from how people are portrayed in images.

A ATTRIBUTE WORDS

The attribute stimuli use positive and negative words selected for high imagery and extreme valence.

  • The words were selected using imagery and valence scores from a laboratory experiment.
  • Positive attributes include baby, ocean, beach, butterfly, gold, rainbow, sunset, money, diamond, flower, and sunrise.
  • Negative attributes include devil, morgue, slum, corpse, coffin, jail, roach, funeral, prison, vomit, and crash.

B STIMULI COLLECTION PROCEDURE

The stimulus collection procedure combines existing CIFAR-100 categories with controlled Google Image Search retrieval, filtering, clarification, and cropping steps.

  • For each verbal stimulus, the procedure records the stimulus, search terms, and number of collected images.
  • Matching CIFAR-100 categories supply random image samples when available.
  • Otherwise, the procedure searches each stimulus verbatim in a private Chrome window with Safe-Search off and accepts the first qualifying results.
  • Accepted images must isolate the specified object, person, or scene and use plain backgrounds without watermarks or text.
  • Clarifying search terms are added when the first 50 results fail the criteria, and each image is cropped squarely for iGPT.
  • Images are centered so the complete object, person, or scene remains visible.
  • Some abstract stimuli were not visualized, while difficult stimuli could receive only one qualifying image.

C DISPARATE BIAS ACROSS MODEL LAYERS

Social biases vary across both model architectures and layers, so embeddings from iGPT’s next-pixel prediction layer do not reproduce the human-bias correspondence observed in image-classification embeddings.

  • C DISPARATE BIAS ACROSS MODEL LAYERS: Biases vary between models trained on the same data and across layers within a single model.The paper contrasts high-quality middle-layer embeddings with iGPT’s next-pixel prediction layer.
  • C DISPARATE BIAS ACROSS MODEL LAYERS: Next-pixel prediction embeddings lack the baseline Insect-Flower valence bias found in image-classification embeddings.They show significant bias only at the 10^-1 level for the Gender-Science and Sexuality IATs.
  • C DISPARATE BIAS ACROSS MODEL LAYERS: The mismatch between middle-layer and projection-head biases is consistent with social bias being encoded disparately across network layers.Different layers learn different levels of abstraction, such as lighting and curves.
  • C DISPARATE BIAS ACROSS MODEL LAYERS: Table 3 reports iEAT association effect sizes d and permutation p-values for iGPT next-pixel prediction embeddings.Effect sizes are categorized as small at 0.2, medium at 0.5, and large at 0.8.
Loading 2010.15052v3…