PaperlayerView exact revision on arXiv

Source-linked AI summary

A Survey on Bias in Visual Datasets

Simone Fabbrizzi, Symeon Papadopoulos, Eirini Ntoutsi, Ioannis Kompatsiaris

arXiv:2107.07919v2cs.CV

TL;DR

Bias in visual datasets matters because computer-vision systems rely heavily on data that can encode and amplify prejudice, while comprehensive coverage of this problem is lacking. The paper surveys bias types, discovery and quantification methods, and bias-aware datasets, then concludes that discovery remains open and proposes a collection checklist.

  • Problem

    Computer-vision systems can learn and amplify biases in their data, yet no comprehensive survey addresses bias in visual datasets.

  • Method

    The paper surveys bias types in visual datasets, reviews bias-discovery and quantification methods, and discusses attempts to collect bias-aware datasets.

  • Results

    Bias discovery and quantification in visual datasets remains an open problem, with room to improve methods and the range of addressable biases.

  • Takeaways & Limitations

    Because bias-free datasets do not exist, researchers and practitioners should make dataset biases explicit and use a checklist during visual-data collection.

  • Takeaways & Limitations

    Class imbalance affects some facial-recognition algorithms but does not explain every performance disparity, since groups may differ in recognition difficulty even with balanced training data.

Abstract

from arXiv · show

Computer Vision (CV) has achieved remarkable results, outperforming humans in several tasks. Nonetheless, it may result in significant discrimination if not handled properly as CV systems highly depend on the data they are fed with and can learn and amplify biases within such data. Thus, the problems of understanding and discovering biases are of utmost importance. Yet, there is no comprehensive survey on bias in visual datasets. Hence, this work aims to: i) describe the biases that might manifest in visual datasets; ii) review the literature on methods for bias discovery and quantification in visual datasets; iii) discuss existing attempts to collect bias-aware visual datasets. A key conclusion of our study is that the problem of bias discovery and quantification in visual datasets is still open, and there is room for improvement in terms of both methods and the range of biases that can be addressed. Moreover, there is no such thing as a bias-free dataset, so scientists and practitioners must become aware of the biases in their datasets and make them explicit. To this end, we propose a checklist to spot different types of bias during visual dataset collection.

1 Introduction

Visual datasets can encode and amplify bias, creating discrimination risks in computer-vision applications. This survey addresses the limited research on biased visual data by reviewing bias types, discovery methods, bias-aware datasets, and a collection checklist.

  • 1 Introduction: Bias can enter data through correlations or causal influences involving protected attributes and through underrepresentation of protected groups.The ways bias manifests depend on the data type.
  • 1 Introduction: Visual data is especially difficult to analyze for bias because interpretation depends on subjects, depiction, spatial relationships, cultural references, background knowledge, and color.These features make understanding bias in images and videos particularly complex.
  • 1 Introduction: Deep-learning computer-vision systems depend heavily on training data, and deployed applications including facial recognition, gender classification, and autonomous driving have exhibited discriminatory behavior.The introduction connects data dependence with ethical risks across multiple CV applications.
  • 1 Introduction: Research on biased computer vision remains more limited than broader AI-bias research despite the abundance and widespread use of visual data.The authors identify no comprehensive survey covering bias in visual datasets, beyond work limited to object-detection datasets.
  • 1 Introduction: The survey describes visual-dataset bias types, reviews bias-discovery methods, examines bias-aware datasets, and proposes a checklist for spotting bias during collection.Its stated aim is to support bias discovery and less-biased data collection.

2 Manifestation of Bias in Visual Data

The survey categorizes visual-dataset bias into selection, framing, and label bias, examining how these arise during collection and affect computer-vision applications. It emphasizes that bias can produce unequal performance, encode problematic categories, and propagate through automated data pipelines.

  • Taxonomy: The survey distinguishes selection, framing, and label bias, while treating negative-class bias as an instance of selection and label bias.Its taxonomy generalizes earlier work focused on object-detection datasets and addresses discrimination against protected groups.
  • Selection bias: Selection bias occurs when the process of including subjects creates disparities or associations in a visual dataset.Examples include overrepresented image viewpoints, locations, demographic groups, or retrieval outcomes.
  • Selection bias: Demographic imbalance can produce severe performance disparities, including dark-skin error rates up to 18 times higher than light-skin error rates in some commercial gender-classification algorithms.Reported dataset proportions also show strong under-representation of darker-skinned females and males in Adience and IJB-A.
  • Selection bias: Selection imbalance reduces statistical certainty, causes standard loss functions to prioritize represented groups, and may require mitigation during training.Algorithmic or search-engine biases can also propagate into later datasets through automated collection and retrieval.
  • Caveats: Class imbalance does not explain every performance disparity because some groups may remain harder to recognize even with balanced training data.Higher within-group variance, such as variation from hairstyle or makeup, can also contribute to disparity.
  • Framing and label bias: Framing bias changes how subjects are depicted, while label bias concerns annotation errors or poorly defined semantic categories.Examples include disproportionate cropping of obese people, greater facial prominence for men in several image settings, and ambiguity around racial categories.
  • Label bias: Biased labels can create a feedback loop in which classification systems reinforce the original bias when labeling newly collected data.The survey also notes that physiognomic applications rely on problematic assumptions about inferring human traits from images.

1. Real world

Bias can shape visual content before collection because real-world inequalities influence how images are generated and what relationships they depict.

  • Real world: MS-COCO was more likely to associate kitchen objects with women, illustrating how social inequalities can be reflected in visual data generation.The dataset is used for object detection, segmentation, and captioning benchmarks.

2. Capture

Visual content capture introduces selection and framing bias through choices about subjects, composition, camera settings, lighting, and other aspects of image-making.

  • Capture: Capture can introduce selection bias when photographers or video makers choose which subjects appear.Selection bias may later appear as under-representation or non-representation of groups in datasets.
  • Capture: Composition, camera settings, and lighting can convey different messages and therefore create framing bias.
  • Capture: Historical discrimination can become selection and framing bias when it is not actively countered.

3. Editing

Digital editing and post-processing are important stages of visual-content production, but their powerful capabilities raise ethical questions about modifying images and videos.

  • Editing: Post-processing has become a fundamental skill alongside camera configuration, lighting, and shooting.
  • Editing: Powerful editing tools raise ethical questions about when and in which contexts digital modification is appropriate.

4. Dissemination

Visual-content dissemination occurs at enormous scale and can introduce selection and framing bias through choices about what is shared, where it appears, and how it is presented.

  • Dissemination: More than 500 hours of video are uploaded to YouTube every minute, making complete inspection of disseminated visual content impossible.
  • Dissemination: Dissemination involves selection and framing bias through chosen images, distribution channels, and attached captions or text.

5. Data collection

Visual-dataset collection can exclude or under-represent groups and can introduce label bias through errors and mislabelling in annotations.

  • Data collection: Collection processes can exclude or under-represent certain groups, creating selection bias in visual datasets.
  • Data collection: Annotations collected with datasets are vulnerable to label bias, including errors and mislabelling.

6. Algorithms

Algorithm training is presented as a key stage for addressing fairness and accountability because AI-powered systems increasingly affect citizens. The surrounding discussion also notes ethical issues from AI-generated media re-entering the Web.

  • Fairness and accountability become pressing concerns during model training as algorithm-powered systems are used pervasively in applications and services affecting citizens.
  • The paper raises implementation questions for fairness in AI and computer-vision algorithms, including their legal framing.
  • AI-generated media can re-enter the Web through the content life cycle, creating additional ethical issues.

3 Bias Discovery and Quantification in Visual Datasets

The survey reviews methods for discovering and quantifying bias in visual datasets, organizing them into four methodological categories. It finds useful but limited approaches whose applicability, interpretability, supervision requirements, and dependence on representations constrain comprehensive bias discovery.

  • The review asks whether the types of bias identified in visual datasets can be discovered or quantified, treating dataset quality as important for fair and trustworthy computer vision.
  • The literature search used keyword combinations across DBLP, arXiv, and Google Scholar, then expanded the manually filtered results through citations and related-work sections.
  • The reviewed methods fall into reduction to tabular data, biased image representations, cross-dataset bias detection, and other methods.
  • Reduction to Tabular Data: Mutual information can define dataset bias between a protected attribute and an output variable, while object- and scene-based metrics quantify co-occurrence patterns.
  • Biased Image Representations: Representation methods reveal biased associations, while latent-space interventions modify generated images along directions associated with selected attributes.
  • Cross-dataset Bias Detection: Cross-dataset methods use generalization and dataset-recognition performance to expose dataset signatures, but cannot identify the bias type without further inspection.
  • Discussion: Tabular reductions may be simplistic and label-dependent, representation methods depend on models and distance metrics, and cross-dataset methods require comparable datasets.

4 Bias-aware Visual Data Collection

The survey examines attempts to construct visual datasets with bias in mind, while showing that addressing one bias can leave or introduce others. It therefore proposes a checklist to help make possible selection, framing, and label biases explicit during collection.

  • Scope and purpose: Bias-aware datasets are purpose-specific attempts rather than universally bias-free resources, so their remaining biases must also be analyzed.The survey presents these datasets as case studies and proposes a checklist for bias-aware visual data collection.
  • Dataset examples: Crowdsourced face annotations can introduce label bias through inconsistent judgments and selection bias when disagreement leads to discarding images.The taxonomy of race categories can also fail to capture the complexity of human diversity.
  • Dataset examples: Controls for object rotation, viewpoint, and background increased diversity but produced indoor, center-aligned, rarely occluded objects and introduced other framing biases.Removing difficult or privacy-sensitive classes could also make negative classes less representative.
  • Dataset examples: The Inclusive Benchmark Database and Non-binary Gender Benchmark Database broaden demographic coverage but expose label-bias risks in representing multifaceted gender identities with simple categories.The authors identify gender as a complex sociocultural construct not necessarily tied to physical appearance.
  • Checklist and mitigation: The checklist asks about dataset purpose, collection procedures, subject imbalance, spurious correlations, harmful framing, fuzzy categories, and labeling practices.It is inspired by documentation and reflexive data practices while adding questions specific to selection, framing, and label bias.
  • Checklist and mitigation: Careful collection and curation are presented as broadly effective mitigation strategies, while selection bias is especially amenable to resampling or reweighting.The survey notes that pre-processing is less effective for framing messages requiring human inspection and analysis.

5 Conclusions and Research Outlook

The survey finds that visual-data bias is pervasive across the content life cycle and that bias-discovery methods remain limited, especially in their representations and coverage. It concludes that bias-free data is unattainable, making explicit bias documentation and improved collection practices essential.

  • Findings: Bias pervades computer vision, accompanies the entire visual-content life cycle, involves multiple actors, and can re-enter through biased algorithms.The survey characterizes selection, framing, and label bias as distinct manifestations in visual data.
  • Research outlook: The systematic review finds that most bias-discovery methods reduce visual data to tabular representations, leaving alternative representations largely unexplored.The authors suggest scene graphs could support methods for rankings, graphical data, and knowledge-graph embeddings.
  • Research outlook: Latent-space geometry may encode bias, but existing studies usually examine this relationship through variations of cosine similarity.The survey identifies more sophisticated geometric analyses, including Topological Data Analysis, as a promising direction.
  • Research outlook: Bias evaluation remains difficult for object-detection datasets because long-tailed object distributions and object cooccurrence affect mitigation methods.The authors call for deeper study of how cooccurrence influences approaches such as oversampling and undersampling.
  • Research outlook: The reviewed literature focuses mainly on images, single protected attributes, selection and framing bias, and facial data, leaving videos, label bias, multi-attribute fairness, and other application areas underexplored.The survey specifically identifies medical imaging and self-driving cars as fields needing improved datasets and collection practices.
  • Practical implications: Because no data is bias-free, researchers and practitioners should identify and explicitly document dataset biases using standardized collection guidance.The survey proposes a checklist to help spot possible bias causes and support less biased visual-data collection.
Loading 2107.07919v2…