Source-linked AI summary
Fairness And Bias in Artificial Intelligence: A Brief Survey of Sources, Impacts, And Mitigation Strategies
Emilio Ferrara
TL;DR
Biased AI systems can produce unfair outcomes, perpetuate inequalities, and reinforce stereotypes, including through generative AI. This survey synthesizes research on bias sources, impacts, mitigation strategies, and ethical challenges, concluding that fairer AI requires representative data, oversight, transparency, and continued attention to generative-model bias.
Problem
AI bias affects healthcare, employment, criminal justice, credit scoring, and generative media, where systems can produce unfair outcomes and perpetuate inequalities.
Method
The paper conducts a systematic, interdisciplinary literature review covering bias definitions, sources, impacts, mitigation strategies, and ethical considerations.
Results
The survey identifies data, algorithmic, human, and generative sources of bias and reviews pre-processing, model-selection, and post-processing mitigation approaches.
Takeaways & Limitations
Fairer AI requires diverse and representative data, transparent documentation, accountability, oversight, continuous evaluation, and strategies tailored to generative models.
Takeaways & Limitations
Mitigation effectiveness is constrained by difficulty collecting diverse representative data, privacy concerns, and differing definitions of fairness across groups and over time.
Abstract
from arXiv · showhide
The significant advancements in applying Artificial Intelligence (AI) to healthcare decision-making, medical diagnosis, and other domains have simultaneously raised concerns about the fairness and bias of AI systems. This is particularly critical in areas like healthcare, employment, criminal justice, credit scoring, and increasingly, in generative AI models (GenAI) that produce synthetic media. Such systems can lead to unfair outcomes and perpetuate existing inequalities, including generative biases that affect the representation of individuals in synthetic data. This survey paper offers a succinct, comprehensive overview of fairness and bias in AI, addressing their sources, impacts, and mitigation strategies. We review sources of bias, such as data, algorithm, and human decision biases - highlighting the emergent issue of generative AI bias where models may reproduce and amplify societal stereotypes. We assess the societal impact of biased AI systems, focusing on the perpetuation of inequalities and the reinforcement of harmful stereotypes, especially as generative AI becomes more prevalent in creating content that influences public perception. We explore various proposed mitigation strategies, discussing the ethical considerations of their implementation and emphasizing the need for interdisciplinary collaboration to ensure effectiveness. Through a systematic literature review spanning multiple academic disciplines, we present definitions of AI bias and its different types, including a detailed look at generative AI bias. We discuss the negative impacts of AI bias on individuals and society and provide an overview of current approaches to mitigate AI bias, including data pre-processing, model selection, and post-processing. We emphasize the unique challenges presented by generative AI models and the importance of strategies specifically tailored to address these.
I. INTRODUCTION
The survey examines the sources, impacts, and mitigation strategies of fairness and bias in AI. It emphasizes systematic discrimination and the need for comprehensive, interdisciplinary responses.
- AI bias can affect groups through discriminatory outcomes and perpetuate systemic inequality.
- The survey covers data, algorithmic, and user biases, together with their ethical implications.
- Proposed mitigation strategies include improving data quality and designing explicitly fair algorithms.
- The paper surveys bias sources, impacts, and mitigation approaches to support more responsible and ethical AI systems.
II. SOURCES OF BIAS IN AI
AI bias can arise from data collection, algorithm design, and human interaction, producing unfair outcomes across applications. The survey also highlights generative bias, where models reproduce skewed representations from training data.
- Bias is a systematic error that can arise from data collection, algorithm design, and human interpretation.
- User bias occurs through biased human interactions with AI systems, complementing data and algorithmic sources.
- Data bias results from unrepresentative, incomplete, or erroneous training data, while algorithmic bias reflects biased assumptions or decision criteria.
- Generative models predominantly depicted CEOs as men and criminals or terrorists as people of color.
- Diverse, balanced training data are presented as necessary for fairer generative outputs because internet-sourced images mirror existing disparities.
- Generative bias produces skewed representations when generated content disproportionately reflects attributes or patterns in training data.
- Text and image generators can overrepresent Western norms or struggle to represent diverse ethnicities when training data lack cultural and demographic diversity.
III. IMPACTS OF BIAS IN AI
Biased AI can discriminate against marginalized groups, restrict access to services and opportunities, and reinforce harmful social narratives. These effects raise ethical concerns about accountability, transparency, trust, and human autonomy.
- Biased AI can perpetuate or amplify existing inequalities, causing discrimination against marginalized groups.
- Biased algorithms can produce unfair criminal-justice outcomes, including wrongful convictions or harsher sentences for people of color.
- Bias can restrict access to healthcare, finance, employment, and other essential opportunities for underrepresented groups.
- Facial recognition and generative models can reinforce gender stereotypes by performing poorly on women or depicting CEOs predominantly as men.
- Generative models can introduce discrimination based on skin color and ethnicity by portraying criminals or terrorists as people of color.
- Widespread deployment can entrench discriminatory narratives, shape cultural norms, undermine trust, and limit human agency and autonomy.
- Developers, companies, and governments share responsibility for fair, transparent systems and accountable responses to discriminatory outcomes.
IV. MITIGATION STRATEGIES FOR BIAS IN AI
The survey reviews data pre-processing, model selection, and post-processing as major approaches to mitigating AI bias, while emphasizing their practical and ethical limitations. It also argues that generative AI requires a holistic strategy combining representative data, transparent models, and careful consideration of societal effects.
- Pre-processing: Pre-processing addresses bias before training through representative data, oversampling, undersampling, synthetic data generation, augmentation, and adversarial debiasing.The survey also emphasizes documenting dataset biases and augmentation procedures.
- Model Selection: Model selection prioritizes fairness through group- or individual-fairness criteria, regularization, ensemble methods, and classifiers targeting demographic parity.These methods seek to penalize discriminatory predictions or reduce bias by combining multiple models.
- Post-processing: Post-processing adjusts model outputs to achieve fairness criteria such as equalized odds across demographic groups.The approach can require substantial additional data and may be complex to implement.
- Generative AI: Generative AI bias requires a holistic strategy beginning with diverse, representative data and extending to transparent model selection and broader ethical considerations.The passage specifically frames generative AI as requiring tailored mitigation beyond standard procedures.
- Limitations and Challenges: Mitigation approaches face limited training-data diversity, difficulty identifying and measuring bias, and trade-offs between fairness and accuracy.Bias may arise from multiple sources, and modifying algorithms can reduce accuracy for some groups or contexts.
- Limitations and Challenges: Ethical choices remain about which forms of bias and affected groups to prioritize, because fairness adjustments can produce trade-offs and unintended outcome distributions.The survey presents bias mitigation as a complex, multifaceted challenge rather than a single technical solution.
V. FAIRNESS IN AI
The survey presents fairness in AI as the absence of bias or discrimination, with group, individual, counterfactual, procedural, and causal formulations. These formulations can overlap or conflict, so fairness requires context-sensitive choices and attention to how AI affects individuals and groups.
- Definition of Fairness in AI: Fairness in AI is a complex, debated concept involving the absence of bias or discrimination and several proposed fairness types.The survey identifies group, individual, and counterfactual fairness among the principal formulations.
- Types of Fairness: Group fairness seeks equal or proportional treatment across groups through criteria including demographic parity, disparate mistreatment, and equal opportunity.These criteria concern outcome distributions or misclassification and positive-rate measures across demographic groups.
- Types of Fairness: Individual fairness seeks similar treatment for similar individuals regardless of group membership, using similarity- or distance-based measures.Counterfactual fairness extends the idea to hypothetical changes in group membership or attributes.
- Types of Fairness: Procedural fairness emphasizes fair and transparent decision processes, while causal fairness focuses on avoiding the perpetuation of historical biases and inequalities.These approaches broaden fairness beyond outcome parity alone.
- Fairness and Bias: Fairness definitions can conflict, and achieving fairness is not one-size-fits-all because context and stakeholders influence the relevant trade-offs.Addressing bias is important but may require additional efforts to achieve fairness.
- Fairness and Bias: Fairness and bias differ because bias is systematic deviation in outputs, whereas fairness concerns discrimination or favoritism based on protected characteristics.The survey characterizes fairness as a deliberate social and ethical goal, not only a technical property.
- Real-world Examples: Examples from criminal justice and recruitment show that biased AI can disadvantage racial or gender groups, while mitigation tools aim to reduce such disparities.The cited examples include COMPAS and gender-decoder tools for job postings.
VI. MITIGATION STRATEGIES FOR FAIRNESS IN AI
The survey describes group and individual fairness, transparency, accountability, and explainability as complementary approaches to fair AI. It emphasizes that these approaches remain constrained by competing fairness goals, ambiguous definitions, limited attention to intersectionality, and possible unintended consequences.
- Fairness Approaches: Transparency makes decision processes visible, accountability assigns responsibility for harm, and explainability makes decisions understandable to users.These approaches address visibility, responsibility, and interpretability as distinct aspects of fair AI.
- Limitations: Group fairness may produce unequal treatment within groups, while individual fairness may fail to address systemic biases affecting entire groups.The survey also notes that group metrics may overlook intersectionality.
- Limitations: Fairness definitions differ among people and groups and can change over time, complicating the design of systems accepted as fair by all stakeholders.This creates difficulty in selecting and balancing fairness criteria for a context.
- Limitations: Statistical fairness approaches may not capture complex human decision-making or interactions among race, gender, and socioeconomic status.The survey identifies this intersectional limitation as a weakness of some group fairness metrics.
- Limitations: Attempts to ensure fairness can have unintended harmful effects, including increased racial disparities in arrests in predictive-policing contexts.The survey presents this as a concern about mitigation outcomes rather than a universal consequence.
- Conclusion: The survey concludes that fair and equitable AI remains an ongoing research challenge requiring approaches sensitive to contextual nuances of fairness and equity.It calls for continued development of approaches across contexts.
- Fairness Approaches: Group fairness targets equal treatment across demographic groups through re-sampling, pre-processing, or post-processing, while individual fairness targets comparable treatment for individuals.Individual fairness can use counterfactual or causal fairness techniques.
VII. CONCLUSIONS
The paper concludes that AI bias can be amplified by generative AI, requiring comprehensive mitigation across the AI development pipeline. It highlights tailored strategies, ethical safeguards, and diversified data and teams as priorities for more responsible systems.
- Fairness requires considering interactions among multiple identity dimensions and addressing intersectionality in AI system design.The paper calls for systems that account for intersecting identities and fairness across multiple dimensions.
- Generative AI introduces distinctive bias challenges that require mitigation strategies specifically tailored to synthetic data and content generation.The paper emphasizes generative models as an emerging concern within its broader discussion of AI bias.
- Bias mitigation must address the entire AI development pipeline through robust data practices, counterfactual fairness, transparency, oversight, and continuous evaluation.The paper identifies these strategies alongside unbiased data collection and diverse, representative datasets.
- Responsible AI development should diversify training data and teams while documenting training data, model choices, and generative processes transparently.The paper presents diversification and documentation as future research and development priorities.
- Ethical and legal frameworks should make privacy, transparency, and accountability foundational elements of the AI development lifecycle.The paper argues these safeguards should not be treated as afterthoughts.