PaperlayerView exact revision on arXiv

Source-linked AI summary

Manifestations of Xenophobia in AI Systems

Nenad Tomasev, Jonathan Leader Maynard, Iason Gabriel

arXiv:2212.07877v2cs.CYcs.AI

TL;DR

ML fairness frameworks have paid comparatively little attention to xenophobia, despite its distinct potential harms. The paper defines a framework for those harms and reviews their possible manifestations across AI domains; it concludes that ethical AI design calls for inclusive, xenophilic technology.

  • Problem

    Prominent ML fairness frameworks do not comprehensively measure or mitigate xenophobic harms, a comparatively neglected form of discrimination in AI fairness research.

    Source: §1 · p. 1

  • Method

    The paper defines xenophobia and its harms, reviews possible manifestations across five AI domains, and draws on those cases to develop design recommendations.

    Source: §1 · p. 2, §4.1 · p. 12

  • Results

    The review finds that historical and contemporary xenophobic attitudes may manifest in AI data and systems as material, representational, and broader societal harms.

    Source: §5 · p. 14

  • Takeaways & Limitations

    The authors argue that ethical AI design calls for xenophilic technology that promotes civic inclusion and opposes the othering of marginalized groups.

    Source: §5 · p. 14

    Participatory AI practices remain challenging to implement, and their outcomes depend on including marginalized communities, especially people facing service-access or language barriers.

    Source: §3.3.2 · p. 8

Abstract

from arXiv · show

Xenophobia is one of the key drivers of marginalisation, discrimination, and conflict, yet many prominent machine learning (ML) fairness frameworks fail to comprehensively measure or mitigate the resulting xenophobic harms. Here we aim to bridge this conceptual gap and help facilitate safe and ethical design of artificial intelligence (AI) solutions. We ground our analysis of the impact of xenophobia by first identifying distinct types of xenophobic harms, and then applying this framework across a number of prominent AI application domains, reviewing the potential interplay between AI and xenophobia on social media and recommendation systems, healthcare, immigration, employment, as well as biases in large pre-trained models. These help inform our recommendations towards an inclusive, xenophilic design of future AI systems.

1 INTRODUCTION

Fairness research has largely centered on race and gender, while xenophobia—discrimination directed at people perceived as foreign—has received comparatively little attention. The paper argues for examining xenophobic harms across AI domains, beyond social-media content moderation.

1 INTRODUCTION

  • Xenophobia is comparatively under-addressed in ML fairness, whose frameworks often focus on legally protected categories such as race and gender [294].The paper describes xenophobia as discrimination directed at people perceived to be foreign.Source: §1 · p. 1
  • Ignoring xenophobia can leave distinct harms tied to perceived foreignness outside the scope of social and national justice.The authors describe this omission as creating a “normative loophole.”Source: §1 · p. 1
  • Existing mitigation often targets legally protected groups, although people deemed foreign may lack such protection and in-group boundaries can shift with data collection.The US census illustrates how changes in data collection may reflect and entrench changing notions of identity over time.Source: §1 · pp. 1–2
  • The paper reviews xenophobia across five AI domains, extending analysis beyond social-media hate- and dangerous-speech moderation to structural barriers facing immigrants and foreign nationals.The domains are social media, healthcare, immigration, employment, and large pre-trained models.Source: §1 · p. 2

2 ON XENOPHOBIA

The paper treats xenophobia as distinct from racism and defines it through perceived foreignness and civic membership. It separates assessments against prevailing social standards from ethical critique of those standards, and identifies three kinds of potential harm.

2 ON XENOPHOBIA

  • Xenophobia includes hostility toward people construed as outsiders and can have both attitudinal and institutional or structural expressions.The paper’s account includes rules and resource distributions that sustain discriminatory practices or outcomes.Source: §2 · p. 2
  • The authors define xenophobia as discrimination based on perceived foreignness, understood as purported lack of full civic membership; actual citizenship does not rule it out.Accent, name, practices, beliefs, or race may lead citizens to be prejudicially deemed foreign.Source: §2 · p. 3
  • The framework distinguishes immanent assessments against societal standards from transcendent assessments against ethical principles that actors may not formally accept.Differential outcomes beyond officially sanctioned citizen–non-citizen distinctions count as immanently xenophobic in the paper’s account.Source: §2 · p. 3
  • The paper identifies material disadvantage, denial of ethical recognition, and restrictions on rights as three potential harms from xenophobic AI.Rights harms may include biased information, unfairly allocated legal access, or unjustified prominence given to groups on speech platforms.Source: §2 · p. 3
  • The paper distinguishes xenophobia from racism: discrimination may target immigrants or refugees even when they are not perceived as racially distinct.The distinction matters because strategies to mitigate racism and xenophobia may differ.Source: §2 · pp. 2–4

3 PRACTICAL CONSIDERATIONS

The paper’s domain analysis is intended to identify how xenophobic discrimination may manifest and where countermeasures may be needed. Because direct testing is scarce, its evidence is suggestive and indirect, motivating more urgent evaluation frameworks.

3 PRACTICAL CONSIDERATIONS

  • The domain review draws on suggestive, somewhat indirect evidence and analogies from wider sociotechnical literature because testing for xenophobia remains limited.The authors argue this evidence gives sufficient reason to think xenophobia is likely to manifest in AI systems.Source: §3 · p. 4
  • The review aims to identify possible manifestations of xenophobic discrimination across AI domains and where countervailing measures may be needed.The stated purpose is to draw attention to a fairness issue rarely addressed by AI researchers.Source: §3 · p. 4
  • The authors call for greater urgency in developing frameworks to evaluate potential xenophobic bias and downstream harms.This call follows from their assessment that the available suggestive evidence warrants concern.Source: §3 · p. 4

3.1 Social media

Social media can both amplify xenophobia and support inclusive discourse, with effects shaped by platform design and social context. The section describes risks from personalized feeds, harmful content, and moderation, alongside monitoring and recommender-system approaches that may help counter them.

3.1 Social media

  • Social media may amplify xenophobia, especially where it is already present, while also offering space for inclusive discourse and support for marginalized groups.Its influence depends on social context, platform design, algorithms, and moderation policies.Source: §3.1 · p. 4

3.1.1 Risks.

  • Personalized feeds and repeated exposure can reinforce stereotypes, while shaming, hate speech, or weaponized moderation may ostracize or silence foreign communities.The authors also describe deepfakes and disinformation as risks in xenophobic narratives.Source: §3.1.1 · pp. 4–5

3.1.2 Promise.

  • Improved NLP monitoring may support targeted counter-speech and interventions, while human-in-the-loop filtering and explainability may help address language-system biases.The section also identifies semiautomatic detection of fake news, bots, and troll accounts as complementary efforts.Source: §3.1.2 · p. 5
  • Diversified recommendations and fairer systems may help avoid filter bubbles, but content diversity needs additional countermeasures and the best contextual approach remains an open question.The section also points to machine translation and better low-resource-language performance as ways to support cross-group communication.Source: §3.1.2 · p. 5

3.2 Immigration

AI-enabled migration management may improve efficiency, but risk-based systems can restrict movement and rights through assumptions tied to data profiles. The section weighs these risks against possible faster, fairer processes and refugee placement, while calling for auditing.

3.2 Immigration

  • Migration policy balances internal security with humanitarianism, while digital control systems risk restricting movement based on assumptions of risk or threat associated with data profiles.The section also notes that marginalized immigrants face additional integration barriers and may conceal identity amid xenophobic attitudes.Source: §3.2 · pp. 5–6

3.2.1 Risks.

  • AI border-control tools can use migrant data, social-media accounts, immigration forms, and facial recognition, while risk assessments may perpetuate historical, national, ethnic, and racial stereotypes.Associative assumptions about risk can produce harmful and unfair outcomes, including bias against immigrant groups in visa assessment and questions about migrants’ human rights.Source: §3.2.1 · p. 6

3.2.2 Promise.

  • AI may support faster, more efficient migration management and refugee matching, but improved auditing is needed to reduce the risk of unjust exclusion from safe refuge.Migration-flow modelling may also support anticipatory planning and sustainable, inclusive communities, while refugee-flow modelling may help expedite humanitarian responses.Source: §3.2.2 · pp. 6–7

3.3 Healthcare

Xenophobia in healthcare can shape policy, clinical practice, and patient trust, with AI potentially encoding or amplifying discriminatory harms. The section also describes possible access and diagnostic benefits, while stressing inclusive participation and practical deployment obstacles.

3.3 Healthcare

  • Medical xenophobia can arise through health-worker attitudes and entrenched public-health systems, disproportionately affecting refugees and undocumented migrants and extending beyond direct medical care.The section calls for holistic mitigation that includes social well-being and social determinants of health.Source: §3.3 · p. 7

3.3.1 Risks.

  • Clinical AI trained on retrospective data may perpetuate biases toward migrants and ethnic minorities, leading to misdiagnosis, suboptimal treatment, resource-allocation harms, and worse outcomes.Identity representation, proxy features, and designs unsuited to minority demographics may contribute to exclusion, delays, and alienation.Source: §3.3.1 · pp. 7–8

3.3.2 Promise.

  • AI healthcare prototypes may support diagnostics and on-device screening for marginalized groups, but safe deployment at scale faces practical obstacles and requires inclusive participation.Participatory development must include marginalized communities, especially people facing barriers to healthcare access or language barriers.Source: §3.3.2 · p. 8

3.4 Employment

Employment discrimination harms foreign workers in ways shaped by immigration status, nationality, and other intersecting factors. AI may reproduce historical discrimination through retrospective data and proxies, while explainability, fairness checks, and auditing offer potential tools for improvement.

3.4 Employment

  • Employment discrimination affects foreign workers differently across citizenship and immigration status, and may intersect with heritage, names, appearance, gender, and integration barriers.Unequal employment access and compensation can affect socioeconomic status, health, housing, safety, livelihood, and identity.Source: §3.4 · pp. 8–9

3.4.1 Risks.

  • AI hiring systems using retrospective data may perpetuate historical discrimination against out-groups, including through proxy features, while legal distinctions between immigrant workers require tailored fairness policies.The section also notes that AI’s wider effects on job quality across industries and regions matter beyond its immediate use in hiring decisions.Source: §3.4.1 · p. 9

3.4.2 Promise.

  • Assistive employment technologies may improve candidate experience and diversity, while ML models, explainability, fairness frameworks, and simulations may help audit hiring practices.The section calls for transparency and explicit fairness checks across candidates’ national and ethnic backgrounds and immigration statuses.Source: §3.4.2 · pp. 9–10

3.5 Stereotypes in Large Pretrained Models

Large pretrained models inherit risks from broad, difficult-to-curate training data and may reproduce xenophobic stereotypes across applications. The section discusses cultural harms and calls for cross-cultural reasoning, varied mitigation approaches, and participatory design.

3.5 Stereotypes in Large Pretrained Models

  • Large pretrained models may carry biases and stereotypes into downstream applications, with risks spanning text, image, and multimodal systems.Cultural artifacts can encode xenophobic narratives through literary erasure, limited translation, and negative portrayals of out-groups, strengthening the case for auditing models and datasets.Source: §3.5 · p. 10

3.5.1 Risks.

  • LLMs may expose private information, disparage out-groups, enable malicious users, reinforce stereotypes through interaction, and contribute to socioeconomic or environmental harms.Prior studies cited in the section report extremist-view emulation and ethnic biases, including in multimodal systems; the authors call for more focused investment in data curation.Source: §3.5.1 · pp. 10–11

3.5.2 Promise.

  • The authors call for AI systems that challenge exclusionary cultural narratives through cross-cultural knowledge and reasoning rather than stochastic reenactment of historical patterns.Potential mitigations include multilingual models, diverse imagery, self-supervised learning, data curation, human feedback, and participatory design; equal performance remains an open challenge for multilingual models.Source: §3.5.2 · p. 11

4 TOWARDS XENOPHILIC AI SYSTEMS

Xenophobic harms can accumulate across digital services, creating added precarity for people perceived as foreign. The section calls for inclusive design and more comprehensive measurement and mitigation, including participatory and interdisciplinary approaches.

4.1 A moral imperative

  • Xenophobic discrimination can compound across digital services, exposing people perceived as foreign to harms in social media, immigration, healthcare, employment, and other domains.The authors describe this as intersectional vulnerability and argue that it warrants special protection in system design and deployment.Source: §4.1 · p. 12
  • Xenophilic technology aims to foster civic inclusion and cross-cultural relationships, while responsible design may help reduce material and representational harms and respect human rights.The authors call for technologists to partner with domain experts and draw on the UDHR.Source: §4.1 · p. 12

4.2 Measurement and mitigation

  • Xenophobic impact is not commonly evaluated, and existing fairness approaches need better data, meaningful group definitions, and benchmarks suited to potentially global systems.The passages highlight limitations in race and ethnicity data, including coarse categories, discrepancies between recorded and self-reported identities, and collection, privacy, and governance challenges.Source: §4.2 · p. 12, §4.2.1 · p. 13

4.2.1 Measuring xenophobia.

  • Counterfactual and contrastive methods may help identify discrimination pathways, but rely on meaningful group categories; individual fairness may help when reliable categories are unavailable.Some social categories may not support meaningful counterfactual manipulation, limiting reliable assessment.Source: §4.2.1 · p. 13
  • Measurement should combine quantitative and qualitative assessment with community participation and local knowledge, while examining perpetrators, in-group radicalization, power shifts, and material, recognition, and rights harms.The account stresses that who is considered foreign depends on local histories and changing cultural and political contexts.Source: §4.2.1 · p. 13

4.2.2 Mitigation strategies.

  • Mitigation requires technical and non-technical strategies; under incomplete data, practitioners may use plausible groupings and robustness approaches, complemented by audits, explainability, escalation, and recourse.Proxy targets are an alternative but carry stereotyping risks, and purely technical approaches rarely map onto socially acceptable outcomes.Source: §4.2.2 · pp. 13–14

5 CONCLUSIONS

The conclusion argues that xenophobic bias in AI remains insufficiently recognized and assessed, and surveys how it can appear across prominent technology domains. It calls for interdisciplinary, participatory development of xenophilic systems as one part of broader solutions.

5 CONCLUSIONS

  • Xenophobic bias remains formally unrecognized and incomprehensively assessed in AI development, risking amplified harms and fear that may contribute to violence and conflict.Source: §5 · p. 14
  • Across social media, immigration, healthcare, employment, and foundation models, historical and contemporary xenophobic attitudes may manifest as material, representational, and societal harms.These harms can include barriers to exercising civic and human rights.Source: §5 · p. 14
  • The authors advocate xenophilic AI that promotes civic inclusion and cross-cultural discourse, developed through coordinated interdisciplinary and participatory action as part of wider societal solutions.Source: §5 · p. 14

Related papers

Loading 2212.07877v2…