Source-linked AI summary
Re-imagining Algorithmic Fairness in India and Beyond
Nithya Sambasivan, Erin Arnesen, Ben Hutchinson, Tulsee Doshi, Vinodkumar Prabhakaran
TL;DR
The paper addresses the gap left by Western-centred fairness frameworks when applied to India’s distinct social, infrastructural, and historical context. It combines 36 qualitative interviews with discourse analysis of Indian AI deployments and policies, using feminist, decolonial, and anti-caste lenses. It finds that assumptions about data, ML makers, and AI adoption are challenged, and proposes an end-to-end fairness agenda centred on context, oppressed communities, and ecosystems.
Problem
Algorithmic fairness research is centred on Western concerns, data, measurements, legal tenets, and values, while substantial fairness research and policy for India’s large AI-interfacing population remain limited.
Method
The paper analyses 36 qualitative interviews and Indian AI deployments and policies through feminist, decolonial, and anti-caste lenses.
Results
The study finds that Indian socio-economic conditions challenge dataset reliability, ML products exhibit double standards, and AI is adopted with unquestioning aspiration.
Takeaways & Limitations
The paper calls for re-contextualising data and model fairness, empowering oppressed communities through participatory action, and enabling fairness ecosystems.
Takeaways & Limitations
Western fairness approaches are often nontrivial to operationalise in India because large Indian name datasets are unavailable and disparities can manifest differently.
Abstract
from arXiv · showhide
Conventional algorithmic fairness is West-centric, as seen in its sub-groups, values, and methods. In this paper, we de-center algorithmic fairness and analyse AI power in India. Based on 36 qualitative interviews and a discourse analysis of algorithmic deployments in India, we find that several assumptions of algorithmic fairness are challenged. We find that in India, data is not always reliable due to socio-economic factors, ML makers appear to follow double standards, and AI evokes unquestioning aspiration. We contend that localising model fairness alone can be window dressing in India, where the distance between models and oppressed communities is large. Instead, we re-imagine algorithmic fairness in India and provide a roadmap to re-contextualise data and models, empower oppressed communities, and enable Fair-ML ecosystems.
1 INTRODUCTION
The paper argues that Western-centred algorithmic fairness rests on assumptions that may not hold in India, where AI power operates across substantial distances from oppressed communities. Drawing on interviews and deployment analysis, it identifies distortions, double standards, and unquestioning aspiration, then proposes a holistic Fair-ML agenda.
- Motivation: Western-centred fairness assumptions about injustice, data, measurements, law, and values may not reflect local contexts such as India.The paper argues that generalising Western fairness can become tokenistic or harmful without engaging non-Western conditions, values, politics, and histories.
- Motivation: India’s prolific high-stakes AI deployments lack substantial policy and research advancing algorithmic fairness for its large population.The paper situates this gap within India’s pluralistic society, vibrant AI workforce, and strong expectations that AI can produce socio-economic benefits.
- Approach: 36 interviews and observations of Indian AI deployments examine algorithmic power through feminist, decolonial, and anti-caste lenses.The analysis focuses on who builds ML for whom, on-the-ground experiences, unfairness processes, exclusions, and their relationship to social justice.
- Findings: Indian infrastructures and social contracts challenge the assumption that datasets faithfully represent people, while models overfit digitally rich profiles and exclude the 50% without Internet access.Caste, gender, and religion require context-specific fairness implementations, and Indic social justice concepts such as reservations introduce additional evaluations.
- Findings: ML makers can apply double standards to Indian users, treating them as extractable data subjects with intrusive models and poor recourse that limits agency.The paper links this distance to the underrepresentation of marginalities among engineers, despite Indians’ participation in the AI workforce.
- Findings: AI’s aspirational status in Indian government, media, and legislation encourages early high-stakes adoption, while weak scrutiny ecosystems inhibit meaningful fairness.The missing ecosystem includes tools, policies, journalists, researchers, and activists able to interrogate deployments.
- Contribution: The proposed roadmap re-contextualises data and model fairness, empowers oppressed communities through participatory action, and enables ecosystems for meaningful fairness.The authors caution that localising model outputs alone can backfire when technical, social, ethical, temporal, and physical distances are large.
2 BACKGROUND
The background literature is dominated by Western institutions, categories, legal concepts, values, and infrastructures. Prior work nonetheless calls for culturally situated justice, global accountability, participation, and India-specific examination of algorithmic harms.
- Western framing: Most fairness and accountability research is framed by Western researchers, Western injustices, Western data, and Western values.Only a handful of FAccT papers from 2019–2020 mention non-Western countries, and one substantially addressed a non-Western context.
- Fairness axes: Fairness research commonly centres racial and gender biases, while categories and proxies such as the Fitzpatrick scale are culturally and historically situated.This focus reflects dominant American public discourse and can omit other dimensions of discrimination.
- Legal foundations: Algorithmic fairness often relies on US laws and legal concepts including disparate impact, disparate treatment, and equal opportunity.These concepts originated in legal and policy contexts involving US law enforcement, housing, loans, and education.
- Values: Fairness scholarship also draws on Western philosophical traditions, whereas Ambedkar’s anti-caste movement represents a distinct social-justice grounding.Power distance and temporal orientation can mediate how cultures value fairness.
- Global perspectives: Social-justice research remains concentrated in the US and Western Europe despite longstanding justice traditions in both Western and Eastern contexts.The literature argues for lenses beyond Euro-American cultural confines.
- Global perspectives: Global AI-accountability principles may be interpreted and implemented differently across regions, motivating incorporation of traditions such as Buddhist, Ubuntu, and Shinto ethics.Researchers have also challenged the normalisation of Western assumptions in specific geographic contexts, including India.
- Accountability: Accountability proposals include participatory design and problem formulation, but cross-cultural systems complicate responsibility through more numerous and remote actors.The literature identifies barriers including many hands, bugs, blaming the computer, and ownership without liability.
- India-specific work: India-specific scholarship documents intrusive Aadhaar data collection and algorithmic biases in policing, hiring, callbacks, wages, and agricultural loans.This work also proposes opportunities for AI and policy-deliberation frameworks attuned to India’s landscape.
3 METHOD
The study combines qualitative expert interviews with discourse analysis of Indian AI deployments and policies. It uses inductive coding and feminist, decolonial, and anti-caste lenses to build a contextual account of algorithmic power.
- Research design: The research synthesises expert interviews and discourse analysis to examine who builds ML for whom, lived experiences, exclusion processes, and social justice.This design aims to provide an expansive account of algorithmic power rather than a narrow model evaluation.
- Interviews: Researchers interviewed 36 experts, including researchers, activists, and lawyers working closely with marginalised Indian communities at the grassroots.Purposeful selection across areas was intended to support a holistic analysis of early public-sector algorithmic deployments.
- Discourse analysis: The team analysed Indian news, policy documents, community media, and prior research on algorithmic deployments and emerging policies beginning with Aadhaar in 2009.Because the analysis used secondary sources, the paper notes that its citations are relatively numerous.
- Interviews: Interviews were conducted in English using purposeful sampling, with recruitment continuing iteratively until saturation.The semi-structured protocol covered discrimination, technology production and consumption, fairness history, biases, data, law, policy, and Indian applications.
- Analysis: Transcripts were inductively coded into seven top-level categories covering discrimination, data and models, law and policy, harms, applications, ML makers, and solutions.The codebook was developed from recurring themes and clustered excerpts.
- Analytical lenses: The analysis applies feminist, decolonial, and anti-caste lenses to locate power in caste, class, gender, religion, and colonial inequalities.The South Asian feminist stance treats oppressed communities as encountering and subverting power, while the anti-caste lens links caste hierarchies and patriarchy.
- Researcher positionality: The authors disclose that all hold privileged class and/or caste positions, while three are Indian and two are White.They describe these positions as shaping their interpretations of research ethics.
4 FINDINGS
The findings show that Indian AI fairness is undermined by unreliable and socially shaped data, models fitted to privileged profiles, and fairness approaches that may not reflect local justice. They also identify weak recourse and a highly aspirational AI environment as conditions limiting meaningful fairness.
- Data considerations: Datasets in India often omit or misrepresent communities because digital divides and social inequalities shape who becomes observable.Half the population lacks Internet access, disproportionately affecting women, rural communities, and Adivasis.
- Data considerations: Safety applications can reproduce social inequality when unevenly visible incidents become area-wide scores that mark Dalit, Muslim, and slum areas unsafe.The resulting data may reflect which incidents receive visibility rather than the distribution of danger itself.
- Data considerations: User practices motivated by privacy, abuse, reputation, or practical needs create off-data traces and inaccurate contextual inferences.Examples include confusing algorithms, cancelling app rides for cash, and calling to clarify landmarks.
- Data considerations: Household power, device sharing, multiple SIM cards, changing numbers, and migrant mobility make identities, locations, and responses unstable.Men may answer surveys on behalf of women, while shared devices and mobile practices disrupt one-to-one user correspondence.
- Model considerations: Indian models overfit digitally rich profiles, typically middle-class men, while caste, gender, and religion require context-specific fairness implementations.Lending apps may define good credit through SMS, calls, contacts, games, and short repayment assumptions, while women sometimes borrow male relatives’ names to avoid perceived bias.
- Model considerations: Reservations offer an Indian justice framework for algorithmic fairness by allocating quotas to historically marginalised groups.Depending on policy, quotas can range from 30% to 80% and extend beyond Dalits and Adivasis to other disadvantaged groups.
- Double standards by ML makers: Indian users are treated as low-agency data subjects for intrusive or low-quality systems, with poor or culturally insensitive recourse, especially for marginalised people.Feedback mechanisms may not be localised, and incidents may receive recognition only after activist intervention.
5 TOWARDS AI FAIRNESS IN INDIA
Algorithmic fairness in India must go beyond model-level fairness because missing local factors, ML disjuncture, and unquestioning AI aspiration can reinforce injustice. The paper calls for rethinking end-to-end algorithmic power and empowering oppressed communities within India’s plural, complex infrastructures.
- Fair-ML in India must go beyond model fairness and address how AI systems are conceived, produced, used, assessed, and contested.
- Missing Indic factors and values in data and models can combine with ML disjuncture and unquestioning AI aspiration to reinforce harms.
- The proposed strategy substantively empowers oppressed communities and enables justice within surrounding socio-political infrastructures.
5.1 Recontextualising Data and Models
Recontextualising fairness for India requires questioning dataset reliability, model categories, and imported evaluation methods. The paper emphasizes community-informed data practices and context-specific models that account for India’s social, infrastructural, and institutional conditions.
- Fairness evaluation and mitigation must be recontextualised because social, economic, and infrastructural factors challenge reliance on Indian datasets.
- Data considerations: Indian datasets require heightened scepticism because completeness and representation are constrained until records become trustworthy.
- Data considerations: Community relationships can improve data quality while treating records as products of both observers and observed communities.
- Data considerations: Fairness research must question whether contextual categories, affects, taboos, and behaviours should be encoded and how their measurement is justified.
- Model and model (un)fairness considerations: Western fairness approaches may not transfer directly because Indian names lack comparable large evaluation datasets and disparities can follow different patterns.
- Model and model (un)fairness considerations: Model interventions must be examined alongside Indian decision infrastructures, including reservation systems and complex seat-allocation processes.
5.2 Empowering Communities
Fair-ML cannot be meaningfully recontextualised without participatory community power in defining problems, fairness expectations, and system designs. This requires grassroots knowledge production, accessible deployments, and scrutiny of extraction and recourse.
- Communities should participate in identifying problems, specifying fairness expectations, and designing systems, because recontextualisation alone has limited reach.
- Participatory Fair-ML knowledge systems: Marginalised communities should produce knowledge about themselves in Fair-ML policies and designs rather than remain outside epistemic production.
- Fair-ML commitments must extend beyond model outputs to accessible deployments that account for connectivity, languages, devices, interfaces, and costs.
- Researchers should examine embedded systems, Indian realities, user feedback, and whether recourse is meaningful amid unequal safeguards and extraction risks.
5.3 Enabling Fair-ML Ecosystems
A sustainable Fair-ML ecosystem in India requires critical engagement with AI’s aspirational role and accountability across civil society, media, industry, the judiciary, and the state. Radical transparency can support a more pragmatic and evolving fairness practice.
- AI’s aspirational and consequential role in non-Western nations makes a critically conscious Fair-ML ecosystem crucial for sustainable impact.
- Ecosystems for accountability: Accountability requires collaboration among civil society, media, industry, judiciary, and the state through partnerships, evidence-based policy, and policymaker education.
- Ecosystems for accountability: Advocacy groups, observatories, and technology journalism can help catalogue harms, demand accountability, and support equitable automation.
- Transparency about datasets, processes, models, limitations, and failures can replace checklist-style fairness with a pragmatic, evolving scientific practice.
6 CONCLUSION
The conclusion argues that algorithmic fairness must be contextualised rather than copied from Western norms. In India, this requires re-contextualising data and models, empowering oppressed communities, and building fairness ecosystems.
- Context matters: Western-normative fairness assumptions are challenged by India’s socio-economic and AI landscape.
- Data is not always reliable because of socio-economic factors, while ML products for Indian users suffer from double standards.
- AI is viewed with unquestioning aspiration, complicating efforts to scrutinise its deployments.
- The paper calls for end-to-end fairness approaches that re-contextualise data and models, empower oppressed communities, and enable Fair-ML ecosystems.
- The authors argue that these considerations extend beyond India and support inclusively evolving global Fair-ML approaches.