PaperlayerView exact revision on arXiv

Source-linked AI summary

RupeeBias: Auditing Demographic Bias in Indian Economic Guidance from Large Language Models

Pavithra P M Nair, Bhavik Talaviya, Shourya Bhushan, Rahul Pankajakshan, Seema Guruvadoo, Avinash Agarwal, Gilad Gressel, Krishnashree Achuthan

arXiv:2609.31245v1cs.CL

TL;DR

Existing LLM bias benchmarks largely miss India-specific demographic axes, and prior evaluations had not examined economic disparities under English and Hinglish prompting. RupeeBias audits nine LLMs using 39,150 controlled prompts across four economic use cases and 87 identifiers; outputs differ by 20.2% on average between otherwise identical prompts. The benchmark provides a targeted resource for studying and reducing demographic bias in Indian economic guidance.

  • Problem

    Existing LLM bias benchmarks largely miss India-specific economic disparity axes, and existing evaluations had not tested allocative bias under Hinglish prompting.

    Source: §1 · p. 2

  • Method

    RupeeBias uses 39,150 single-attribute counterfactual prompts across four economic use cases and 87 demographic identifiers in English and Hinglish to audit nine LLMs.

    Source: §1 · p. 2, §7 · p. 10

  • Results

    20.2% average output difference separates otherwise identical prompts that differ only in demographic identifier, with substantial shifts across the covered economic guidance tasks.

    Source: §7 · p. 10

  • Takeaways & Limitations

    RupeeBias provides a targeted resource for studying and reducing demographic bias in LLM-generated economic guidance across Indian demographic and economic contexts.

    Source: §7 · p. 10

    The benchmark covers four Indian technology-sector use cases with synthetic single-turn prompts and explicit single-attribute identifiers, not implicit or intersectional identities.

    Source: §7 · p. 9

Abstract

from arXiv · show

Individuals turn to large language models (LLMs) for guidance across a wide range of economic tasks, from comparing loan options and planning savings to deciding what raise to ask for or how much to charge for their services. LLMs are known to reproduce social biases, and biased economic guidance may influence what users believe they are worth, what they ask for, and what they ultimately accept. This risk is especially salient in India, where economic outcomes are shaped by demographic categories such as caste and urban-rural location. Existing LLM bias benchmarks, however, are largely designed around Western demographic categories and therefore miss key axes of economic disparity in the Indian context. We introduce RupeeBias, a benchmark for auditing demographic bias in LLM-generated economic guidance across Indian economic settings. RupeeBias consists of 39,150 prompts spanning four use cases: salary estimation, salary increment estimation, counter-offer recommendation, and service pricing recommendation. The benchmark follows a single-attribute counterfactual design, holding the description of the user's qualifications, experience, or service offering fixed while varying one demographic identifier at a time. RupeeBias covers 87 India-specific demographic identifiers across six axes: caste, religion, regional identity, gender, disability, and urban-rural location, with all prompts constructed in both English and Hinglish. We evaluate nine LLMs on RupeeBias and find systematic demographic disparities across all six axes. For otherwise identical prompts that differ only in demographic identifier, LLM-generated economic outputs differ by 20.2% on average. We publicly release RupeeBias to support future research on demographic bias in LLM-generated economic guidance across India-specific demographic and economic contexts.

1 Introduction

LLMs provide specific economic guidance that can shape users’ beliefs and decisions, making demographic differences in such outputs consequential. RupeeBias addresses this concern with an India-focused benchmark spanning economic use cases, demographic axes, and English and Hinglish prompts.

1 Introduction

  • Specific LLM-generated economic numbers may shape what users believe, ask for, or accept, so demographic shifts in those outputs may translate into economic disadvantage.The paper highlights salary expectations, appraisal increments, counter-offers, and service prices as consequential recommendations.Source: §1 · pp. 1–2
  • The benchmark addresses an India-specific evaluation gap: existing fairness benchmarks miss salient axes such as caste, regional identity, and urban-rural location, and prior evaluations had not examined economic disparities under English and Hinglish prompting.The paper also identifies Hinglish as a common register of Indian digital communication.Source: §1 · p. 2
  • RupeeBias introduces a benchmark for auditing demographic bias in Indian economic guidance across four use cases and 87 identifiers spanning six demographic axes.The use cases are salary estimation, salary increment estimation, counter-offer recommendation, and service pricing recommendation; the axes are caste, religion, regional identity, gender, disability, and urban-rural location.Source: §1 · p. 2
  • RupeeBias holds qualifications, achievements, and service descriptions constant while varying one demographic identifier at a time.All prompts are constructed in both English and Hinglish.Source: §1 · p. 2

2 Related Work

Prior LLM fairness work has emphasized representational stereotypes, while a smaller body of research examines allocative outcomes such as hiring and salary advice. The paper identifies gaps in the Indian axes and prompting languages covered by allocative-bias evaluations.

Representational Bias in LLMs; Allocative Bias in Economic and Advisory Settings

  • Representational benchmarks test stereotypical associations, while allocative-bias studies examine outputs such as occupations, shortlisted candidates, salaries, and opening offers.Indian representational benchmarks include IndiBias, INDIC-BIAS, Indian-BhED, and DECASTE; prior Indian allocative-bias work has also examined caste-linked resume screening and occupational assignment.Source: §2 · p. 3

Allocative Bias in Economic and Advisory Settings

  • Existing Indian allocative-bias work focuses mainly on caste, leaving religion, regional identity, disability, gender, and urban-rural location unexamined.The passage also notes that some attempted debiasing interventions introduced harms in caste-linked screening and occupational assignment.Source: §2 · p. 3
  • Existing studies had not evaluated allocative bias under Hinglish prompting, despite Hinglish’s prominence in online communication among Hindi-speaking Indian users.RupeeBias responds by auditing four economic use cases across 87 India-specific demographic identifiers in English and Hinglish.Source: §2 · p. 3

3 RupeeBias Dataset

RupeeBias is a controlled dataset of prompts for Indian technology workers and graduates, built by varying demographic identifiers while holding other scenario details fixed. It spans distinct compensation and freelance-pricing scenarios and uses structured prompts in English and Hinglish.

3 RupeeBias Dataset

  • RupeeBias comprises 39,150 prompts simulating Indian technology workers or fresh graduates, with non-demographic details held constant as one demographic identifier changes.The prompt-construction pipeline combines a language, demographic identifier, use case, base profile, and question variant.Source: §3 · p. 3, Figure 1 · p. 4

3.1 Use Case Design and Base Profiles

  • The dataset’s use cases cover campus-placement salary estimates, appraisal hikes, counter-offers, and freelance service pricing for technology workers or graduates.Base profiles hold academic credentials, workplace achievements, professional background, or service offerings fixed across demographic variants.Source: §3.1 · pp. 3–4, §3.1.1 · p. 4

3.1.1 Compensation Use Cases

  • Compensation profiles vary institution and CGPA for salary estimates, experience, company, and achievement for appraisal hikes, and experience, company, and offer strength for counter-offers.Counter-offer profiles set offers at 70%, 100%, or 130% of market rate, using Levels.fyi 2025 compensation medians to determine market rates.Source: §3.1.1 · pp. 4–5

3.1.2 Service Pricing Recommendation Use Case

  • Freelance pricing uses separate vendor-identity and customer-identity conditions, with nine profiles crossing three service types and three vendor-rating tiers.The service types are website development, Android app development, and SEO optimization.Source: §3.1.2 · p. 5

3.2 Demographic Identifiers, Prompt Templates, and Languages

  • The benchmark covers 87 identifiers across six axes and uses a fixed identifier-phrase, base-profile, and question-variant template, with prompts in English and Hinglish.Hinglish components were constructed to follow natural code-mixing patterns and independently validated by two bilingual speakers for fluency, naturalness, and semantic equivalence.Source: §3.2 · p. 5

4 Experimental Setup

The evaluation uses nine LLMs spanning frontier, cost-efficient, open-weight, and India-developed models, and measures demographic output gaps, large-gap frequency, and ranking consistency. It compares otherwise matched demographic variants and treats 10% as an interpretable materiality threshold, not a universal fairness boundary.

4.1 Models

  • Nine LLMs spanning frontier, cost-efficient, open-weight, and India-developed models were evaluated, with temperature set to 0 and configurable reasoning modes disabled.Model selection drew on public rankings and usage signals.Source: §4.1 · p. 6

4.2 Evaluation Metrics

  • Mean ∆% averages symmetric percentage gaps across matched counterfactual pairs, while Violation@τ measures the share whose gap exceeds threshold τ.Violation@10% is used as an interpretable materiality threshold, not a universal fairness boundary.Source: §4.2 · p. 6
  • Kendall’s W measures whether the same demographic identifiers are consistently ranked higher or lower across prompts, with significance assessed using Friedman tests and Benjamini–Hochberg FDR correction.Higher W indicates more consistent rankings; lower W indicates more variation across prompts.Source: §4.2 · p. 6

5 Results

Across models, demographic output gaps vary by language, demographic axis, and use case, while ranking consistency also differs substantially. Magnitude and consistency provide distinct views of the results.

5.1 Magnitude of Demographic Output Gaps

  • 8.1 is Gemini 3 Flash’s pooled Mean ∆%, the lowest among evaluated models; Sarvam 105B has the highest pooled Mean ∆% at 37.9.Table 2 reports aggregate Mean ∆% and Violation@10% by model and language.Source: §5.1 · p. 7, Table 2 · p. 7
  • Language shifts gap magnitude: Claude Haiku 4.5 falls from Mean ∆% 14.29 in English to 9.25 in Hinglish, while DeepSeek V3.2 rises from 26.71 to 38.66.Claude Haiku 4.5 has the lowest violation rates, rather than Gemini 3 Flash.Source: §5.1 · p. 7
  • Urban–rural location is the largest-gap axis for most models; the largest-gap use case varies, including counter-offers for DeepSeek V3.2 and salary estimation for Sarvam 105B.Salary increment estimation is the most common high-gap setting among the other models described.Source: §5.1 · p. 7, Figure 2 · p. 7

5.2 Demographic Ordering Consistency

  • All reported Kendall’s W values are significant after global Benjamini–Hochberg FDR correction; English leaders include Claude Opus 4.7 (W = 0.274), Claude Haiku 4.5 (W = 0.273), and Gemini 3 Flash (W = 0.270).In Hinglish, Claude Opus 4.7 leads with W = 0.344; DeepSeek V3.2 and Sarvam 105B have the lowest Mean W in both languages.Source: §5.2 · pp. 7–8, Figure 3 · p. 8
  • Gap magnitude and ordering consistency differ: Claude Opus 4.7 combines moderate Mean ∆% with high W, while DeepSeek V3.2 and Sarvam 105B show larger gaps but lower W.Figure 3 presents ordering consistency and the comparison between Mean ∆% and Mean Kendall’s W.Source: §5.2 · p. 8, Figure 3 · p. 8

6 Analysis and Insights

The analyses show that gap size and the stability of demographic rankings capture different patterns, and that identifier-level results depend on the model. Prompt language also changes both gap magnitude and demographic ordering.

6 Analysis and Insights

  • Open-weight models generally occupy the high-gap, low-consistency region, but Kimi K2.5 resembles proprietary models and GPT 5.4-mini is a proprietary exception.Claude Opus 4.7 has moderate gaps and the highest ordering consistency, while DeepSeek V3.2 and Sarvam 105B have larger, less directionally predictable gaps; the authors argue audits should report both metrics.Source: §6 · p. 8
  • Identifier-level effects are model-specific: Claude Haiku 4.5 shows the widest displayed religious separation, while Kimi K2.5 has the largest gender spread.Haiku favors Jain and disfavors Muslim, Christian, and Sikh; Kimi favors Non-binary and disfavors Male.Source: §6 · p. 8, Figure 4 · p. 9
  • English–Hinglish ranking agreement and gap shifts vary by model and axis: urban/rural rankings are relatively stable, religion agreement is weaker, and Hinglish can amplify or reduce gaps.The reported examples include amplification for DeepSeek V3.2 and reductions for Sarvam 105B and GPT 5.4-mini.Source: §6 · p. 9, Figure 5 · p. 9

7 Limitations and Conclusion

RupeeBias outputs differ by 20.2% on average between otherwise identical prompts that vary by demographic identifier, but the benchmark is a controlled, limited snapshot. The appendices examine how gaps vary by axis and use case and how demographic rankings hold across tasks and question variants.

7 Limitations and Conclusion

  • RupeeBias uses synthetic single-turn prompts in four Indian technology-sector use cases and explicit single-attribute identifiers, and evaluates nine model versions as a time-specific snapshot.It does not model implicit cues or intersectional identities and is not a complete model of real-world economic advice.Source: §7 · p. 9
  • Outputs differ by 20.2% on average between otherwise identical prompts that vary by demographic identifier; the benchmark covers four tech-sector use cases and 87 identifiers across six axes in English and Hinglish.The authors limit its purpose to auditing, mitigation, and accountability, not judging any group's merit or appropriate compensation.Source: §7 · p. 10

A.1.1 Axis-wise Mean∆%

  • Axis-wise Mean ∆% results for English and Hinglish, with 95% confidence intervals, complement the aggregate leaderboard.The appendix presents these results in Tables 3 and 4.Source: §A.1 · p. 14, §A.1.1 · p. 14, Table 3 · p. 15

A.1.2 Use-case-wise Mean ∆%

  • Mean ∆% gaps vary substantially by model and use case in both languages; several models show especially large gaps for counter-offer recommendations and salary-increment estimates.Other use cases show smaller but still non-zero gaps; Figure 6 reports use-case results with 95% confidence intervals.Source: §A.1.2 · p. 14, Figure 6 · p. 16

A.2.1 Axis-wise Kendall’s W Leaderboards

  • Axis-wise Kendall’s W leaderboards combine Friedman-test p-values across use cases using Fisher’s method, then apply Benjamini–Hochberg FDR correction.Tables 5 and 6 show English and Hinglish results; Mean W averages across six axes.Source: §A.2.1 · p. 16, Table 5 · p. 17, Table 6 · p. 17

A.2.2 Use-case-wise Kendall’s W

  • Use-case-wise Kendall’s W plots show how strongly demographic identifier rankings are preserved within each task, with hatching marking values not significant after FDR correction.The plots distinguish task-specific patterns from the axis-wise aggregate leaderboards.Source: §A.2.2 · p. 16

A.3 Demographic Identifier-Level Gap Analysis

The identifier-level analyses complement aggregate gap measures with pre-specified pair comparisons and the largest observed gap among all identifier pairs. They show stable directional disparities for some contrasts and more variable patterns for others.

A.3 Demographic Identifier-Level Gap Analysis

  • Canonical Pair Gap compares pre-specified, socially salient identifier pairs, which are interpretable reference contrasts but need not be the most extreme pairs.The pairs include Hindu–Muslim, Brahmin–Dalit, Gujarati–Santhali, Male–Female, No disability–Intellectual disability, and Tier 1 city–Village.Source: §A.3 · p. 17
  • Disability and urban/rural contrasts show directionally stable gaps in several settings, while counter-offer gaps are larger but less consistent in direction.No-disability prompts often receive higher outputs than prompts mentioning intellectual disability; service-pricing recommendations often favor Tier 1 city over village customers.Source: §A.3 · p. 18
  • Maximum Observed Gap scans all identifier pairs within an axis and reports the largest observed gap for each model and use case.It is unsigned and non-negative, and is at least the corresponding CPG magnitude unless the canonical pair is itself the most extreme.Source: §A.3 · p. 18
  • Urban/rural gaps recur across models and use cases, while other axes show more task-dependent patterns, including smaller salary-increment gaps and larger service-pricing or counter-offer gaps.Figures 8 and 9 respectively encode canonical-pair gaps and maximum observed gaps across models and use cases.Source: Figure 8 · p. 19, Figure 9 · p. 20, §A.3 · p. 21

B Question-Variant Robustness

The paper tests whether demographic rankings persist across alternative phrasings of the same economic question. Rankings are often only moderately stable, with robustness varying by model, language, and axis.

B Question-Variant Robustness

  • For each model, language, axis, and use-case setting, the benchmark computes pairwise Spearman’s ρ across question variants and uses the minimum as a conservative stability measure.A high minimum requires all variants to produce similar demographic orderings; tables report mean minimum ρ and bootstrap confidence intervals.Source: B Question-Variant Robustness · p. 21, Table 7 · p. 21
  • No model reaches the strong-robustness threshold in English; in Hinglish, Gemini 3 Flash reaches it for gender, while DeepSeek V3.2 and Sarvam 105B show near-zero or negative mean robustness in both languages.Disability and urban/rural location are the most stable axes on average.Source: B Question-Variant Robustness · p. 21

C Identifier-Level Deviation Results

Identifier-level deviations show each identifier’s relative standing against its demographic-axis average, rather than absolute output levels. The plots include every identifier for smaller sets and selected identifiers for larger sets.

C Identifier-Level Deviation Results

  • Identifier-level deviations are computed relative to the mean output across identifiers on the same axis; positive and negative values indicate above- and below-axis-average outputs.These relative standings should not be interpreted as absolute output levels.Source: C Identifier-Level Deviation Results · pp. 21–22
  • The plots show all five gender and urban/rural identifiers, six prominent religions, and five selected identifiers each for caste, region, and disability.Figures 10 and 11 show caste/region and disability/urban-rural results; Figure 4 shows religion/gender results.Source: C Identifier-Level Deviation Results · p. 22, Figure 10 · p. 22, Figure 11 · p. 23

D Base-Profile Analysis

The appendix tests whether models respond to profile merit and preserve profile rankings across demographic identifiers. It finds that sensitivity and ranking consistency vary by use case and model.

Merit Sensitivity

  • Merit sensitivity measures output changes across base profiles while holding demographic identity fixed; higher values indicate greater responsiveness to qualifications, experience, or service characteristics.The measure averages relative gaps across profile pairs for each identifier and question-variant slot.Source: D Base-Profile Analysis · p. 23
  • Salary increment estimation has the highest merit sensitivity for most models, while salary estimation is least sensitive on average.Claude Haiku 4.5 and DeepSeek V3.2 show the highest overall sensitivity; GPT 5.4 and Gemini 3 Flash are closer to the middle.Source: D Base-Profile Analysis · p. 23, Figure 12 · p. 24

Profile Rank Agreement

  • Profile rank agreement measures whether demographic identifiers preserve which profiles rank higher, distinguishing ordering changes from output-level shifts.The analysis builds a profile-by-identifier matrix from mean outputs across question variants and computes mean pairwise Spearman’s ρ between identifier rankings.Source: D Base-Profile Analysis · pp. 23–24
  • Counter-offer recommendation shows markedly lower profile rank agreement across many models, while salary and salary-increment rankings are near-perfect for most models.Service pricing also degrades for several models; Gemini 3 Flash is comparatively stable, while Sarvam 105B has especially low agreement in service pricing.Source: D Base-Profile Analysis · p. 24

E Fairness-Instruction Ablation

This appendix tests an explicit fairness instruction in salary estimation and documents the benchmark’s base profiles and validation survey. The instruction usually reduces bias, but its effects vary by model.

E Fairness-Instruction Ablation

  • The ablation compares each model’s standard system prompt with an augmented prompt asking it to ensure its response is fair and unbiased.It measures the change in Mean ∆% for salary estimation in English and Hinglish; negative changes indicate reduced bias, positive changes increased bias.Source: E Fairness-Instruction Ablation · pp. 24–25
  • Fairness instructions usually reduced bias but were model-dependent; Sarvam 105B fell from 46.8 to 34.9 (−11.9 points, 25% relative reduction) in English, while GLM 5.1 backfired in both languages.The Hinglish pattern was broadly similar, with one fewer backfire; Sarvam 105B remained the most biased model overall after the intervention.Source: E Fairness-Instruction Ablation · pp. 25–26, Figure 14 · p. 26

F.1 Salary Estimation

  • Salary estimation uses 18 fresher profiles crossing six institutions with three CGPA levels, spanning elite, mid-ranked, and lower-ranked institutions.The CGPA values are 9.1, 7.2, and 5.8.Source: §F.1 · pp. 26–27, Table 9 · p. 27

F.2 Salary Increment Estimation

  • Salary increment estimation uses 27 profiles crossing three experience levels, three companies, and three workplace-achievement levels.Achievement descriptions and rating terminology are company-specific and reused across experience levels.Source: §F.2 · p. 27, Table 11 · p. 28, Table 10 · p. 28

F.3 Counter-Offer Recommendation

  • Counter-offer profiles cross three experience levels, three companies, and three offer-strength levels; lowball and above-market amounts are set at about 70% and 130% of market rate.Offer amounts are grounded in Levels.fyi 2025 compensation medians.Source: §F.3 · p. 27, Table 12 · p. 29

F.4 Service Pricing Recommendation

  • Service-pricing profiles cross three service types with three vendor-rating tiers, using the same nine profiles for vendor-side and customer-side conditions.The vendor’s identity varies in the vendor-side condition; the customer’s identity varies in the customer-side condition.Source: §F.4 · pp. 27, 29, Table 13 · p. 30

G.4 Results

  • Nine engineers—three each from TCS, Flipkart, and Google India—reviewed workplace-achievement descriptions, and all descriptions were judged acceptable with no negative responses.The review assessed naturalness and rating-label accuracy.Source: §G.2 · p. 30, §G.3 · p. 31

G.4 Results

The appendix reports validation of workplace-achievement descriptions and documents RupeeBias identifiers, prompt construction, translation validation, and output-format checks.

G.4 Results

  • All workplace-achievement descriptions were judged acceptable for naturalness and rating-label accuracy, with no negative responses from survey participants.Two “Mostly yes / Partially” responses concerned naturalness, and one concerned rating-label accuracy.Source: §G.4 · p. 31, Table 14 · p. 31

H Demographic Identifiers

  • RupeeBias’s 87 demographic identifiers span caste, religion, regional identity, gender, disability, and urban-rural location, with English and Hinglish phrases listed for each identifier.The appendix provides the full identifier list and summarizes each axis’s source taxonomy and labor-market motivation.Source: H Demographic Identifiers · pp. 31–32, Table 16 · p. 32

I Prompt Templates and Question Variants

J.1 Construction

  • Hinglish prompts use natural code-mixing and syntax rather than word-for-word translation, and are intended to remain semantically equivalent to English prompts.Proper nouns, salary figures, CGPA values, and other technical terms remain in English.Source: §J.1 · p. 34

J.2 Validation Procedure

  • Two independent native-speaker validators assessed fluency, natural code-mixing, and semantic equivalence; flagged items were revised and rechecked, and all three agreement scores fell in the “almost perfect” range.Initial ratings had only a small number of minor issues and no “Not acceptable” ratings.Source: §J.2 · pp. 35–36

K System Prompts

  • Fixed system prompts require numeric, single-line outputs and prohibit ranges, explanations, extra text, and non-standard numeric units.The appendix lists the exact prompts for each use case.Source: K System Prompts · p. 37, Table 21 · p. 37

L Parse Health and Model Refusal Rate

Loading 2609.31245v1…