AI Alignment for Digital Democracy: Fairer Voters Representation against LLM Biases

AI representation

• LLMs can emulate votes from real voter profiles and aggregate them into a synthetic electorate; individual fidelity and aggregate accuracy are distinct, and aggregate agreement can conceal concentrated group errors.The paper describes applications including polling, estimating preferences of missing voters, synthetic populations, and deliberation.Source: §1 · p. 1

Research questions

• The study asks whether group vote-share errors differ across demographic groups and whether steering individual voters before aggregation reduces misrepresentation without lowering individual accuracy.It evaluates eligible respondents from ANES 2012, 2016, and 2020 across base, prompt-based, and decomposed-prompt control, with activation-level control for the open-source model.Source: §1 · pp. 1–2

Motivation

• Unequal, directional errors can shift a group’s emulated vote share and, when errors do not cancel, the overall share or winner in a close election.Repeated sampling can reduce random variation but cannot correct a mistake the model makes consistently.Source: §1 · p. 2

Contributions

• Removing gender reduces the largest group error by 22–42%, removing education reduces it by 64%, and removing self-reported race increases mean group error by over 40%, without lowering accuracy.Repeated queries, aggregation, fairness instructions, and activation-level control do not reduce misrepresentation; a fairness instruction can double a group’s error.Source: §1 · p. 2

2.1 Generative AI Representation of Voters

• Earlier synthetic-sample research primarily examined correspondence with human response distributions; later work found reduced variation and subgroup distortions despite average-pattern agreement.The paper builds on this work by comparing each group’s emulated vote share with its reported share and testing alignment interventions.Source: §2.1 · p. 2
• Earlier voter-emulation studies report varying predictive accuracy across contexts and overstated demographic differences, while collective-choice work focuses on robustness after aggregation.The paper’s focus is whether systematic demographic errors persist before aggregation and distort group vote shares.Source: §2.1 · p. 2

2.2 Fairness Analysis in LLM Decision Making

• Fairness methods designed for decisions such as hiring often seek similar treatment across groups, whereas voter emulation must reproduce groups’ actual vote shares; equalizing predictions can increase misrepresentation.The paper evaluates interventions before aggregation and at the representative process.Source: §2.2 · pp. 2–3
• Table 1 compares prior studies across six fairness-evaluation dimensions, and none of the studies considered combines all six in one evaluation framework.The dimensions include group performance, formal fairness metrics, race or ethnicity, multiple election cycles, model types, and direct voter emulation.Source: §2.2 · p. 3, Table 1 · p. 3

3 Problem Formulation

• Each respondent’s emulated vote is compared with their reported vote, then group votes are aggregated and compared with that group’s reported results.The workflow may query each voter repeatedly and combine predictions by majority before group aggregation.Source: §3 · p. 3

3.1 Voter Profiles and Voter Emulation

• Voter profiles contain individual information, group attributes, and reported votes; individual fidelity is agreement between an emulated vote and that respondent’s reported vote.Individual accuracy is the share of respondents whose emulated votes match their reported votes.Source: §3.1 · p. 3

3.2 Demographic Grouping

• Respondents are partitioned by a demographic attribute so the analysis can test whether errors are unevenly distributed across groups.Race or ethnicity is the focal grouping, but the framework also applies to attributes such as gender, age, education, religion, or region and to combinations of attributes.Source: §3.2 · p. 3

3.3 LLM Repeated Vote Generation

• Because stochastic decoding can produce different votes for the same profile, the model is queried K times per voter and the majority prediction becomes that voter’s emulated vote.Odd K avoids ties; within-voter run combination is distinct from aggregation across respondents.Source: §3.3 · pp. 3–4

3.4 Group Misrepresentation Metrics

• Group misrepresentation is the gap between a group’s emulated and reported vote shares, with the largest group gap serving as the primary fairness metric.The framework also reports mean group error and population-level error, and uses confidence intervals to identify groups whose signed error excludes zero.Source: §3.4 · p. 4
• The framework evaluates errors in vote-share points against the same respondents’ reported votes, making errors comparable across groups within a dataset.Small groups have noisier estimates, so group sizes and 95% confidence intervals are reported.Source: §3.4 · p. 4

3.5 From Individual to Group Errors

• Majority votes can leave group shares unchanged when a voter’s Democratic-vote probability changes without crossing 0.5, then change the share by 1/N_r when it crosses.Probability aggregation can change shares continuously; majority-vote aggregation has no corresponding linearity guarantee.Source: §3.5 · p. 4

4.3 Models and Levels of Control

• Models receive voter profiles at base and prompt-controlled levels; decomposed control separates individual signals from community representation, and Llama also receives activation-level control.The base setting is sampled once or five times per voter; controlled settings are sampled once.Source: §4 · p. 4, §4.3 · pp. 4–5

4.1 Experimental Design

• Claude Sonnet 4.6 is prompt-controlled, while open-weight Llama-3.1-8B-Instruct also permits internal activation control; comparing control levels and run counts separates their effects.The study compares both individual accuracy and group misrepresentation error.Source: §4.1 · p. 4

4.2 Datasets and Voter Profiles

• The evaluation includes ANES 2012, 2016, and 2020 respondents with two-party votes and complete profiles, grouped by race, gender, age, and education.Groups with fewer than 30 respondents are excluded.Source: §4.2 · p. 4, Table 2 · p. 4

4.4 Evaluation Metrics

• The primary fairness measure is worst-group error, alongside individual accuracy, mean group error, and population-level error; a group is misrepresented when its 95% interval excludes zero.Source: §4.4 · p. 5

5.1 Misrepresentation Across Models

• In ANES 2020, Claude and Llama place Native American Democratic share 13.5 and 13.6 points above reported share, respectively, while both models’ Black-voter errors remain within 1 point.Source: §5.1 · p. 5

5.2 Robustness

• Five-run majority voting does not reduce worst-group racial error: it is statistically indistinguishable between one and five runs for both models in every election.Claude gives the same vote in all five runs for 99.3–99.6% of respondents; Llama does so for 84–90%.Source: §5.2 · p. 6
• Across all three elections, both models understate Democratic choices for male voters and those with at most high-school education; Claude's emulated gender gap is about three times the reported one.Source: §5.2 · p. 6, Figure 4 · p. 6
• Splitting racial or ethnic groups by gender raises Claude's worst-group error, beyond a shuffled-label baseline in 2012 and 2016 but not 2020; for Llama, this occurs only in 2016.Source: §5.2 · p. 6

5.3 Source of Misrepresentation

• Attribute-only prompts exaggerate group gaps severalfold; full profiles bring race amplification close to 1 but leave gender and Claude's education amplification above 1.Claude's attribute-only answers are deterministic by group; Llama's gender-only prompts often receive no answer.Source: Figure 5 · p. 7, §5.3 · p. 7
• Removing gender cuts the largest group error 42% for Claude and 22% for Llama; removing education cuts Claude's 64%, while removing race raises mean error about 40% for both.Accuracy never decreases significantly when gender, education, or race is removed.Source: §5.3 · p. 7, Table 3 · p. 7

• In ANES 2020, both models' emulated Democratic share is about 14 points higher than reported for Native American voters, but within 1 point for Black voters.Source: Answering RQ1 · p. 7
• Across all three elections, both models understate Democratic choices for male voters and voters with at most a high-school education, exaggerating real group differences.Source: Answering RQ1 · p. 7
• Removing gender or education reduces group errors, whereas removing self-reported race increases them, without improving accuracy from including gender or education.Giving the models gender makes the largest gender-group error 29% for Llama and 73% for Claude larger; giving Claude education makes its largest education-group error 2.8 times larger.Source: Answering RQ1 · p. 7

6.1 Effect of Aggregation

• Probability aggregation changes worst-group error by at most 1.5 points for Claude and 1.4 points for Llama, with no consistent direction.Llama’s exaggerated gender gaps also remain about 2.4–2.9 times the reported gap under either aggregation rule.Source: §6.1 · pp. 7–8

6.2 Effect of Control

• Prompt-based control roughly doubles Claude’s worst racial error, while neither control reduces the persistent 8–11-point male gender error.For Native American voters in 2020, the emulated Democratic share rises from 12.4 to 25.8 points above the reported share; the figure shows group errors by control level.Source: §6.2 · p. 8, Figure 6 · p. 8
• For Llama in 2012 and 2016, decomposed control reduces population-level error and worst-group errors by removing an overall Democratic-share shortfall, not by correcting particular groups.Male voters and voters with at most a high school education remain 5–12 points below their reported Democratic shares; in 2020, neither control significantly changes worst-group error.Source: §6.2 · p. 8

6.3 Gaps under Control

• Control does not bring exaggerated demographic gaps close to reported gaps: Claude’s racial amplification rises under prompt-based control, while gender and education gaps remain exaggerated.Claude’s gender amplification falls in some settings, but its emulated gender gap remains 2.6–3.6 times the reported gap; Llama’s gender amplification falls significantly only in 2016.Source: §6.3 · pp. 8–9

6.4 Prompt vs. Activation Control

• Activation-level control raises Llama’s population-level error from 1.4 to 6.6 points and worsens gender and education errors without reducing race error.It shifts choices almost entirely toward the Democratic candidate, raising every group’s share of emulated Democratic choices; accuracy is unchanged by −0.6 points.Source: §6.4 · p. 9

Answering RQ2

• Neither aggregation nor voter-level control reliably reduces misrepresentation; group-level evaluation is needed because individual accuracy can conceal whether a control helps or harms.Prompt-based race targeting leaves the largest gender and education errors in place, while activation-level control shifts the electorate and worsens misrepresentation.Source: Answering RQ2 · p. 9

7 Conclusion

• Removing gender reduces the largest group error by 22–42%, and removing education reduces it by up to 64%, while removing self-reported race increases mean group error by over 40%.Common alignment strategies do not reduce these errors and can double them; future work will test attribute combinations and alignment interventions across other elections, countries, and multi-party systems.Source: §7 · p. 9

A.1 Notation

• The notation defines each surveyed voter by a profile, demographic group, and reported vote, with groups indexed by demographic attribute.The listed attributes are race, gender, age, and education.Source: §A.1 · p. 11, Table 5 · p. 11

A.2 Probability Aggregation Preserves Error Bounds

• If each component emulation’s group error is at most ε, their convex combination also has group error at most ε.This follows because group share is linear in the emulation and a convex combination cannot exceed its largest component error.Source: §A.2 · p. 11
• Probability averaging repeated runs is never worse than the average run, whereas majority voting has no such guarantee and can exaggerate group differences.Thresholding voter probabilities before averaging is nonlinear; the appendix gives an example where majority voting changes a 0.6 group share to 1.Source: §A.2 · p. 11

A.3 Interpretation

• The bound does not ensure that aggregation removes misrepresentation: shared directional errors can leave every component error large.Improving individual representations is not necessarily sufficient under majority voting.Source: §A.3 · p. 11

A.4 Repeated Runs under Independent Noise

• Repeated-run error reduction requires independent errors with per-run error probability below one-half, an idealized condition not met by these models.Claude repeats the same vote for 99.3–99.6% of voters and Llama for 84–90%; one and five runs yield statistically indistinguishable worst-group errors.Source: §A.4 · p. 11

B.1 Data

• The study uses ANES respondents from 2012, 2016, and 2020 with complete demographic and political-profile data; group shares are unweighted and describe surveyed respondents, not the national electorate.Groups with fewer than 30 respondents are excluded from group-level analyses.Source: §B.1 · pp. 11–12

B.2 Models

• The comparison uses Claude Sonnet 4.6 and locally run Llama-3.1-8B-Instruct, with three prompt-control levels and separate run counts.Base emulations use five runs, while controlled emulations use one run so levels are compared on the same footing.Source: §B.2 · p. 12, §B.3 · p. 12

B.4 Activation-Level Control

• Activation-level control adds a scaled steering vector to Llama’s layer-16 hidden state, with the direction derived from White and other-race voter samples.The scale is set to α=1; the value is chosen to keep the overall Democratic share within 10 points of the uncontrolled model on an initial 200-voter sample.Source: §B.4 · p. 12, Table 7 · p. 12

B.5 Experiments

• The experiments compare each attribute alone, the full profile, and the full profile without that attribute.For the attribute-alone condition, each group is prompted 20 times with alternating candidate order; the full and attribute-omitted conditions use five Llama runs and one Claude run.Source: §B.5 · p. 12, Table 8 · p. 12

B.6 Statistical Analysis

• Group-error intervals use paired per-voter differences, and paired bootstrap tests compare conditions on the same voters.The analysis also tests whether intersectional worst-group errors exceed size-driven expectations by shuffling gender within racial groups while holding group sizes and race-level errors fixed.Source: §B.6 · pp. 12–13
• Gap amplification is the weighted least-squares slope of emulated on reported group shares: 1 indicates reproduction, above 1 exaggeration, and below 1 flattening.For two groups, it is the ratio of the emulated to reported gap.Source: §B.6 · p. 13

C Number of Runs

• Compared with one run, five-run majority voting does not significantly change racial worst-group error in any election for either model (paired bootstrap, p≥0.51); gender and education errors stay essentially unchanged.Claude’s accuracy and gender- and education-based worst-group errors change by at most 0.1 points, with runs agreeing for 99.3–99.6% of voters. For Llama, accuracy rises by at most 1.1 points and worst-group error changes by at most 1.5 points, except race in 2020, which changes from 0.169 to 0.136 for the Native American group (n=89); Claude’s race-in-2020 error changes from 0.124 to 0.135 for that group.Source: C Number of Runs · p. 13

D Prompt Templates

• Prompt templates randomize candidate order per voter and keep it fixed across models and control levels.The base template is unchanged for activation-level control.Source: D Prompt Templates · p. 13

E Intersectional Results

• Intersectional splits raise worst-group error from 0.05–0.14 for race alone to 0.10–0.23, with much of the increase reflecting smaller subgroups.Across elections and models, Hispanic male voters and Hispanic voters with at most a high school education have emulated Democratic shares 15–22 points below reported shares.Source: E Intersectional Results · p. 13

F Statistical Tests for the Profile Variants

• Leaving out gender significantly reduces worst-group error for both models, while leaving out race significantly increases mean group error; accuracy never decreases significantly.For Llama, leaving out education also slightly increases accuracy.Source: F Statistical Tests for the Profile… · p. 13

G Discussion and Limitations

• Individual accuracy of 0.80–0.86 coexists with group errors of 10 points or more, and repeated runs or aggregation do not remove the most consistent gender and education errors.Which racial groups are misrepresented varies across elections, limiting assurance from auditing just one model and election.Source: G Discussion and Limitations · pp. 13–14
• The tested controls do not support a claim of reduced misrepresentation: race-targeted prompts leave gender and education errors in place, while activation-level control worsens misrepresentation.The tested controls target race, whereas the largest errors concern gender and education; removing a demographic label can trade one error for another.Source: G Discussion and Limitations · p. 14
• Small groups limit precision: the largest racial error cited rests on 89 Native American voters, and several intersectional subgroups have fewer than 50.The authors do not claim the intersectional results reveal additional misrepresentation beyond what smaller samples produce.Source: G Discussion and Limitations · p. 14
• The steering vector encodes racial identity rather than representation accuracy and shifts the electorate toward the Democratic candidate.At the tested strength the Democratic share rises by 7 points; larger strengths collapse almost all predictions to one party.Source: G Discussion and Limitations · p. 14
• Prompt-based and decomposed control change prompt length, structure, and instruction content together, so their effects cannot be attributed to block structure alone.A same-length, same-content single-block prompt would be needed for that comparison; activation-level control retains the base prompt unchanged.Source: G Discussion and Limitations · pp. 14–15
• Published group vote choices may appear in model training data, so close agreement with published shares may reflect recall rather than conditioning on the voter profile.The source analysis finds responses to the full profile but cannot distinguish profile conditioning from recall of published group statistics.Source: G Discussion and Limitations · pp. 15–16
• The evaluation covers two models, one per access regime, and one survey series; the group shares are unweighted and describe surveyed voters rather than the national electorate.Findings beyond two-party U.S. presidential elections, this survey series, or the six-field profiles remain untested; party identification is excluded.Source: G Discussion and Limitations · p. 16