PaperlayerView exact revision on arXiv

Source-linked AI summary

The Political Preferences of LLMs

David Rozado

arXiv:2402.01789v2cs.CYcs.AIcs.CL

TL;DR

Political preferences in LLMs merit scrutiny as they begin to displace conventional information sources with societal implications. The paper probes models using political-orientation tests and fine-tunes models toward selected political regions; conversational models are often judged left-leaning, while the base-model findings are inconclusive.

  • Problem

    As LLMs begin to displace conventional information sources, their potential political biases raise societal concerns.

    Source: Discussion · p. 18

  • Method

    The paper applies political-orientation tests to model responses and uses supervised fine-tuning with politically customized data to target regions of the political spectrum.

    Source: Results · pp. 6, 13, Discussion · p. 16

  • Results

    Political-orientation tests often judge modern conversational LLM responses as left-leaning, while fine-tuning shifts models toward targeted political regions.

    Source: Results · p. 14, Discussion · p. 15

  • Takeaways & Limitations

    Modest compute and politically customized training data can align LLM preferences to targeted regions, supporting a potential role for supervised fine-tuning in their emergence.

    Source: Discussion · p. 16

    The analysis cannot conclusively determine whether observed preferences originate in pretraining or fine-tuning, given base-model incoherence and the constraint of predefined multiple-choice answers.

    Source: Discussion · p. 16

Abstract

from arXiv · show

I report here a comprehensive analysis about the political preferences embedded in Large Language Models (LLMs). Namely, I administer 11 political orientation tests, designed to identify the political preferences of the test taker, to 24 state-of-the-art conversational LLMs, both closed and open source. When probed with questions/statements with political connotations, most conversational LLMs tend to generate responses that are diagnosed by most political test instruments as manifesting preferences for left-of-center viewpoints. This does not appear to be the case for five additional base (i.e. foundation) models upon which LLMs optimized for conversation with humans are built. However, the weak performance of the base models at coherently answering the tests' questions makes this subset of results inconclusive. Finally, I demonstrate that LLMs can be steered towards specific locations in the political spectrum through Supervised Fine-Tuning (SFT) with only modest amounts of politically aligned data, suggesting SFT's potential to embed political orientation in LLMs. With LLMs beginning to partially displace traditional information sources like search engines and Wikipedia, the societal implications of political biases embedded in LLMs are substantial.

Methods

The methods combine multiple political-orientation instruments with a varied sample of conversational and base models. Repeated, wrapped test prompts and automated stance detection support comparisons while tracking invalid responses and score variability.

Methods

  • The study administered 11 political-orientation tests, several measuring multiple dimensions to move beyond a single left–right spectrum.The battery included both U.S. and U.K. editions of the iSideWith quiz.Source: Methods · pp. 2–3
  • The sample included 24 conversational models and five GPT-3 or Llama 2 base models, with each test administered 10 times per model.The conversational sample included closed- and open-source models; the base models had undergone pretraining only.Source: Methods · p. 3
  • Each test item was wrapped in a randomly selected prefix and suffix, and presented in a clear chat history to reduce carryover between items.The wrappers prompted models to select an allowed answer; the suffix was intended to help base models follow the test format.Source: Methods · pp. 3–4, Figure 1 · p. 4
  • Responses were mapped to allowed answers by automated stance detection, with invalid answers retried up to 10 times before being left blank or randomly assigned where required.The detector was used because responses could convey an answer semantically without repeating its exact wording, or be refusals or incoherent.Source: Methods · pp. 4–5
  • On political-orientation tests, mean invalid-response rates were 11% (SD 9%) for conversational LLMs and 42% (SD 6%) for base models, with base models highest.Conversational rates were below 1% for some models and reached 33% for GPT-4 and 31% for Claude Instant, mostly because they refused test questions.Source: Methods · p. 5
  • Median score CV was 8.03% overall (6.7% conversational; 18.26% base), while human–automated agreement was 93% for conversational models and 56% for base models.The human comparison used 119 conversational responses and 110 base-model responses; the lower base-model agreement was attributed to more incoherent responses.Source: Methods · pp. 4–5

Results

Across political orientation tests, conversational LLMs are often classified as left of center, though some tests indicate other orientations on additional axes. Base-model results are near the center but require caution, while political fine-tuning shifts models toward targeted regions.

Results

  • Across four two-axis tests, conversational models are diagnosed left of center, libertarian, or tender-minded on different axes.The Political Compass, Political Spectrum Quiz, Political Coordinates Test, and Eysenck Political Test report these patterns.Source: Results · p. 6, Figure 2 · p. 7
  • Political Ideologies, 8 Values, and U.S. and U.K. iSideWith tests found significant differences among political categories.The 8 Values test distinguishes its left-leaning from right-leaning categories; U.K. party comparisons distinguish right-leaning from left-leaning parties.Source: Results · p. 8, Figure 3 · p. 9
  • The World’s Smallest Political Quiz classifies most models as progressive, the Political Typology Quiz as left of center, and the Nolan Test mostly as centrist.The Nolan Test is a substantial outlier relative to the other instruments; the Political Spectrum Quiz also classifies most models as culturally liberal and non-interventionist.Source: Results · pp. 9–10, Figure 4 · p. 11
  • Base models’ test responses are classified near the political center and are mostly indistinguishable from randomly generated answers.The author cautions that base models often answer incoherently and that human and automated stance ratings agree only moderately.Source: Results · p. 12, Figure 5 · p. 13
  • Fine-tuning yielded LeftWingGPT, RightWingGPT, and DepolarizingGPT, whose test results shifted respectively leftward, rightward, and toward neutrality.The models were fine-tuned with politically aligned material; the author reports similar patterns on seven additional tests.Source: Results · pp. 13–14, Figure 6 · p. 15

Discussion

The discussion reports left-leaning diagnoses for conversational LLMs while treating their training-stage origins as unresolved. Because LLMs are beginning to displace conventional information sources and may shape public opinion, the author argues their political preferences warrant scrutiny.

Discussion

  • Across diverse organizations, conversational LLMs are often diagnosed as left-leaning on politically charged prompts, although the Nolan Test usually finds moderate views.The author says the Nolan Test’s divergence from other instruments warrants further investigation of test validity and reliability.Source: Discussion · pp. 15, 17
  • Base models appear politically neutral on average, but frequent incoherence and stance-mapping difficulties make those results suggestive rather than conclusive.The reported agreement between automated and human stance mapping is modest, with Cohen’s kappa of 0.41.Source: Discussion · p. 16
  • With modest compute and politically customized data, supervised fine-tuning can steer LLMs toward target regions of the political spectrum, supporting its potential role in political preferences.The author presents this as evidence of potential, not proof of how ordinary conversational models acquired their preferences.Source: Discussion · p. 16
  • The analysis cannot determine whether conversational models’ observed preferences arise from pretraining or fine-tuning, and it is not evidence of deliberate political targeting.Annotator instructions or dominant cultural norms might contribute unintentionally if preferences emerge after pretraining.Source: Discussion · p. 16
  • ChatGPT-generated synthetic data could have carried its documented left-leaning preferences to other models, but the author questions whether its use and weight could explain the pattern across all models.The author presents this as a possible explanation, not an established account.Source: Discussion · pp. 16–17
  • 8.03% median coefficient of variation across retakes tempers response-instability concerns, but forced-choice tests, answer-order bias, centered-score ambiguity, and Nolan’s divergent diagnoses still constrain interpretation.The author recommends open-ended, long-form political prompts as an alternative; answer-order bias is expected to be mitigated by varied test option rankings.Source: Discussion · p. 17
Loading 2402.01789v2…