We're Cooked! - Probing LLM Political Alignment Via Conflict-Framed Recipe Translation

1 Introduction

• Terms such as “enemy” and “aggressor” carry clear connotations for people experiencing conflict, while food and its origins can become proxy battlegrounds over cultural ownership.Disputes over belonging and authenticity may span centuries.Source: §1 · p. 1
• LLMs may express political positioning directly or indirectly through cultural connotations in their outputs, including on ostensibly neutral tasks such as translation.Prior work has examined direct political questioning, but implicit positioning in apolitical tasks, especially across model families, remains largely unexplored.Source: §1 · pp. 1–2
• The study probes whether attributing an unspecified translation target to an “enemy” language reveals implicit political assumptions across languages and model families.Recipes serve as a culturally loaded but ostensibly apolitical probe, and the authors expect behavior to vary with instruction language and framing term.Source: §1 · p. 2

2.1 Geopolitical and Cultural Bias In LLMs

• Prior work reports cultural misrepresentation and language-dependent ideological outputs in LLMs, including conservative or liberal shifts across languages.Such outputs may shape public opinion and reinforce dominant narratives rather than reflect diverse perspectives.Source: §2.1 · p. 2
• Geopolitical bias has been reported to originate significantly in post-training and to be amplified by prompt language (Bladon and Bent, 2026).Source: §2.1 · p. 2

2.2 Language-Dependent Political Bias

• Prior studies find politically non-neutral LLM translation that shifts with language and framing, alongside language-dependent bias in disputed retrieval and cross-lingual stance adaptation to user cues (Smirnov, 2026).Source: §2.2 · p. 2
• The study addresses that scope gap by testing whether terms such as “aggressor,” “enemy,” or “coloniser” trigger implicit alignment in an apolitical translation task.It anticipates different sensitivity profiles because these near-synonyms carry different connotations across conflicts.Source: §2.2 · p. 2

3.1 Task and Design

• Models translate English recipes into an unnamed target language, using origin-matched dishes and culturally neutral pancakes to isolate framing effects.Prompts may include recipe-origin context; four target-language framing terms are varied, with “neighbour” as the politically neutral control.Source: §3.1 · p. 3

3.2 Models

• The study tests eight US, EU, and Chinese models across 196 combinations and 10 repeats per combination, for 15,680 total responses.Request and context sentences were translated using Google Translate and validated by native speakers for several languages.Source: §3.2 · p. 3

3.3 Annotation

• Responses are classified into six outcomes, with additional annotations for translation language, explicit reasoning, and unprompted political or moral commentary.Explicit language-choice justifications are categorized as historical/conflict, linguistic, official status, vague, or other.Source: §3.3 · p. 3

4.1 Recipe translation

• Claude Sonnet 4.5, GPT-5, and Mistral Large show sparser translation networks, while the other models produce denser networks and more translation directions.Source: §4.1 · p. 3
• Every model agrees on several language routes, and Russian is the dominant sink, receiving up to nine distinct source languages.Source: §4.1 · pp. 3, 5
• Claude has 11 distinct edges and GPT-5 has 25; Mistral is sparse, frame-skewed toward aggressor/enemy, and essentially drops its graph under coloniser.Source: §4.1 · p. 5

4.2 Recipe variant

• Across both recipe variants and almost every model-frame pair, Russian is the most frequent non-English target, and no model defaults to its lab-origin language.Source: §4.2 · p. 5
• The recipe variant changes concentration more than target identity: cultural dishes spread target distributions thinner than neutral pancakes.Source: §4.2 · p. 5

4.3 Reasoning analysis

• Qwen shows the least explicit reasoning, while Kimi, Mistral, DeepSeek, and GLM show the most, dominated by conflict reasoning; GPT-5 and Claude give vague responses.Source: §4.3 · p. 5

4.4 Graph distance

• The Chinese models form the tightest cluster; GPT-5 switches like the Chinese cluster, Mistral is most distant, and neighbour versus coloniser framing pulls distances down versus separates models.Source: §4.4 · p. 5

• The study probes implicit inference with a charged opponent-language term in open-ended recipe translation, rather than direct political questioning or explicit ideological framing.The authors describe the task as more ecologically valid than forced-choice tests and as mitigating sycophancy.Source: §5 · p. 5
• Models vary in translation and reasoning behavior, often defaulting to Russian and resolving requests through active or historical conflicts, though some connections lack historical explanation.Source: §5 · p. 5
• The interpretation of English output as political alignment remains uncertain because English-training-corpus dominance cannot be ruled out.Source: §5 · p. 6
• Even subtle changes among framing terms appear sufficient to modulate behavior consistently across models.Source: §5 · p. 6
• Rejections are rare; GPT-5 and Claude request clarification most often, more often surfacing ambiguity than silently resolving it.Source: §5 · p. 6
• In conflict-adjacent deployment, prompt language and framing may yield qualitatively different implicit judgments without signaling users.Source: §5 · pp. 6–7

Limitations

• Several prompt languages were translated by LLMs, so their linguistic authenticity and cultural accuracy cannot be fully verified.Native speakers reviewed several languages, but not the listed language prompts.Source: Limitations · p. 7
• Author-selected dishes may carry unintended cultural connotations, while language choices represent broad linguistic regions rather than single polities.A different dish might shift a model’s inferred target language.Source: Limitations · p. 7
• The single-turn design may not reflect multi-turn interactions in which political context accumulates.Source: Limitations · p. 8
• LLM-as-judge evaluation may introduce circularity if the judge shares architectural or training similarities with evaluated models, potentially inflating or suppressing patterns.Source: Limitations · p. 8
• The exploratory study reports descriptive patterns without inferential statistics; language choices do not establish political intent, and possible contributing factors were not examined.The authors cite training-data distributions and safety fine-tuning among factors not examined.Source: Limitations · p. 8
• The study cannot isolate the contributions of model size, architecture, or training-data composition.Source: Limitations · p. 8

A Experimental scenarios

• Figure 13 presents the distribution of response outcomes.Source: Figure 13 · p. 10