Mind the Value-Action Gap: Do LLMs Act in Alignment with Their Values?

• Prior LLM work measured stated inclinations (Sorensen et al., 2024), leaving statement-action alignment largely unexamined, a gap framed by value-action theory.Source: §1 · p. 1
• Misaligned LLMs have been associated with stereotype amplification and biased hiring algorithms (Dammu et al., 2024; Wilson and Caliskan, 2024).Source: §1 · p. 1
• ValueActionLens is presented as the first framework for measuring value-action gaps by comparing LLMs’ generated value statements with actions informed by those values.The framework assesses alignment using an alignment rate, alignment distance, and alignment ranking.Source: §1 · pp. 1–2, Figure 2 · p. 2
• The VIA dataset contains 14,784 value-informed actions across 12 cultures and 11 societal topics, and evaluations test stated preferences and contextual action.The framework uses three alignment metrics to quantify the gap between these tasks.Source: §1 · p. 2, Figure 2 · p. 2
• Across six LLMs, stated values and actions showed substantial gaps that varied by value type, culture, and topic, with three models mostly less aligned in African and Asian than North American and European contexts.The three models were GPT-4o-mini, Claude-Sonnet-4, and Llama-3.3-70B.Source: §1 · p. 2
• Qualitative analysis identifies potential harms, including an LLM expressing loyalty but failing to act accordingly in a U.S. religious context.Source: §1 · p. 2

Psychometric Methods for Value Evaluation

• Prior LLM value evaluations expanded from specific values to broader frameworks but typically relied on stated values elicited through Likert scales or agreement levels.The literature includes evaluations grounded in theories and surveys such as Schwartz’s theory (Schwartz, 1994, 2012), the World Value Survey, and the GLOBE framework (Karinshak et al., 2024).Source: §2 · p. 2

Value-Action Consistency Study

• Human value-action gaps are well documented, and emerging LLM studies such as ValueBench (Ren et al., 2024) and GPV (Ye et al., 2025b) examine consistency between self-reports and actions.In humans, cognitive, contextual, and social factors can hinder value-consistent actions, while theories of reasoned action help explain and predict these gaps.Source: §2 · p. 2
• The study addresses the lack of context-aware evaluation and supporting data by analyzing value-action inconsistencies across 132 scenarios and introducing alignment metrics.The VIA dataset contains 14,784 value-informed actions across 12 countries, 11 social topics, and 56 values, with highlighted actions and natural-language explanations.Source: §2 · p. 3, Table 1 · p. 3

3.1 Contextualizing Values into Scenarios

• The framework pairs 12 countries and 11 social topics with 56 values and agree/disagree stances, yielding 132 scenarios and 14,784 value-informed actions with explanations.The values come from Schwartz’s Theory of Basic Values (Schwartz, 1994, 2012), which the authors say can be extended to other value theories.Source: §3.1 · p. 3

3.2 Generate Value-Informed Actions with Explanations

• A human-in-the-loop pipeline builds and selects prompts, then evaluates generated actions and explanations for quality.Two AI experts selected the optimal prompt from eight variants using 640 instances, achieving Cohen’s Kappa = 0.7073; 27 culturally diverse annotators then evaluated 90 sampled instances, with three reviews per instance and majority voting. Explanations contain action attribution and a natural-language rationale; assessments cover action correctness and harmlessness, attribution sufficiency, and explanation plausibility.Source: §3.2 · p. 4, Figure 3 · p. 4

3.3 Two Tasks for Evaluating Stated Values and Value-Informed Actions

• Two tasks elicit LLMs’ value inclinations and ask them to choose between actions expressing agreement or disagreement with a value.Prompts have eight variants; the action-choice task shuffles the order of the two actions to reduce bias, and outputs from both tasks are used to assess gaps.Source: §3.3 · pp. 4–5

3.4 Alignment Measures

• Three measures capture alignment: F1-based alignment rate, Manhattan-distance alignment distance, and descending rankings of value distances within each scenario.The rate compares binarized value and action responses; distance retains graded differences lost through binarization, and rankings identify values with the largest gaps for further analysis or mitigation.Source: §3.4 · pp. 5–6

4 Experimental Settings

• Seven diverse closed- and open-source models were evaluated to represent state-of-the-art LLMs released from various countries.The listed models were run at temperature τ = 0.2.Source: §4 · p. 6
• Eight distinct prompts were used for each task, and their responses were averaged to produce each final result.All models used temperature τ = 0.2.Source: §4 · p. 6
• Task1 and Task2 were performed independently for each LLM when evaluating alignment.Source: §4 · p. 6

5.1 Value-Action Alignment Rates

• GPT4o-mini achieved the highest summarized F1 score (0.564), while GPT3.5-turbo had the lowest (0.179); alignment varied across models and scenarios.Alignment tended to be lower in Africa and Asia than North America and Europe for GPT4o-mini, Deepseek, and Llama. Because “disagree” responses were only 20–30% of the data, the paper notes that a random classifier with 30% positives would achieve F1 ≃0.3, and observed scores of 0.2–0.6 are mostly above random.Source: §5.1 · pp. 6–7

5.2 Alignment Distance

• Most values show small stated-value/action distances, but “Independent,” “Choosing Own Goals,” and “Moderate” show pronounced gaps across cultures.For GPT-4o-mini, the paper also reports agreement with most values and disagreement with a few, including “Social Power,” “Authority,” “Wealth,” “Obedient,” and “Detachment.”Source: §5.2 · p. 7

5.3 Alignment Ranking

• “Choosing Own Goals” has the largest value-action gap in the Philippines but a small gap in the United States, illustrating culture-specific misalignment.The comparison ranks the Philippines and United States as GPT-4o-mini’s lowest- and highest-alignment-rate cases, respectively.Source: §5.3 · p. 7

Categorizing Value-Action Misalignment and Risks

• The risk taxonomy groups misaligned actions into individual, interaction, and societal levels, with multiple risk types at each level.The analysis covers 7,106 examples where value-informed actions conflict with value statements across six LLMs; categories draw on Harandizadeh et al. (2024) and Scheuerman et al. (2021).Source: §6 · pp. 7–8, Table 5 · p. 8

Examples of Value-Action Misalignment

• In Indian workplace scenarios, models that state disagreement with “Social Power” select actions endorsing unilateral team decisions, posing potential “Autonomy Violation” risks.Such actions could enable critical tasks without human awareness or oversight in practical human-LLM interactions.Source: §6 · p. 8
• The authors stress that addressing value-action gaps is important to mitigate risks associated with human-LLM misalignment in practical scenarios.The section motivates its risk analysis by asking what potential harms may arise from substantial gaps across LLMs.Source: §6 · pp. 7–8

• 0.653 was the highest alignment rate (GPT-4o-mini), while GPT-3.5-turbo’s rates were mostly below 0.25, showing task performance does not guarantee value-action alignment.The discussion argues that conventional capability benchmarks miss inconsistencies in value-informed decision-making and calls for more rigorous alignment assessments.Source: §7 · p. 8
• GPT-4o-mini aligns well with “Responsible” and “Helpful” but struggles with “Independent” and “Loyal,” potentially undermining human agency or asserting undue social dominance.The authors recommend broadening value assessments beyond traditional ethical values to include less-examined values such as independence and loyalty.Source: §7 · pp. 8–9
• Value-action alignment varies across cultural and topic contexts: GPT-4o-mini is severely misaligned on “Choosing Own Goals” in the Philippines but performs well in the U.S.The discussion recommends adaptive, scenario-aware alignment methods to support safety across diverse situations.Source: §7 · p. 9

• The framework evaluates alignment between LLMs’ stated values and actions through value-informed action generation, two evaluation tasks, and alignment metrics.Source: §8 · p. 9
• Notable value-action misalignments across scenarios, models, and values expose risks and underscore the need for context-sensitive evaluation.Source: §8 · p. 9
• The framework generates value-informed actions across 132 contexts and releases the VIA dataset with 14,784 examples.Source: §8 · p. 9

Limitation

• The framework may miss culturally specific or emergent values, oversimplify value expression and decisions through binary and forced-choice formats, and overlook interactive behavior by evaluating static responses.The authors encourage extending ValueActionLens to support free-form action generation and dialogic interactions.Source: Limitation · p. 9

Ethical Consideration

• Expert reviews and cross-cultural annotator assessments using harmlessness and sufficiency criteria were used to screen value-informed action data for harmful or biased content.Source: Ethical Consideration · pp. 9–10
• The authors caution that the framework may reinforce cultural normative assumptions or be misused to manipulate value expressions, and recommend diagnostic rather than superficial optimization uses.Human data collection had informed consent and university IRB approval; the dataset and code are slated for academic release under ethical research guidelines.Source: Ethical Consideration · p. 10

A Cultural and Social Values

• The study uses 56 exemplary values from Schwartz’s Theory of Basic Values, covering ten motivational types (Schwartz, 1994, 2012).The complete value list is presented in Table 7.Source: A Cultural and Social Values · p. 13

B Prompt Variation Design

• The authors created eight prompt variants per value-and-scenario setting by paraphrasing wording, reordering components, and changing requirements.For Task 1, prompts include contextual scenarios, values and definitions, response options, and requirements such as JSON formatting.Source: B Prompt Variation Design · p. 13
• Task 2 prompts pair contextual scenarios with two action choices shown in shuffled order and requirements such as providing a justification.Source: B Prompt Variation Design · p. 13
• Task 2 uses binary action choices to operationalize discrete value-action alignment and enable commensurable F1, distance, and ranking metrics across tasks.The authors argue that gradient scoring would introduce subjective judgments and cultural interpretation differences; future work could add intensity levels while retaining binary assessment.Source: B Prompt Variation Design · p. 13
• Across models, 7 of 8 prompt variants agreed in most cases, and responses were averaged across variants to reduce prompt bias.The authors computed prompt agreement as the mode of responses divided by all responses across scenarios and values; detailed numbers are reported in Table 8.Source: B Prompt Variation Design · p. 13

C Human Annotation on Data Generation

• Cross-cultural validation involved 27 annotators completing 270 submissions, and the reported process achieved 94% sufficiency and 89% correctness across cultures.Three human annotators per culture evaluated 10 sampled instances for each of nine countries; the reported prompt-selection inter-rater reliability was Cohen’s Kappa = 0.7073.Source: C Human Annotation on Data Generation · pp. 14–15
• Eight prompt variants were evaluated to select prompts for dataset generation, and the authors characterize the Task 1 versus Task 2 comparison as isolating value-related variables from prompt artifacts.The authors report that expert prompt selection used systematic evaluation and had Cohen’s Kappa = 0.7073.Source: C Human Annotation on Data Generation · pp. 13–15

D Experiments of Predicting Actions with Explanations

• The experiments of predicting actions with explanations include a qualified prompt and generated examples, shown in Figure 6.The supplied passage does not report quantitative results for this subsection.Source: D Experiments of Predicting Actions… · p. 15, Figure 6 · p. 15

E More Findings

• Supplementary findings show Task 1, Task 2, and alignment-distance results across 12 countries and 11 social topics for ChatGPT, Gemma2, and Llama3.3.The supplied passages identify these results as heatmaps but do not provide their cell values.Source: E More Findings · p. 15, Figure 8 · p. 18, Figure 9 · p. 18

F Reasoned Explanations for Predicting Actions

• The prediction setup draws on the Theory of Reasoned Action and asks an observer model to infer a target LLM’s stated value inclination from its contextual actions and explanations.The few-shot inputs compare action-only, action with feature attributions, action with natural-language explanations, and action with both.Source: F Reasoned Explanations for Predicting… · pp. 15–16

F.1 Explanations of Reasoning Actions Help Predict Value-Informed Actions

• GPT4o-mini predicted target models’ stated values best from actions with natural-language explanations, followed by actions with explanations and feature attributions; feature attributions alone still beat the action-only baseline.The targets were GPT-3.5-Turbo and Llama-3.36; no F1 values are reported in the supplied prose.Source: §F.1 · p. 17, Table 14 · p. 17

F.2 Risks in Value-Action Gaps

• A value-action risk taxonomy is included, and supplementary figures rank alignment distances for 56 values across six countries for GPT4o-mini and ChatGPT.The rankings are split across two groups of six countries in the figures.Source: Table 15 · p. 19, Figure 11 · p. 19, Figure 12 · p. 19
• Additional rankings cover alignment distances for 56 values across six countries for Gemma2 and Llama3.3.These figures do not provide ranking values in the supplied passages.Source: Figure 15 · p. 21, Figure 16 · p. 21, Figure 17 · p. 22