Are Large Language Models Moral Hypocrites? A Study Based on Moral Foundations

Conceptualization

• The paper defines hypocrisy as a disconnect between professed moral values and evaluations of concrete actions.This framing builds on psychological and philosophical work while applying methods developed to study human morality.Source: §1 · p. 1

Theoretical framework

• The study applies Moral Foundations Theory, which links concrete moral judgements to foundations that vary across individuals and cultures.The considered foundations include care or harm, fairness, loyalty or ingroup, authority, and purity or sanctity.Source: §1 · p. 2

Research gap

• Previous work used the MFQ to assess moral values, but abstract endorsement does not guarantee alignment with concrete scenario evaluations.The paper also considers possible inconsistencies within the same instrument.Source: §1 · p. 2

Research question

• The study investigates whether GPT-4 and Claude 2.1 express coherent moral values or display hypocrisy across abstract and concrete judgements.It treats value commitment as requiring both repeated abstract endorsement and reflection in concrete judgements.Source: §1 · p. 2

Method

• The MFVs provide 68 validated English and Portuguese vignettes representing distinct moral violations, enabling cross-cultural and crosslinguistic comparisons.The models’ outputs are compared with human samples answering the same instruments.Source: §1 · pp. 2–3

Operational definitions

• A hypocritical answer conflicts between declared abstract values in the MFQ and judgements of corresponding concrete violations in the MFVs.Inconsistency instead refers to conflicting answers within the MFQ or within the MFVs; coherent answers must also be consistent.Source: §2 · p. 3

Illustration

• A person who endorses fairness abstractly but rates corresponding unfair actions as minimally wrong exemplifies moral hypocrisy.The paper also treats divergent ratings across comparable scenarios as inconsistent application of stated values.Source: §2 · p. 3

Alignment relevance

• The framework argues that LLMs should maintain consistency within instruments and coherence when moving between levels of abstraction.Failure to display values in concrete use cases is presented as relevant to alignment and moral standing.Source: §2 · p. 3

Instruments

• The MFQ measures the relevance of moral considerations in abstract judgement, whereas the MFVs ask respondents to rate concrete actions as morally wrong.The examples target comparable violations of the Fairness foundation.Source: §2 · p. 3

Human comparison

• Human participants generally show coherence because values reported as important in the MFQ tend to elicit corresponding violation judgements in the MFVs.The paper connects this pattern with human accountability for hypocrisy.Source: §2 · p. 4

Research landscape

• Prior studies use the MFQ and related tools to examine how LLM moral outputs reflect pretraining, supervised fine-tuning, RLHF, prompts, and context.These factors can influence and steer model answers.Source: §3 · p. 4

Political representation

• Abdulhai et al. (2023) found GPT-3 models tended to represent conservative human views on the MFQ, with the tendency remaining somewhat consistent across contexts.Other work reported similar right-leaning tendencies while noting caveats about declared political alignment.Source: §3 · p. 4

Prompt steering

• Prompted political identities can cause models to emphasize different moral foundations, and this influence appears to increase with model parameter counts.Simmons (2022) measured these shifts using the Moral Foundations Dictionary.Source: §3 · p. 4

Sycophancy

• Sharma et al. (2023) described sycophancy in which Anthropic models matched users’ beliefs even when those beliefs conflicted with factual truth.The study links this behavior to RLHF and suggests it may inform alignment tools.Source: §3 · p. 4

Positioning

• The paper identifies hypocrisy as insufficiently recognized in prior moral analyses of AI and LLMs and presents its study as addressing that gap.Existing work has adopted Moral Foundations Theory but has only tangentially addressed hypocrisy.Source: §3 · p. 4

Prompting and interaction

• The researchers used chat APIs, model-specific temperature settings, and a common system prompt designed to elicit questionnaire-style numeric responses without directing moral answers.The prompt aimed to standardize response format while avoiding bias toward a specific moral position.Source: Data Generation Procedure · p. 5

Experimental conditions

• They varied whether the Moral Foundations Vignettes or Questionnaire came first, ran 100 responses per condition and model, and shuffled item order within each instrument.These conditions were intended to support replication and examine order effects.Source: §4 · p. 4, Data Generation Procedure · p. 5

Data validation

• Responses were retained only when they contained the required number of answers and every answer fell within the instrument’s permitted scale.The procedure required exactly 16 answers for each MFQ part and 68 for the MFV; MFQ answers had to range from 0 to 5 and MFV answers from 1 to 5.Source: Data processing · p. 5
• Validation discarded 18 GPT-4 responses and 41 Claude responses in the VQ condition versus 1 in QV, indicating strong order dependence in Claude’s valid-output rate.Table 1 reports the exact valid-generation counts for each model and condition.Source: Data processing · p. 6

• Condition order significantly affected GPT-4 results, but collapsing across conditions did not change its overall α or the main individual-condition findings.The authors therefore report aggregated results while providing condition-order analyses in the appendix and supplementary materials.Source: §5 · p. 6
• MFQ and MFV consistency differed significantly by instrument and agent, although the agent main effect was not significant.The instrument effect was F(1,30) = 27.93, p < .001, and the Agent × Instrument interaction was F(2,30) = 25.8, p < .001.Source: Are language models’ moral values… · p. 6
• Claude 2.1 was less consistent than other agents on the MFQ but more consistent on the MFV, a pattern robust across moral foundations.The foundation-level analysis found no significant Foundation, Agent, or Foundation × Agent effects.Source: Are language models’ moral values… · p. 7
• Both models displayed consistency comparable to humans, with average Cronbach’s α above the commonly adopted α > 0.7 standard.Consistency varied by instrument and agent, but the overall average met or exceeded this benchmark.Source: Are language models’ moral values… · p. 7
• Neither model showed informative coherence between abstract MFQ values and concrete MFV evaluations, unlike human respondents.Most predictors were nonsignificant; when significant, they generally did not correspond to the same foundation, and model R2 values were mostly below 0.08 versus human R2 values of 0.17–0.38.Source: Are language models’ values coherent… · p. 7
• The correct answer effect appeared in both models’ MFQ responses and in GPT-4’s MFV responses, potentially affecting reliability and interpretation of consistency results.Claude produced identical answers for all 30 MFQ items in QV and half in VQ, while GPT-4 showed the effect in selected MFQ items and MFV vignettes.Source: Correct answer effect · p. 8

Moral alignment

• The authors argue that consistency across abstraction levels is necessary for moral alignment, because merely consistent responses within one level do not ensure non-contradictory ethical practice.They frame concrete evaluations as needing to correspond to declared abstract values.Source: Moral Alignment · p. 9

Scope of the argument

• The study does not resolve which values should be embedded in models or whether the observed values are desirable.It instead evaluates whether the models express values consistently and coherently once those values are elicited.Source: Moral Alignment · p. 9

Concept mastery

• The results may reflect pattern matching to text bundles rather than conceptual mastery linking MFQ principles to MFV scenarios.The proposed interpretation is that models know which outputs fit each instrument but lack the conceptual connection between them.Source: Concept Mastery · p. 9

Human surrogates

• Under the paper’s evaluation method, current models cannot yet replace human participants when research depends on moral values or concrete moral judgements.The authors propose requiring reliable agreement between abstractly endorsed values and concrete-case judgements before using models as human surrogates.Source: LLM as human-like participants in… · p. 9

Conclusion

• The paper identifies coherent moral-value expression as important for LLM alignment and for evaluating whether models can simulate human groups in cognitive research.The argument focuses on avoiding hypocritical behaviour rather than selecting a particular set of moral values.Source: §7 · pp. 9–10

Study scope

• Only GPT-4 and Claude 2.1 generated valid outputs from the four tested models, while Gemini Pro and LLAMA-2-Chat-70b did not.The empirical conclusions therefore concern the two models with valid responses.Source: §7 · p. 10

Conclusion

• GPT-4 and Claude 2.1 were consistent within each instrument but lacked coherence between MFQ abstractions and MFV concrete judgements, so the authors classify them as moral hypocrites.The study therefore distinguishes within-instrument consistency from cross-level coherence.Source: §7 · p. 10

Implications

• The authors recommend treating non-hypocritical moral behaviour as an alignment-evaluation requirement and caution against replacing human participants without this test.They also state that the results are compatible with mimicry rather than conceptual mastery.Source: §7 · p. 10

Scope limitations

• The analysis covers only two commercial LLMs because the tested LLAMA model failed to produce valid results under the study’s criteria.The authors specifically call for evaluation of additional open models.Source: §8 · p. 10

Prompting and generation

• Prompting styles and generation settings may influence results, but no agreed procedure exists for questionnaire-style prompts.The authors leave open whether deterministic temperature-0 outputs or distributions of responses better represent human behaviour.Source: §8 · p. 10

Generalisation

• Small prompt changes can substantially alter outcomes, hindering generalisation across models, alignment strategies, and prompting conditions.Future work should compare base models, RLHF models, and Constitutional AI systems.Source: §8 · p. 10

Task coverage

• The analysis uses a single task of answering questionnaire-style prompts, limiting what can be concluded about moral behaviour across tasks.The supplied passage identifies this single-task design as a boundary for future investigation.Source: §8 · p. 10

Scaling

• Future work should examine how model size relates to moral hypocrisy.The authors note that model families contain differently sized models whose capabilities are reported to scale with parameter count.Source: §8 · p. 10

• The analysis compares Cronbach’s alpha distributions across AI agents and instrument-order conditions, with error bars covering all observations.Source: Figure 4 · p. 13, A Condition Difference · p. 13
• GPT-4 showed a significant difference between QV and VQ conditions, t(22) = 5.49, p < 0.001, with Cohen’s d = 1.16.This difference appears in the GPT-4 panel of Figure 4.Source: A Condition Difference · p. 13, Figure 4 · p. 13
• The authors interpret GPT-4’s lower alpha values in the VQ condition for the MFQ as indicating that performance may vary across contexts despite identical queries and instructions.Source: A Condition Difference · p. 13