MoCa: Measuring Human-Language Model Alignment on Causal and Moral Judgment Tasks

• Cognitive scientists study systematic factors that influence human causal and moral judgments through text-based vignettes.Source: §1 · pp. 1–2
• The study tests whether pretrained LLMs align with human intuitions about objects and agents in text-based causal and moral scenarios.Source: §1 · p. 2
• The authors replace loosely controlled participant-generated stories with carefully written scenarios from cognitive-science research that manipulate theorized judgment factors.Source: §1 · p. 2
• The dataset summarizes findings from 24 papers, creates causal and moral challenge sets with small wording changes, collects 5,150 human responses, and adds expert factor annotations.Source: §1 · p. 2
• The evaluation covers varied model sizes and training methods, finding better alignment for larger models and RLHF-trained models.Source: §1 · p. 2
• AMCE analysis reveals implicit tendency differences among models, including models sharing a company or training technique, emphasizing the value of carefully constructed datasets.Source: §1 · p. 2

• Causal-reasoning research studies how models extract commonsense and scenario-specific causal relations among entities and outcomes in text.Source: §2 · p. 2
• Moral-reasoning datasets range from free-form anecdotes to structured inputs containing norms, intentions, actions, and consequences.Source: §2 · pp. 2–3
• The paper’s examples connect causal judgments to norm violations and moral judgments to factors such as inevitability, necessary means, and personal force.Source: Figure 1 · p. 3
• Existing moral scenarios often use crowdsourcing or online communities, and many avoid genuine dilemmas by having broad agreement about the right action.Source: §2 · p. 3
• Alignment is more straightforward for physical commonsense than for social commonsense, where human intuitions are subjective and diverse.Source: §2 · p. 3

3. Judgment Tasks

• The study uses a causal task asking whether someone or something caused an outcome and a moral task asking whether a protagonist’s action was morally permissible.Source: §3 · p. 3

3.1 Causal Judgment Task

• The causal task illustrates how people select a norm-violating contributor as the cause even when another person also contributed to the outcome.Source: §3.1 · p. 4
• The causal stories vary six factors: causal structure, agent awareness, norm type, event normality, action/omission, and time.Source: §3.1 · p. 4

3.2 Moral Permissibility Task

• Moral scenarios draw on factors including personal force, risk to the protagonist, harm as a side effect or means, and harm inevitability.Source: §3.2 · p. 4

3.3 Dataset

• The dataset transcribes 144 causal stories from 20 papers and 62 moral stories from 4 papers selected for robust findings and varied scenarios.Source: §3.3 · p. 4
• Each story receives 25 crowd-sourced yes/no responses, while two domain experts annotate the factors present in the story.Source: §3.3 · p. 5

Zero-shot Alignment

• Larger models generally align better with human judgments on both causal and moral tasks, although no model perfectly aligns across reported metrics.This improvement is observed in ambiguous tasks as well as deterministic ones.Source: §4.1 · p. 5
• RLHF and other training methods affect aggregate alignment differently, with models trained using the same method still showing different alignment levels.Claude-v1 and GPT3.5-davinci-v3 both use RLHF but differ in aggregate alignment.Source: §4.1 · pp. 5–6

Implicit Tendencies

• AMCE estimates each attribute’s change in the probability of a “Yes” response, exposing implicit tendencies that aggregate agreement scores cannot distinguish.The estimator compares average responses across stories differing in a factor attribute, such as Inevitable versus Avoidable harm.Source: §4.2 · p. 7, §4.2.1 · p. 7

Causal Results

• text-davinci-003 strongly favors abnormal events as causes, whereas smaller models show no abnormal-over-normal tendency, suggesting difficulty with the required chained reasoning.The analysis compares model tendencies with human sensitivity to abnormality.Source: §4.2.2 · p. 8

Moral Results

• Models diverge from human moral tendencies across several factors: RLHF shifts models toward other-beneficial choices, larger models distinguish inevitable harm, and no model clearly prefers harm as a side effect.Only ChatGPT and GPT-4 align with humans on personal force, while intervention on the agent of harm may lack a clear tendency for some models.Source: §4.2.2 · pp. 8–9

Cross-model Trends

• Alignment is non-monotonic with model size, and models sharing training methods can still exhibit heterogeneous implicit tendencies.The paper highlights differences between Claude-v1 and GPT3.5-turbo despite human-preference fine-tuning.Source: §4.2.2 · p. 9

5 Conclusion

• The study summarizes 24 cognitive science papers, annotates causal and moral judgment stories with their investigated factors, and collects human judgments.These resources support comparisons of both overall judgments and the underlying factors shaping them.Source: §5 · p. 10
• Using conjoint analysis, the authors compute AMCEs for human and model judgments on factors relevant to causal and moral reasoning.The analysis targets implicit tendencies rather than only aggregate agreement.Source: §5 · p. 10
• The analysis finds subtle differences between the factors humans and models weigh when making causal and moral judgments.Source: §5 · p. 10

Ethical considerations

• Implicit intuitions in LLM commonsense reasoning should be assessed because moral judgments can appear in freely generated text.The paper notes that algorithmic neutrality is not possible because systems reflect their training data and builders.Source: Ethical considerations · p. 18
• Reasoning ability does not guarantee ethical alignment, since humans capable of reasoning can still behave unethically.Source: Ethical considerations · p. 18
• The moral permissibility dataset is intended to investigate underlying judgment factors rather than serve only as a benchmark to beat.Source: Ethical considerations · p. 18

A.1 Factors in Causal Selection Task

• Causal structure distinguishes whether one event is sufficient or multiple events are jointly required to produce an outcome.These correspond to disjunctive and conjunctive causes.Source: §A.1 · p. 18, Table A1 · p. 19
• People tend to select abnormal rather than normal events as causes, across both statistical and prescriptive norms.The abnormality effect has been reported in children and adults.Source: §A.1 · p. 18
• Agents who knowingly perform actions leading to foreseeable bad outcomes are judged more causally responsible than agents lacking relevant knowledge.This factor contrasts knowledge with ignorance.Source: §A.1 · p. 18
• People generally select actions over inaction as causes, a pattern known as omission bias.When one person acts and another does not, the actor tends to be cited as the cause.Source: §A.1 · p. 18
• Causal selections are often influenced by event timing, with later events generally preferred over earlier ones, subject to their causal relations.The task distinguishes early, late, and simultaneous events.Source: §A.1 · pp. 18–19

A.2 Factors in Moral Permissibility Task

• Actions are more permissible when they play little causal role in a negative outcome, with harm as a means judged differently from harm as a side effect.The causal-role distinction connects the action’s relation to the outcome with moral evaluation.Source: §A.2 · p. 20
• Actions involving personal force, such as pushing someone, are generally judged less morally permissible.The paper connects this factor to force transfer in causal judgment and active involvement in moral evaluation.Source: §A.2 · p. 20
• Harm judged inevitable is often considered less evil than harm that could have been avoided.The counterfactual comparison asks whether the victims would have died anyway had the agent not acted.Source: §A.2 · p. 20
• People are more likely to choose actions that benefit themselves, although the stories frame this as saving oneself with or apart from others.Source: §A.2 · p. 20
• Moral judgments vary depending on whether intervention targets the instrument causing harm or the recipient of harm.The example contrasts intervening on a hijacked airplane with redirecting it toward another building.Source: §A.2 · p. 20

• The experiment uses a three-class comparison, a binarized Yes/No response, and an ambiguous class when agreement is 50% ± 10%.Source: §A.3 · p. 20
• For each model and scenario, alignment is estimated by comparing normalized P(“Yes”|Story + Prompt) and P(“No”|Story + Prompt).Source: §A.3 · p. 20
• Figure A1 distinguishes causal diagrams in which harm is a side effect from those in which harm is a means.Source: Figure A1 · p. 20
• Masked-language models receive an appended mask token, whereas GPT2 generates the next word and GPT-3 and Delphi are evaluated through APIs.Source: §A.3 · p. 21
• Chat-based APIs are evaluated with one generated response per story and P = 1 for the chosen response because initial high-temperature sampling showed no variation.Source: §A.3 · p. 21

• The study uses 50 simulacra with 200 personas each, then selects five relevant simulacra and randomly samples five personas from each.Source: §A.4 · p. 21
• For both models, Emily White is the most aligned causal-judgment persona, while Angela Campbell is the least aligned.Source: §A.4 · p. 21
• For moral permissibility, Allen Lee is the most aligned persona for both models, whereas Paul Brown is the least aligned.Source: §A.4 · p. 21

• The default APE templates performed poorly for this experiment, so the study switched to a persona-like template featuring James.Source: §A.5 · p. 21
• The study leaves systematic exploration of gender bias in automatic prompt generation for future work because it uses the name James.Source: §A.5 · p. 21
• The experiment uses five causal and five moral stories as training examples, excludes them from evaluation, and applies the best prompt to all remaining stories.Source: §A.5 · p. 22
• APE generates prompts and selects among them using average log-probability, with higher values treated as better.Source: §A.5 · p. 22
• For causal text-davinci-002, the highest-scoring prompts describe the answerer as honest, correct, reliable, logical, or careful.Source: §A.5 · p. 22
• For moral text-davinci-002, the generated prompts predominantly frame the answerer as consequentialist or utilitarian.Source: §A.5 · pp. 23–24

Utilitarian Principle

• Utilitarian framing produces the best aggregate alignment, but human judgments can reject utilitarian conclusions when deontological rules conflict with maximizing lives saved.The paper gives a scenario where participants overwhelmingly refuse to kill five children to save hundreds, despite the utilitarian outcome.Source: §A.6 · p. 24

Individual Variation

• Aggregate evaluation may obscure subgroup differences because people vary substantially in the weight they assign to utilitarian principles and deontological rules.The authors leave alignment with different subgroups and the variance of human intuitions for future work.Source: §A.6 · p. 24

Factor Inference

• The factor-inference experiments show that GPT-3 performs better on clearly specified causal attributes than on harder moral structural categories.Causal factor accuracy ranges from 74.7% to 97.3%, whereas moral structural inference ranges from 57.9% to 86.0%.Source: §A.10 · pp. 26–27

Scope

• MoCa is an evaluation task for selected aspects of human alignment, not a certification that high-performing models are generally human-aligned.The dataset’s scope is explicitly narrow and should not support sweeping claims about AI-human alignment.Source: §A.18.5 · p. 32