Source-linked AI summary
Mathematical foundations of moral preferences
Valerio Capraro, Matjaz Perc
TL;DR
Social preferences have been challenged as an explanation for one-shot anonymous unselfish behavior. The paper reviews experimental and theoretical work on moral preferences, develops a mathematical formalism, and concludes that this framework unifies diverse behaviors and has practical applications.
Problem
The social preference hypothesis faces a crisis because evidence such as truth-telling and intrinsic costs of lying are not fully explained by preferences over others’ payoffs.
Method
The paper reviews experimental and theoretical research and outlines a unified mathematical formalism for moral preferences.
Results
Moral preferences provide a framework for understanding one-shot unselfish behaviors including cooperation, altruism, truth-telling, altruistic punishment, trustworthiness, and equality-efficiency trade-offs.
Takeaways & Limitations
The formalism can inform future models, while making personal norms salient has shown practical promise for increasing charitable donations.
Takeaways & Limitations
The review documents unresolved limitations of social-preference accounts, including truth-telling evidence and intrinsic costs of lying beyond preferences over others’ outcomes.
Abstract
from arXiv · showhide
One-shot anonymous unselfishness in economic games is commonly explained by social preferences, which assume that people care about the monetary payoffs of others. However, during the last ten years, research has shown that different types of unselfish behaviour, including cooperation, altruism, truth-telling, altruistic punishment, and trustworthiness are in fact better explained by preferences for following one's own personal norms - internal standards about what is right or wrong in a given situation. Beyond better organising various forms of unselfish behaviour, this moral preference hypothesis has recently also been used to increase charitable donations, simply by means of interventions that make the morality of an action salient. Here we review experimental and theoretical work dedicated to this rapidly growing field of research, and in doing so we outline mathematical foundations for moral preferences that can be used in future models to better understand selfless human actions and to adjust policies accordingly. These foundations can also be used by artificial intelligence to better navigate the complex landscape of human morality.
I. INTRODUCTION
The review examines why people act unselfishly in one-shot anonymous economic games and argues that personal moral norms complement monetary-payoff models. It synthesizes evidence for a unified moral-preference framework and outlines its mathematical and practical implications.
- Motivation: One-shot anonymous unselfishness is commonly studied through economic games involving monetary decisions and other-regarding behaviour.The review defines selfishness and other-regarding behaviour with respect to monetary payoffs, while noting that psychological benefits and costs may also matter.
- Earlier explanations: Social-preference models initially explained unselfishness by assuming that people care about others’ monetary payoffs.These models were developed to explain behaviour beyond concern for the decision maker’s own payoff.
- Theoretical shift: Experiments challenged monetary-payoff explanations for altruistic punishment and altruism, motivating preferences for following personal norms beyond economic consequences.Personal norms are described as what people personally believe to be the right thing to do, and they may vary across cultures and individuals.
- Theoretical shift: The moral preference hypothesis organises cooperation, altruism, altruistic punishment, trustworthiness, honesty, and the equality-efficiency trade-off.The framework treats these behaviours as reflecting preferences for following personal norms in addition to monetary consequences.
- Applications: Making the rightness of an action salient can promote desirable behaviour, and nudges toward doing the right thing have increased charitable donations.The review presents this as a practical application of the moral preference hypothesis beyond the laboratory.
- Review contribution: The review surveys economic games, social-preference models, empirical challenges, moral-preference models, practical applications, and future research.It proposes a mathematical formalism intended to inform models of selfless action and artificial intelligence that seeks to emulate counterintuitive human decision-making.
II. MEASURES OF UNSELFISH BEHAVIOUR
Researchers use simple, incentivised games and decision problems to measure distinct forms of one-shot unselfishness. The review distinguishes pure unselfishness from strategically unselfish behaviour that may increase the decision maker’s own payoff.
- Measurement approach: Behavioural scientists use simple scenarios with real consequences and monetary payoffs to measure different forms of unselfish behaviour.These games are designed to represent prototypical decisions while incentivising participants’ choices.
- Games and behaviours: The dictator game measures altruism, the prisoner’s dilemma cooperation, and the sender-receiver game truth-telling.The review assigns each game to a specific behavioural domain.
- Games and behaviours: The trade-off game measures equality-efficiency choices, the trust game trustworthiness, and the ultimatum game altruistic punishment.These measures target distinct forms of unselfish behaviour in one-shot interactions.
- Pure versus strategic unselfishness: The review also considers strategic trust and strategic fairness, where behaviour may maximise the decision maker’s payoff depending on beliefs about the other player.These behaviours are contrasted with purely unselfish actions that incur a cost to benefit another person regardless of the other person’s behaviour.
III. SOCIAL PREFERENCES AND THEIR LIMITATIONS
Social-preference models explain unselfishness through monetary payoffs, including concern for inequity and others’ outcomes. A series of experiments showed that behaviour changes with context even when monetary consequences remain unchanged, exposing limits of purely monetary accounts.
- Social-preference models: Social-preference models represent utility as a function of monetary payoffs and include competitive, inequity-aversion, altruistic, and social-efficiency preferences.The review focuses on models intended to apply across one-shot anonymous interactions involving unselfish behaviour.
- Empirical limitations: In ultimatum games, responders’ acceptance of (8,2) changes with the proposer’s alternative despite identical monetary consequences for the offer.Responders prefer accepting (8,2) when the alternative is (10,0), but prefer rejecting it when the alternative is (5,5).
- Empirical limitations: In sender-receiver games, people are less likely to implement an allocation when doing so requires misreporting private information.The finding indicates an intrinsic cost of lying beyond preferences over monetary outcomes.
- Empirical limitations: When lying benefits sender and receiver equally, a significant proportion of participants still tell the truth, contrary to social-preference predictions.Purely monetary utility functions predict that everyone would lie in this condition.
- Empirical limitations: Dictator-game choices also change when participants can exit or take money, although the relevant monetary outcomes can remain comparable.Some people prefer giving when forced to play but prefer keeping when they can avoid the interaction or take money from the recipient.
- Empirical limitations: After 2013, challenges extended to the prisoner’s dilemma, trust game, and trade-off game, producing a broader crisis for the social-preference hypothesis.These results motivated alternatives that incorporate factors beyond monetary payoffs.
IV. THE RISE OF THE MORAL PREFERENCE HYPOTHESIS
The moral preference hypothesis emerged as a unified account of one-shot anonymous unselfishness, explaining behaviour through personal norms in addition to monetary consequences. Evidence across games suggests that personal norms often explain context and framing effects better than social or descriptive norms.
- Norms and context: In ultimatum games, responders’ fairness judgments and rejection decisions vary with the proposer’s choice set, consistent with personal norms beyond monetary consequences.Responders tend to reject offers they consider unfair, and the same offer can be rejected at different rates depending on available alternatives.
- Norms and context: Evidence from dictator games indicates that injunctive norms can explain context effects, but personal norms are more likely to motivate behaviour in private anonymous decisions.Publicizing choices increased pro-sociality, whereas private norm information did not significantly increase it; other findings link behaviour to perceived moral rightness.
- Norms and framing: Moral framing affects dictator donations, trade-off decisions, and cooperation, and people tend to follow personal norms even when descriptive norms conflict.The review also reports that framing effects correlate with dictator giving and prisoner’s-dilemma cooperation.
- Conclusions: The framework links altruism, altruistic punishment, truth-telling, cooperation, trustworthiness, and equality-efficiency choices within one account.The review presents mathematical formalisation as a basis for future models of selfless action and artificial intelligence.
- Cross-game evidence: Truth-telling, dictator-game giving, and prisoner’s-dilemma cooperation are positively correlated, suggesting a shared normative basis across behaviours.The review interprets this pattern as evidence that lying aversion may also be driven by personal norms.
- Moral preference hypothesis: The moral preference hypothesis explains unselfish behaviour by assuming preferences for following personal norms beyond monetary consequences.Personal norms are internal standards about what is right or wrong in a given situation.
V. PRACTICAL APPLICATIONS
Norm-based interventions can foster pro-social behaviour while avoiding the monitoring costs associated with punishment and rewards. Research suggests that targeting personal norms can increase donations and cooperation, reduce in-group favouritism, and produce effects beyond the immediate interaction.
- Norm-based interventions have been criticised for violating freedom of choice and for potential exploitation by malicious institutions.
- Norm-based interventions manipulate descriptive or injunctive norms and measure effects on behaviour in the same context.
- Targeting personal norms may be more effective and cheaper than targeting social norms or using punishment and rewards.It avoids monitoring costs and the costs of collecting information about behaviour or others’ moral judgments.
- Making personal norms salient increases donations and cooperation while decreasing in-group favouritism, at least on average.
- Asking people what they personally think is morally right increases crowdsourced charitable donations by 44%.
VI. MODELS OF MORAL PREFERENCES
Formal models of moral preferences extend payoff-based utility by representing the moral costs or benefits of actions. The reviewed literature increasingly motivates models based on personal norms, which predict behaviour across a broader set of games than models based only on social norms.
- The personal-norm utility function has greater predictive power than counterparts based only on social norms because it explains behaviour in a larger set of games.
- Existing models represent utility as monetary payoff plus a moral cost or benefit associated with the action.
- Levitt and List model moral costs using observation, consequences for others, and consistency with social norms or legal rules.
- Krupka and Weber model injunctive norms with N(a), which represents how socially appropriate society considers action a, and elicit it through modal ratings.
- Kimbrough and Vostroknutov construct an injunctive norm axiomatically from the game and relate it inversely to players’ overall dissatisfaction.
- A limitation of the axiomatically constructed injunctive-norm approach is that it always prefers Pareto-dominant allocations, unlike some experimental choices.
- Personal-norm models are consistent with truth-telling despite Pareto-dominant lying and with choosing morally framed Pareto-dominated options.
- Personal-norm utility adds the individual’s concern for doing what they personally regard as morally right, represented by µ_iP_i(a).P_i(a) is individual-specific and may differ from the social norms represented by m(a), N(a), and η(a).
VII. FUTURE WORK
The field offers a unified view of human choices across decision-making contexts and has practical implications, while leaving several questions for future research.
- Moral-preference research provides a unified view of human choices across several decision-making contexts.
- The literature also has significant practical implications, but several questions remain for future research.
A. The utility function
The proposed utility function models decision makers as balancing monetary payoff against doing what they personally regard as morally right. Future work should improve measurement of personal norms and account for how language and emotional content shape moral perceptions.
- The paper frames moral preferences as a longstanding open question, even in one-shot anonymous interactions.
- The proposed utility function combines monetary payoff with the extent to which a player values personally defined moral rightness: u_i(a) = v_i(π_i(a)) + µ_iP_i(a).
- This personal-norm utility function outperforms counterparts based on social norms and social preferences.
- Future experiments should detect small variations in how people weight personal norms against monetary incentives.
- Personal moral judgments should be estimated without requiring participants to report them in a separate experiment.
- Changing one word in instructions can change perceived moral rightness, suggesting that P_i(a) partly depends on how an action is presented.
- Emotional content in instructions and sentiment analysis may help estimate P_i(a) as an additional motivation or obstacle beyond monetary consequences.
- The proposed utility function is only a first candidate, motivating further formalisation of moral preferences.
B. Evolution of norms
Personal norms may arise from behaviours that are costly short term but beneficial in the long run. Evolutionary approaches are being extended beyond cooperation to other unselfish behaviours.
- B. Evolution of norms: Personal norms may reflect behaviours that are not individually optimal in the short term but are optimal in the long run.
- B. Evolution of norms: Future research should identify which unselfish behaviours can be selected over time and under what conditions.
- B. Evolution of norms: Evolutionary game theory and statistical physics are used to study conditions promoting cooperation on networks.
- B. Evolution of norms: Related work applies similar evolutionary techniques to truth-telling, trustworthiness, and ultimatum-game choices.
C. Personal norms versus social norms
The reviewed evidence supports personal norms as a framework for unselfish behaviour while also indicating that social and injunctive norms can influence decisions. Effects vary with which norms are made salient and how they interact.
- C. Personal norms versus social norms: One-shot anonymous unselfishness can be unified under a framework in which people have preferences for following personal norms.
- C. Personal norms versus social norms: Making personal norms salient affects altruism, cooperation, altruistic punishment, and trade-off decisions between equality and efficiency.
- C. Personal norms versus social norms: Nudging injunctive norms significantly affects decisions in one-shot anonymous prisoner’s-dilemma and trade-off games.
- C. Personal norms versus social norms: Evidence comparing norm types remains limited: one dictator-game study found that people tend to follow descriptive norms.
- C. Personal norms versus social norms: Personal and injunctive norms have similar effects in the trade-off game, but conflicting norms lead some people to follow each type.
- C. Personal norms versus social norms: Future experiments should compare different norms and examine situations where multiple norms are simultaneously salient or choices are observable.
D. Boundary conditions of interventions based on personal norms
Interventions targeting personal norms show pro-social effects, including outside the laboratory, but their durability, decisional scope, and relationship to multidimensional morality remain open questions.
- D. Boundary conditions of interventions based on personal norms: Personal-norm interventions can last for several interactions within an experiment, but their effects are not expected to persist indefinitely.
- D. Boundary conditions of interventions based on personal norms: Repeated injunctive-norm interventions can produce diminishing effects that are restored after sufficient time between interventions.
- D. Boundary conditions of interventions based on personal norms: The effectiveness of targeting personal norms may vary across behavioural domains, including risky cooperation in the stag-hunt game.
- D. Boundary conditions of interventions based on personal norms: 44%: nudging personal norms increased crowdsourced charitable donations to real humanitarian organisations in the only study beyond laboratory decisions.
- D. Boundary conditions of interventions based on personal norms: The moral phenotype may be multidimensional, unlike the unidimensional cooperative phenotype, because different personal norms may underlie different unselfish behaviours.
- D. Boundary conditions of interventions based on personal norms: Existing research has not yet established how different personal norms relate to different forms of one-shot unselfish behaviour.
- D. Boundary conditions of interventions based on personal norms: Whether strategic fairness and trust belong to the moral phenotype remains unresolved, with evidence linking these behaviours to norms being mixed.
G. A dual-process approach to personal norms
Research on cognitive processing suggests that whether personal norms guide unselfish behaviour automatically or deliberatively depends on behavioural context and decision-maker characteristics. The moral preference hypothesis is developing into a unified framework with practical implications for policy and artificial intelligence.
- Cognitive basis: Whether personal norms arise automatically or require deliberation may depend on the behavioural context and the individual decision maker.More work is needed to identify which norms become internalised as automatic reactions, for whom, and in which contexts.
- Cognitive basis: Research using time pressure and cognitive load examines whether unselfish behaviour reflects instinctive responses.Promoting intuition favours cooperation and altruistic punishment, whereas evidence for altruism is mixed.
- Cognitive basis: Intuition decreases truth-telling when lying harms abstract others but leaves truth-telling unaffected when it harms concrete others.
- Cognitive basis: Evidence is inconclusive about intuition’s role in trustworthiness and the equality-efficiency trade-off.
- Framework and implications: The moral preference hypothesis is emerging as a unified framework for cooperation, altruism, altruistic punishment, truth-telling, trustworthiness, and the equality-efficiency trade-off.
- Open questions: Future research should examine mathematical formalisations, norm internalisation, intervention boundaries, norm targeting, moral-phenotype topology, and cognitive foundations.
- Open questions: The review advocates interdisciplinary work because sustainable social welfare and organisation require cross-disciplinary approaches.
- Framework and implications: The review outlines a mathematical formalism intended to inform models of selfless actions and artificial intelligence that emulates human decision-making.The authors also connect this formalism to more efficient policies and interventions that increase good virtues and decrease bad ones.