PaperlayerView exact revision on arXiv

Source-linked AI summary

Towards Formal Definitions of Blameworthiness, Intention, and Moral Responsibility

Joseph Y. Halpern, Max Kleiman-Weiner

arXiv:1810.05903v1cs.AIcs.CY

TL;DR

Moral responsibility requires judgments about causality, blameworthiness, and intention, but trolley cases show that these concepts can be difficult to assess. The paper formalizes blameworthiness and intention using causal models, probabilities, and utility; its examples show that intention can depend on how a case is modeled, so the two notions remain separate components of responsibility.

  • Problem

    Moral-responsibility judgments require integrating causality, blameworthiness, and intention, concepts whose assessment is illustrated as difficult by trolley and loop cases.

    Source: §1 · p. 1

  • Method

    The paper defines blameworthiness and intention using causal models augmented with probabilities and, for intention, an expected-utility assumption.

    Source: §1 · p. 1

  • Results

    The trolley examples show that intention can depend on the causal model’s variables, and the authors leave intention and blameworthiness as separate components of moral responsibility.

    Source: §4 · p. 8, §7 · p. 12

  • Takeaways & Limitations

    The authors present the framework as relevant to moral-responsibility judgments as autonomous vehicles, robots, and software assistants develop.

    Source: §7 · p. 12

    Intention is model dependent and can change with the choice of variables used to represent a case.

    Source: §4 · p. 8

Abstract

from arXiv · show

We provide formal definitions of degree of blameworthiness and intention relative to an epistemic state (a probability over causal models and a utility function on outcomes). These, together with a definition of actual causality, provide the key ingredients for moral responsibility judgments. We show that these definitions give insight into commonsense intuitions in a variety of puzzling cases from the literature.

1 Introduction

Judging moral responsibility matters in ethics and law and is increasingly relevant to autonomous AI; the paper formalizes blameworthiness and intention alongside causality. Trolley and loop cases illustrate why these notions are difficult to assess.

1 Introduction

  • Autonomous AI makes moral-responsibility evaluation newly relevant, and the paper formalizes blameworthiness and intention to complement causality in responsibility judgments.The introduction notes broad agreement that responsibility requires causality, blameworthiness, and intention (Cushman 2015; Malle, Guglielmo, and Monroe 2014; Weiner 1995).Source: §1 · p. 1
  • In the trolley case, pulling the lever saves five at the cost of one, while the loop variant is typically judged less morally permissible.The loop version rejoins the main track, and the agent knows the person on the loop will stop the train before it reaches the five.Source: §1 · p. 1
  • The framework builds on structural-equations causality, adds probabilities to model nondeterministic action effects, and uses expected utility to define intention.Blameworthiness becomes a degree in [0,1]; intention concerns whether the agent would have acted if the action had no impact on the outcome.Source: §1 · p. 1
  • The paper reviews causality, defines blameworthiness and intention, discusses computational issues and related work, and concludes.Source: §1 · p. 1

2 Structural equations and HP causality

The section represents actions and outcomes with variables in structural causal models, then uses counterfactuals to define actual causality. The HP conditions address cases where a simple but-for test misses a cause and require actual occurrence and minimality.

2 Structural equations and HP causality

  • A causal model assigns structural equations to endogenous variables, while exogenous variables capture factors outside the model and an action variable represents an agent’s choices.In the trolley example, the action determines whether five people die and whether the person on the side track dies.Source: §2 · pp. 1–2
  • The framework restricts attention to acyclic models, which have a unique solution for each context and allow variables to be solved in causal order.Source: §2 · p. 2
  • An intervention replaces selected variables’ equations with fixed values, and a counterfactual formula states whether an outcome would hold under that intervention.The intervention model otherwise remains identical to the original model.Source: §2 · p. 2
  • The HP account uses counterfactuals but adds conditions to handle cases such as Suzy’s throw causing a bottle to shatter despite Billy’s backup throw.The account permits testing what happens when Suzy does not throw while holding fixed that Billy did not throw in the actual world.Source: §2 · pp. 2–3
  • AC1 requires the candidate cause and outcome to occur, while AC3 removes inessential conjuncts so causes remain minimal.AC2 supplies the core counterfactual test, with contingencies used to address the backup-throw case.Source: §2 · pp. 2–3

3 Degree of blameworthiness

The section defines blameworthiness for a particular outcome by combining how much more likely that outcome is under an action than an alternative with the relative cost of the action. Examples show why outcome-specific counterfactuals matter, while extensions to praise and group blame expose further modeling choices and unresolved questions.

3 Degree of blameworthiness

  • The assessment uses a pre-action probability over causal settings and a utility function over worlds, and the probability need not be the agent's subjective belief.The probability can instead represent what the agent should have believed, such as after reading relevant warnings.Source: §3 · pp. 3–4
  • Action cost is modeled through outcome variables representing features such as difficulty, danger, or emotional upheaval, with their utility impact depending partly on the modeler.The framework defines expected cost from the utility difference between the world without the action and the world reflecting its action-cost variables.Source: §3 · pp. 3–4
  • Blameworthiness combines the probability improvement available from an alternative with the relative cost of the chosen action, maximizing over alternatives.When the alternative is more costly, its cost mitigates blame; N controls how strongly cost is weighted.Source: §3 · pp. 3–4
  • The lever example assigns blame for the sixth death, but not the five deaths that either action leaves unchanged.The alternative reduces the sixth person's death probability to 0.2, illustrating that a better action matters only when it changes the outcome being assessed.Source: §3 · p. 4
  • For good outcomes, the paper proposes measuring praiseworthiness against a default action and adjusting for the cost of the least-effortful sufficiently effective act.The default is often doing nothing, and the proposed measure is bounded rather than increasing without limit with effort.Source: §3 · pp. 4–5
  • Praiseworthiness is zero for an unintended outcome, and the proposed comparison excludes cheaper acts that are almost as effective.The authors identify considering those nearly as effective acts, and testing how people judge them, as future work.Source: §3 · p. 5
  • In the fishermen example, individual blame decreases when changing one agent's action is less likely to change the overfishing outcome, including when more fishermen participate.A fisherman who catches less fish is also less blameworthy under the account.Source: §3 · p. 5
  • Individual blame assessment holds the other fishermen's actions fixed, while assigning collective blame leaves the question of distributing that blame among individuals unresolved.The authors note that individual blame can fall well below an intuitive 1/N in the stated setting and call for further research on group attribution.Source: §3 · p. 5

4 Intention

The section formalizes intention in terms of an agent’s utilities, possible actions, and causal model, distinguishing intentions to act from intentions about outcomes. It also considers a Frankfurt-style election case in which Jones’s blameworthiness depends on the certainty that the army will install Smith if Jones loses.

Intention definitions

  • An action counts as intended only if it was actually performed, another possible action existed, and its expected utility was at least as high as that of alternatives.The availability of alternatives matters: if the agent had no real choice, the action is not treated as intended.Source: §4 · p. 6
  • An agent intends to affect a minimal set of outcome variables when fixing their action-produced values makes a reference action at least as good, and considers the intended outcome possible and highest in expected utility.The definition allows a relevant superset of the target variables when the action would not be abandoned by considering each variable alone.Source: §4 · pp. 7–8

Reference actions

  • The reference set can change which outcomes an action intends: comparing a job choice with the expected job excludes earnings, while comparing it with doing nothing includes them.Default and socially appropriate actions can serve as reference points, so these judgments may vary with the viewpoint represented in the model.Source: §4 · pp. 7–8

Limits of intention

  • For probabilistic actions, the highest-payoff possible outcome may be something an agent hopes for rather than intends, as with a lottery win or a hoped-for cancer remission.The passage also notes that “hopes” may apply to several reasonably good outcomes, not only the best one.Source: §4 · p. 8

Model dependence

  • Intended outcomes depend on model variables: in the loop trolley case the agent intends to hit but not kill, and separating means from ends can change the student’s intended outcomes.With separate variables for hitting and death, the trolley agent intends to hit the person but not kill him; the student intends both grades and a job only after the model adds a sense-of-accomplishment variable.Source: §4 · pp. 8–9

Intention and responsibility

  • Jones can intend Smith’s death yet have zero blameworthiness if Black is certain to persuade him otherwise; an accidental act does not become intended merely because it serves an agent’s prior goal.If there is positive probability that Black could not persuade Jones, Jones has positive blameworthiness; it rises as the probability of Black’s successful persuasion falls.Source: §4 · pp. 9–10

4 Intention

  • If the army is certain to declare Smith the victor even if he loses, Jones’s degree of blameworthiness for the outcome is 0.The passage presents this as a more realistic Frankfurt-style problem in which Jones votes for Smith.Source: §4 · p. 10
  • It is unreasonable to be certain that the army can install Smith if he loses, even when that outcome seems likely.The probability of the army’s ability to intervene may be high without being 1.Source: §4 · p. 10
  • The higher the probability that the army can install Smith if he loses, the lower Smith’s degree of blameworthiness.Source: §4 · p. 10

5 Complexity considerations

The section contrasts the complexity of causal-cause testing with polynomial-time computation of blameworthiness and intention over an epistemic state. Intention testing is also polynomial under a bound on the number of outcomes considered.

5 Complexity considerations

  • Blameworthiness and intention can be computed in time polynomial in the size of the epistemic-state set and the number of actions.The difference measure δ is polynomial in |K|; the stated bounds for blameworthiness and intention also depend on the cardinality of the action range.Source: §5 · p. 10
  • Causal-cause testing is Σp2-complete, while the similar complexity of determining intention is conjectured to be Σp2-complete.The causal test has an “there exists–for all” structure.Source: §5 · p. 10
  • If the number of outcomes determining utility is bounded by fixed k, intention testing is polynomial in the model variables and actions, with degree k.The authors consider a relatively small k reasonable for practical applications, making the problem tractable under that assumption.Source: §5 · p. 10

6 Related work

Although many accounts discuss moral responsibility and intention, the authors report little formal work on degree of blameworthiness. Related approaches differ in their treatment of counterfactuals, utility, intention, and group responsibility.

6 Related work

  • The authors report little formal work on moral responsibility and no formal definition of degree of blameworthiness that they are aware of.They note that other work has used probability and utility, but has not supplied such a definition.Source: §6 · p. 10
  • Chockler and Halpern (2004) use expected degree of responsibility to model blame, but omit utility, alternative actions, and intention.Their responsibility measure captures diffusion across agents, including cases where several vote changes would be needed to make a voter critical.Source: §6 · p. 10
  • Cohen and Levesque (1990) formalize goals and intentions in modal logic, while Kleiman-Weiner et al. (2015; 2016) use counterfactual reasoning and expected utility for intention.Kleiman-Weiner et al. also develop a theory of moral permissibility tested against human judgments across moral dilemmas.Source: §6 · pp. 10–11
  • Van de Poel et al. (2015) address allocating group responsibility, while Gaudini, Lorini, and Mayor (2013) use a game-theoretic approach to group responsibility.The former’s CEDL variant cannot directly capture counterfactuals or quantitative probability; the latter does not discuss single-agent responsibility and has no causal model.Source: §6 · p. 11
  • STIT logic and event-calculus approaches express some responsibility concepts but lack counterfactuals, limiting their reach for subtle causal examples.Lorini, Longin, and Mayor (2014) focus on collective responsibility, while Barreby, Bourgne, and Ganascia (2015) show how their approach can capture the trolley problem.Source: §6 · p. 11
  • Vallentyne (2008) links responsibility to whether an action raises an outcome’s probability, without using utility or counterfactuals.The account also uses an agent’s disposition to act as an input, which the authors note may be hard to determine.Source: §6 · p. 11
  • Braham and van Hees (2012) make responsibility depend on causation, intended action, and lack of an eligible action with higher avoidance potential.They assume intention in relevant cases and, in a single-agent setting, do not allow uncertainty about outcomes.Source: §6 · p. 11

7 Conclusion

The paper formalizes blameworthiness and intention as components of moral responsibility, while leaving their combination open. Its framework is intended to inform responsibility judgments for autonomous systems, with important modeling and terminology caveats.

7 Conclusion

  • The authors formalize degree of blameworthiness and intention as components of moral responsibility, but leave how to combine them unresolved.They also identify causality as a component of people’s moral responsibility ascriptions.Source: §7 · p. 12
  • The definitions depend on a probability measure and utility function, whose construction is demanding and whose appropriate values may be disputed.The authors also question whether expected-utility maximization is the right decision rule and flag individual versus group responsibility for further study.Source: §7 · p. 12
  • The paper uses “blameworthy” in a sense closer to what some people call “responsible,” amid broader ambiguity among these terms.The authors say causal models can help distinguish related notions, including their blameworthiness and Chockler and Halpern’s responsibility and blame.Source: §7 · p. 12
Loading 1810.05903v1…