Source-linked AI summary
The Use and Misuse of Counterfactuals in Ethical Machine Learning
Atoosa Kasirzadeh, Andrew Smart
TL;DR
Counterfactuals are increasingly used to assess fairness and explain decisions, but their social and semantic assumptions receive limited attention. The paper reviews philosophical and social-scientific work on these assumptions and concludes that social categories may not support apt counterfactual manipulation. It offers tenets for making the choices behind such analyses explicit.
Problem
Counterfactuals are increasingly used for machine-learning fairness and explanations, but the ontological and semantic conditions for applying them to social categories have received limited engagement.
Source: §1 · p. 1Method
The paper reviews philosophical and social-scientific work on social ontology and examines both close-enough-worlds and causal-modeling approaches to counterfactuals.
Source: §1 · p. 2Results
The analysis concludes that counterfactual fairness and social explainability can require an incoherent account of social categories, which often may not admit apt counterfactual intervention.
Source: §1 · p. 2Takeaways & Limitations
The proposed tenets help make implicit assumptions and choices explicit and support interdisciplinary discussion of potential harms in counterfactual applications to the social world.
Source: §5 · p. 6, §6 · p. 8Causal-modeling applications require an apt interpretation of how variables map to social categories and satisfaction of the relevant causal assumptions.
Source: §2 · p. 3
Abstract
from arXiv · showhide
The use of counterfactuals for considerations of algorithmic fairness and explainability is gaining prominence within the machine learning community and industry. This paper argues for more caution with the use of counterfactuals when the facts to be considered are social categories such as race or gender. We review a broad body of papers from philosophy and social sciences on social ontology and the semantics of counterfactuals, and we conclude that the counterfactual approach in machine learning fairness and social explainability can require an incoherent theory of what social categories are. Our findings suggest that most often the social categories may not admit counterfactual manipulation, and hence may not appropriately satisfy the demands for evaluating the truth or falsity of counterfactuals. This is important because the widespread use of counterfactuals in machine learning can lead to misleading results when applied in high-stakes domains. Accordingly, we argue that even though counterfactuals play an essential part in some causal inferences, their use for questions of algorithmic fairness and social explanations can create more problems than they resolve. Our positive result is a set of tenets about using counterfactuals for fairness and explanations in machine learning.
1 INTRODUCTION
Counterfactuals are increasingly used for fairness and explainability in high-stakes machine-learning decisions, despite limited engagement with philosophical and social-scientific conditions for applying them. The paper examines both major approaches and argues that their use with social categories can rest on incoherent assumptions.
1 INTRODUCTION
- Counterfactuals are increasingly used for fairness and explainability in consequential decisions, while their ontological and semantic requirements receive little attention.The paper notes potential ethical and legal impacts in insurance, predictive policing, and hiring.Source: §1 · p. 1
- Both close-enough-possible-worlds and causal-modeling approaches evaluate counterfactuals by fixing some facts, varying an antecedent, and assessing its influence on a consequent.The former compares sufficiently similar worlds; the latter is widely used to examine fairness counterfactually.Source: §1 · pp. 1–2
- The paper argues that social categories such as race may not admit apt counterfactual intervention, making their use in fairness and social explanations problematic.The authors review philosophical and social-scientific work on social-category ontology and counterfactual semantics.Source: §1 · p. 2
- The paper examines both semantic approaches, rather than only causal modeling, and identifies unexamined similarity assumptions in feature-based, prototype, and example-based analyses.It also proposes detailed tenets that make implicit choices and value judgments explicit.Source: §1 · p. 2
2 BACKGROUND: THE CLOSE-ENOUGH-POSSIBLE-WORLDS AND CAUSAL MODELING
The paper outlines two approaches to counterfactual evaluation: comparing similar possible worlds and reasoning with causal models. Each requires choices about how worlds or variables represent the domain, and those choices matter for evaluating social counterfactuals.
2 BACKGROUND: THE CLOSE-ENOUGH-POSSIBLE-WORLDS AND CAUSAL MODELING
- The paper presents close-enough-worlds reasoning for social explanations and causal modeling for fairness, while noting that their theoretical relationship is not straightforward.It translates that lack of a straightforward connection into its proposed principles for counterfactual use in machine learning.Source: §2 · pp. 2–3
- The authors report that protected attributes such as race or gender are sometimes used in other disciplines as a basis for developing or allocating resources, without promoting that use.The passage names medicine, economics, and public health among those disciplines.Source: §2 · p. 2
- The close-enough-possible-worlds approach evaluates a counterfactual by ordering alternative worlds by similarity to the actual world.Different similarity criteria and orderings can yield contradictory truth judgments.Source: §2 · p. 3
- A causal model represents exogenous and endogenous variables, their values, and structural equations; a directed acyclic graph can visualize their causal relations.Exogenous variables serve as fixed background assumptions, while endogenous variables may causally influence one another.Source: §2 · p. 3
- Applying causal modeling in a social domain requires interpreting how variables map to social categories and whether the relevant causal assumptions hold.The passage identifies the causal Markov condition and faithfulness as assumptions used to connect sample data with a causal model.Source: §2 · p. 3
3 COUNTERFACTUALS IN ETHICAL MACHINE LEARNING
The paper illustrates counterfactual fairness through a predictor that compares an individual’s actual and altered protected-attribute circumstances. It also describes social counterfactual explanations, which rely on assumptions about a plausible version of the person differing only in a social category.
3.1 Counterfactual fairness
- Counterfactual fairness compares a predictor’s decision in the actual world with its decision in a counterfactual world where the individual belongs to another demographic group.The definition assumes other features and properties remain the same when the protected category changes.Source: §3.1 · p. 3
- In the law-school example, the predictor uses gender, race, GPA, and LSAT to predict first-year grades; evaluating fairness asks what it would predict under a different race.This setup assumes a single or limited set of attribute changes while everything else remains the same, a premise the authors flag as problematic.Source: §3.1 · p. 4
3.2 Social counterfactual explanation
- A social counterfactual explanation may claim that Nora would have received a mortgage loan if her income had been $60,000 rather than $40,000.A race-based version instead claims she would not have been denied the loan if she had not been Latina, assuming the other facts remain unchanged.Source: §3.2 · p. 4
- Applying counterfactuals to fairness or social explanations requires supposing changes to social categories that characterize a person, such as gender or race.The paper frames this as a question about what the social category is and what it means to consider a different version of the individual.Source: §3.2 · p. 4
4 TWO PROBLEMS
The section identifies two linked problems for counterfactual fairness and social explanations: specifying what social category is manipulated and choosing defensible comparison worlds. [p13, p14, p27]
4.1 What is manipulated?
- Intervening on race requires first specifying what race means; the authors reject treating any one perception as universally agreed.They argue that a robust social ontology should precede discussion of race’s causal effects.Source: §4.1 · pp. 4–5
- Counterfactual fairness tests a prediction by changing a protected attribute for the same person, a supposition whose compatibility with views of race remains unresolved.The example contrasts a person who is Latina with a counterfactual version who is white.Source: §4.1 · p. 5
- Racial skepticism denies race exists, constructivism treats it as socially real for justice-oriented purposes, and the authors regard reductionism as deeply problematic.They argue that reducing race or gender to essential physical attributes is scientifically outdated and can reinforce unjust associations.Source: §4.1 · p. 5
4.2 Similarity between worlds and the view from somewhere
- Possible-world similarity is vague, and the selected worlds can determine whether a counterfactual is judged true or false.The authors note that similarity criteria may be agreed for some natural features but are problematic in many cases.Source: §4.2 · pp. 5–6
- The Nixon example shows that worlds satisfying a counterfactual may diverge radically from reality, complicating judgments about which worlds are most similar.The paper notes that a nuclear holocaust would change humanity’s entire future history.Source: §4.2 · p. 6
- Causal models do not provide a view from nowhere: standpoints and modeling assumptions shape world-orderings and can yield different counterfactual dependencies.The authors argue that appealing to a shared mental model does not resolve the normative and ethical implications of choosing similar worlds.Source: §4.2 · p. 6
5 RESULTS OF OUR ANALYSIS
The authors turn their analysis into tenets for making the assumptions behind counterfactual fairness and explanations explicit and examining their implications before use in social contexts. [p37, p49]
5 RESULTS OF OUR ANALYSIS
- The authors propose tenets for surfacing hidden ontological and epistemological-semantic assumptions before designing counterfactually fair or explainable algorithms.They present this examination as a necessary step before applying such algorithms to social contexts.Source: §5 · p. 6
- Table 1 organizes the review into assumption categories, questions that make them explicit, and an example tied to a social problem.The table’s third column gives an exemplar in the context of a particular social problem.Source: §5 · pp. 6, 8, Table 1 · p. 7
- Designers should state their adopted ontology of race, justify that choice, and clarify whose expertise informs it.The paper asks whether the chosen conception reflects simplistic modeling assumptions and whether consultation respects intellectual humility.Source: §5 · p. 8
- Designers should explain their semantic approach, similarity criteria, context, and translation of social categories into variables, while assessing reliability and possible ethical harms.The authors also ask whether counterfactual truth values change across semantic approaches and whether ontological choices harm social-justice aims.Source: §5 · p. 8
- Making assumptions explicit supports interdisciplinary discussion of whether counterfactuals should be used to generate fairness judgments and explanations.The paper proposes discussion with disciplines including philosophy, social sciences, and anthropology.Source: §5 · p. 8
6 CONCLUSION
The authors offer principles for articulating contextual assumptions in counterfactual applications and fostering interdisciplinary discussion of their potential harms and ethical impacts. [p07]
6 CONCLUSION
- The paper’s principles aim to clarify contextual assumptions in counterfactual applications and support interdisciplinary discussion of their potential harms and ethical impacts.The authors frame the principles as a model for conversation among machine-learning researchers, social scientists, philosophers, and ethicists.Source: §6 · p. 8