Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPR

I. INTRODUCTION

• The GDPR provides no legally binding right to explanation, and any such right would apply only to certain solely automated decisions with significant effects [2, 3].The stated scope is limited to negative decisions with legal or similarly significant effects.Source: §I · p. 3
• Explaining complex algorithmic systems and their case-specific rationales is technically challenging [4].Source: §I · p. 3
• Explanations may provide little meaningful information to data subjects, while controllers have an interest in withholding algorithm details [5].Source: §I · p. 3

II. COUNTERFACTUALS

• A counterfactual pairs a decision with a hypothetical change, such as a loan denial at £30,000 income becoming an offer at £45,000.It describes how the world would have to differ for a desirable outcome to occur.Source: §II · p. 5
• The “closest possible world” is the smallest change that could produce a desirable outcome, but multiple counterfactuals may lead to different desirable outcomes.Several close possible worlds may offer a more helpful range of explanations than only the closest one.Source: §II · p. 6
• The smallest change is not always the most helpful counterfactual; relevance can depend on a variable’s mutability and the real-world probability of changing it.Source: §II · p. 6
• Where typical explanations aim to convey an algorithm’s internal logic, counterfactuals describe dependencies on the external facts that led to a decision.Modern machine-learning systems may have millions of intricately connected variables, making their internal state extremely challenging to explain to laypeople.Source: §II · p. 6
• The machine-learning and legal communities have taken relatively restricted views of explanation, while machine learning has focused primarily on debugging and approximating algorithms.The passage describes those approximations as intended for programmers or researchers.Source: §II · p. 6

A. HISTORIC CONTEXT AND THE PROBLEM OF KNOWLEDGE

• Analytic philosophy has a long history of analyzing the necessary conditions for propositional knowledge.Source: A. HISTORIC CONTEXT AND THE PROBLEM OF… · p. 7

C. ADVERSARIAL PERTURBATIONS AND COUNTERFACTUAL EXPLANATIONS

• Adversarial Perturbations generate nearby synthetic inputs that change classifications (49, 50); DeepFool52 defines the smallest such image change, equivalent to a counterfactual under a suitable distance.Finding the closest possible world with a changed classification is the same as finding the smallest change to the input, given the right distance function.Source: C. ADVERSARIAL PERTURBATIONS AND… · pp. 13–14
• Differentiability makes counterfactual generation efficient, allowing optimization techniques from adversarial-perturbation research to apply even to cutting-edge architectures.The section notes that some of the largest and deepest neural networks are used for computer-vision tasks such as image labelling.Source: C. ADVERSARIAL PERTURBATIONS AND… · pp. 13–14
• Standard adversarial-perturbation methods often use unsuitable distance functions and favor small changes across many variables over sparse, human-interpretable changes (53).Source: C. ADVERSARIAL PERTURBATIONS AND… · p. 14
• Small image perturbations can be barely perceptible yet drastically change classifier responses, and may produce images outside the space of real images (54, 55).Source: C. ADVERSARIAL PERTURBATIONS AND… · p. 14
• Reliable image counterfactuals must be both close to the starting example and belong to a possible world; further research is needed to characterize high-dimensional, structured data (54, 55).Source: C. ADVERSARIAL PERTURBATIONS AND… · p. 14

D. CAUSALITY AND FAIRNESS

• Kusner et al. [58] require a decision to remain unchanged when the subject’s race or sex changes; this paper instead considers counterfactuals in which the decision changes.Source: D. CAUSALITY AND FAIRNESS · p. 15
• Individual explanations may not reveal systematic bias: understandable models can mislead intuitions, and selecting features people perceive as fair slightly increased racism while decreasing accuracy [59].Large-scale statistical analysis is considered more likely to uncover systematic biases than explanations of individual decisions [60].Source: D. CAUSALITY AND FAIRNESS · p. 15
• A counterfactual that changes a protected attribute can evidence its influence on a decision, but one that leaves it unchanged cannot establish irrelevance or prove illegal discrimination.Counterfactuals describe only some dependencies between a decision and external facts.Source: D. CAUSALITY AND FAIRNESS · pp. 15–16

• Hold model weights fixed and minimize an objective to find the nearest input whose predicted outcome equals the new target.The method iteratively solves for the counterfactual while increasing a constraint weight until a sufficiently close solution is found.Source: §III · p. 16
• Random initializations help address local minima; the best minimizer is selected, while different minima can provide diverse counterfactuals.The authors use ADAM, while noting that the choice of optimizer is relatively unimportant.Source: §III · p. 16
• A weighted L1 distance using inverse median absolute deviation is suggested as a starting choice, subject to subject- and task-specific refinement.It accommodates features that vary widely in the data, is robust to outliers, and encourages sparse changes that are easier to communicate and understand.Source: §III · p. 17
• Counterfactuals must also correspond to valid data points, with discrete features posing a challenge that requires remedies.The section indicates that varying the distance function affects LSAT counterfactuals, but supplies no specific comparison in these passages.Source: §III · pp. 17–18

A. LSAT DATASET

• The experiments use a fairness-literature LSAT dataset that predicts first-year average grades from race, pre-law-school GPA, and entrance exam scores, and a three-layer neural network with 941 weights.The dataset is used because classifiers trained on it naturally exhibit bias against black people; the authors say the network’s interdependent components make its internal state challenging to convey.Source: A. LSAT DATASET · p. 18
• Clamping race to 0 or 1 in two separate runs and selecting the closest counterfactual avoids fractional or negative values; hybrid results always change race to ‘white’ and indicate ‘black’ students would get better scores.The dataset encodes race as 0 for ‘white’ and 1 for ‘black’.Source: A. LSAT DATASET · p. 19
• With squared Euclidean distance, counterfactuals vary GPA more than exam scores because GPA has a smaller range, reflecting the distance function’s preference for small, distributed changes.The authors propose normalizing each feature by its standard deviation to adjust for the difference in ranges.Source: A. LSAT DATASET · p. 19
• After feature normalization, GPA changes become more consistent, while counterfactuals for black students still vary race, revealing dependence between the decision and a legally protected attribute.The normalized squared Euclidean results are shown in Table 2.Source: A. LSAT DATASET · p. 19
• Inverse-median-absolute-deviation-weighted L1 distance produces similar but sparser counterfactuals to weighted squared Euclidean distance, with GPA unchanged.The results are shown in Table 3.Source: Table 3 · p. 20

B. PIMA DIABETES DATABASE

• Counterfactuals for the Pima diabetes classifier identify small, readable changes to reach a 0.5 risk score.The classifier uses eight variables and a three-layer fully connected neural network with two hidden layers of 20 neurons; changes are sparsified with an L1 distance weighted by inverse median absolute deviation and constrained to training-data ranges. Examples change one or two insulin or glucose values, and explanations can combine factors to provide personalized risk information.Source: B. PIMA DIABETES DATABASE · pp. 20–21

C. CAUSAL ASSUMPTIONS AND COUNTERFACTUAL EXPLANATIONS

• The counterfactuals assume independent variables, supporting guided audits for data errors without requiring difficult causal models that may be irrelevant.Accurate causal models may still help experts choose interventions, but the paper argues they are unnecessary to make counterfactual explanations useful.Source: C. CAUSAL ASSUMPTIONS AND… · pp. 21–22

IV. ADVANTAGES OF COUNTERFACTUAL EXPLANATIONS

• Counterfactuals bypass explaining complex model internals while offering digestible, practical information to understand, challenge, and improve decisions.The paper also notes that deep networks can involve more than a thousand composed functions and more than ten million parameters, complicating meaningful explanations of internal logic.Source: §IV · p. 22

V. COUNTERFACTUAL EXPLANATIONS AND THE GDPR

• The GDPR does not explicitly define explanation requirements, and Recital 71 is its only mention of an ex post explanation, with little guidance on content.The paper reads Recital 71 as separating pre-decision specific information from post-decision safeguards, while leaving the latter’s content unclear.Source: §V · pp. 23–24

A. EXPLANATIONS TO UNDERSTAND DECISIONS

• GDPR information provisions support broad understanding of automated decision-making, but do not appear to require opening the black box of individual decisions.Counterfactuals can align with those informational requirements while offering insight into the reasons for a particular decision.Source: A. EXPLANATIONS TO UNDERSTAND DECISIONS · pp. 25–26

1. Broader possibilities with the right of access

• The right of access can be invoked at any time, unlike notification duties that apply prior to processing.The passage introduces this contrast as opening up the possibility of providing further information, but does not specify what that information is.Source: §1 · p. 32

2. Understanding through counterfactuals

• Counterfactuals use concise “if-then” statements to meet and exceed GDPR transparency, notification, and access mechanisms while giving data subjects insight into their own circumstances and decision reasons.They are less likely to expose trade secrets or others’ privacy because they need not disclose other people’s data or detailed algorithmic information.Source: §2 · p. 34
• Counterfactuals reveal some rationale for specific decisions through limited dependencies between external variables and outcomes, without explaining internal logic or disclosing source code, weights, or the full variable set.The passage says the GDPR requires some information about the “logic involved,” but opening the black box is not legally required.Source: §2 · pp. 34–35
• By communicating minimal information about the “logic involved” in specific decisions, counterfactuals go beyond explicit GDPR access requirements while avoiding the need to solve interpretability challenges for non-experts.This approach minimizes regulatory burden for controllers.Source: §2 · p. 35

B. EXPLANATIONS TO CONTEST DECISIONS

• Article 22(3) safeguards a right to contest automated decisions, and explanations may help individuals assess the reasons for an adverse or undesired outcome.Contesting may seek to reverse or nullify a decision or obtain an alternative decision.Source: B. EXPLANATIONS TO CONTEST DECISIONS · p. 35
• Four contesting models remain possible: a human may decide anew, assess algorithmic input, monitor an algorithmic decision, or allow a new automated decision that can be contested again.The first two human decision models would no longer be solely automated and would not retain Article 22(3) safeguards; the other models can retain them, though implementation remains unclear.Source: B. EXPLANATIONS TO CONTEST DECISIONS · pp. 35–36
• A rationale explanation may identify grounds such as inaccurate input data, problematic inferences, or flaws in algorithmic reasoning when algorithmic involvement remains.If a human instead disregards the algorithm and decides anew, the original rationale may inform the individual but will not practically affect that decision.Source: B. EXPLANATIONS TO CONTEST DECISIONS · pp. 36–37
• The GDPR does not explicitly link explanation to contesting: Recital 71 specifies neither an aim nor required information, and does not require disclosure of internal logic.Explanations may help contesting, but are not a necessary precondition; Article 22(3) safeguards need not be exercised together.Source: B. EXPLANATIONS TO CONTEST DECISIONS · p. 37
• Transparency, notification, and access mechanisms are not explicitly linked to contesting, so the information they provide need not be tailored to help individuals contest decisions.Articles 12–15 can nevertheless assist by supporting the exercise of GDPR rights and increasing individual control over personal-data processing.Source: B. EXPLANATIONS TO CONTEST DECISIONS · p. 37
• Articles 13–15 provide a general overview of automated decision-making, not individual decision rationales, and do not require informing individuals about Article 22(3) safeguards or a right to contest.Their references to Article 22(1) and (4), rather than Article 22(3), indicate that this information is not designed as individual-level support for contesting.Source: B. EXPLANATIONS TO CONTEST DECISIONS · p. 38

1. Contesting through counterfactuals

• Arts. 13–15 do little to facilitate contesting automated decisions: they need not disclose safeguards or the internal logic that could help a data subject challenge an outcome.The passage argues that data subjects should be informed about the right not to be subject to an automated decision and its safeguards.Source: §1 · pp. 40–41
• Counterfactuals reveal external factors and key variables behind a decision, helping data subjects contest outcomes without understanding the system’s internal logic.For example, learning that low income led to a loan denial could support a challenge based on inaccurate or incomplete financial data.Source: §1 · p. 41
• Counterfactuals compactly show which data influenced a decision, reducing the burden of checking a large corpus for inaccurate data when seeking correction and contesting the outcome.Although Art. 16 provides a right to correct inaccurate data, data subjects need not be told which data affected the decision.Source: §1 · p. 41

C. EXPLANATIONS TO ALTER FUTURE DECISIONS

• When an accurate decision is unfavorable and contesting it is unlikely to succeed or too costly, a data subject may prefer to change her situation and seek a new decision.This future-oriented use of explanations is distinct from contesting a decision.Source: C. EXPLANATIONS TO ALTER FUTURE… · p. 42
• The GDPR does not directly address using explanations to guide behavior, but counterfactuals can indicate whether a change, such as an immediate pay rise, could make reapplying for a loan reasonable.The Article 29 Working Party also noted that tips for improving habits and lowering insurance premiums could be useful to data subjects (201). Technical information about logic, significance, and consequences is not guaranteed to help in this situation.Source: C. EXPLANATIONS TO ALTER FUTURE… · p. 42
• By identifying influential variables and close possible worlds with different outcomes, counterfactuals can guide behavior when future decisions use a low-variability or artificially frozen model.Controllers could also contractually promise the preferred outcome if the counterfactual’s terms are met within a specified period.Source: C. EXPLANATIONS TO ALTER FUTURE… · p. 42
• Unanticipated dependencies between changed attributes and other variables can weaken counterfactuals as behavioral guidance, although they can address simultaneous changes to multiple variables.Even when guidance is not useful, counterfactuals can still help explain which data and variables influenced the prior decision.Source: C. EXPLANATIONS TO ALTER FUTURE… · p. 43

CONCLUSION

• Counterfactual explanations identify external facts that could change to produce a desired outcome, without explaining a black-box system’s internal logic; they are efficiently computable for many standard classifiers.The authors ground this lightweight form of explanation in philosophical literature and early AI work.Source: CONCLUSION · p. 43
• The GDPR provides little support for understanding, contesting, or changing automated decisions; Recital 71 does not specify explanations’ purpose or content, and explanations may be voluntary and post-decision.The paper finds no clear requirement to open the black box or to provide an explanation as a precondition for contesting a decision.Source: CONCLUSION · p. 43
• GDPR notification and access rights offer limited help with specific decision rationales or contesting outcomes, and technical explanations of internal logic are not legally mandated.Generic information under Arts. 13–14 is provided before a decision; Art. 15 offers similar information, while others’ rights can limit access.Source: CONCLUSION · p. 44
• The authors recommend unconditional counterfactual explanations on request, regardless of outcome or whether a decision meets GDPR automation and significant-effect thresholds, whose scope and exemptions complicate safeguards.They present this as an initial step balancing transparency and accountability with business burdens and others’ privacy, and potentially supporting public acceptance.Source: CONCLUSION · pp. 44, 45, 47
• Implementation could compute diverse counterfactuals at decision time or preserve the historical model for later requests, with automated disclosure or auditing APIs recommended within a reasonable time.Later computation requires access to the model used for the decision; preserving audit logs can record its state.Source: CONCLUSION · pp. 45–46
• Counterfactuals can explain a decision’s basis, help challenge inaccurate input data, and offer limited guidance on changes that could produce a desired future result.They avoid disclosing internal system logic beyond limited dependencies and are less likely than full disclosure to expose privacy, trade secrets, or opportunities to game a system, given reasonable limits on their number.Source: CONCLUSION · p. 46
• Counterfactuals may be insufficient when system functionality or decision rationale must be understood, and they do not provide statistical evidence to assess fairness or racial bias.The authors call for further work on distance metrics and suitable sets of counterfactuals across sectors, alongside broader interpretability to support accountability and validation of fairness and functionality.Source: CONCLUSION · pp. 45–47

APPENDIX 1: SIMPLE LOCAL MODELS AS EXPLANATIONS

• Local approximations make a three-way trade-off among approximation quality, ease of understanding, and the size of the domain they describe.The passage identifies approaches such as LIME214 as making this trade-off.Source: APPENDIX 1: SIMPLE LOCAL MODELS AS… · p. 48
• The same score curve can yield opposing local explanations as the input range changes, making the most helpful scale nontrivial to choose.The direction and magnitude of the linear approximation vary with the domain; Figure 1 illustrates this range dependence.Source: APPENDIX 1: SIMPLE LOCAL MODELS AS… · pp. 48–49, Figure 1 · p. 49
• For a desired score of -10 or below, local approximations give misleading or unusable guidance: predictions include an actual score of 91.5 and a local maximum.The top-left approximation predicts -10 at x = -2.5, while the bottom-left suggests it near 0.9; the centre approximations suggest only score 0 is possible.Source: APPENDIX 1: SIMPLE LOCAL MODELS AS… · p. 49
• For the question “Why was the score not below -10?”, the counterfactual answer is that the x value was not 2.15.The passage illustrates the counterfactual with a green dot in Figure 2.Source: APPENDIX 1: SIMPLE LOCAL MODELS AS… · pp. 49–50
• Unlike local models, counterfactuals can indicate whether a pay rise makes reapplying for a loan worthwhile.The approaches are generally incomparable: counterfactuals do not provide the local linear approximation a person might want about their data point.Source: APPENDIX 1: SIMPLE LOCAL MODELS AS… · p. 50

APPENDIX 2: EXAMPLE TRANSPARENCY INFOGRAPHIC

• Proposed European Parliament icons were not adopted as a standard, but their simplicity suggests Art. 13–14 information targets a broad audience and need not be individualized.The passage contrasts this with a ‘de facto’ right to explanation comparable to Recital 71.Source: APPENDIX 2: EXAMPLE TRANSPARENCY… · p. 51
• Although the icons were rejected, Art. 12(8) tasks the EC with developing standardized icons, indicating icons are considered acceptable for conveying Art. 13–14 information.Source: APPENDIX 2: EXAMPLE TRANSPARENCY… · p. 51