Source-linked AI summary
European Union regulations on algorithmic decision-making and a "right to explanation"
Bryce Goodman, Seth Flaxman
TL;DR
The paper examines how the GDPR’s restrictions on significantly affecting automated decisions and its explanation requirements challenge routine machine-learning practice. It analyzes discrimination and explanation issues, showing how uncertainty bias can disadvantage underrepresented groups and arguing for more transparent and fair algorithms. The paper also emphasizes that these challenges are not a comprehensive account of all future engineering requirements.
Problem
The GDPR creates unresolved machine-learning challenges around preventing discrimination in profiling and providing meaningful information about algorithmic decision logic.
Method
The paper analyzes GDPR provisions through conceptual discussion and a synthetic loan-approval example involving representation, prediction uncertainty, and risk-averse decisions.
Results
The analysis shows that underrepresentation combined with risk-averse confidence thresholds can deny loans to a group whose true repayment probability equals that of another group.
Takeaways & Limitations
The paper highlights opportunities to design machine-learning algorithms and evaluation frameworks that are more transparent, interpretable, and fair.
Takeaways & Limitations
The paper is not a comprehensive overview of all challenges engineers may face under the GDPR, including requirements involving human intervention or user input.
Abstract
from arXiv · showhide
We summarize the potential impact that the European Union's new General Data Protection Regulation will have on the routine use of machine learning algorithms. Slated to take effect as law across the EU in 2018, it will restrict automated individual decision-making (that is, algorithms that make decisions based on user-level predictors) which "significantly affect" users. The law will also effectively create a "right to explanation," whereby a user can ask for an explanation of an algorithmic decision that was made about them. We argue that while this law will pose large challenges for industry, it highlights opportunities for computer scientists to take the lead in designing algorithms and evaluation frameworks which avoid discrimination and enable explanation.
1 Introduction
The GDPR introduces comprehensive EU-wide rules for personal-information governance and directly regulates automated individual decision-making. Its Article 22 restrictions and explanation-related requirements create significant implications for machine-learning systems.
- The GDPR is a comprehensive EU regulation governing the collection, storage, and use of personal information.
- Article 22 potentially prohibits automated decisions, including profiling, that legally or significantly affect individuals.
- The regulation could require major changes to widely used algorithms in recommendation, credit, insurance, advertising, and social-network systems.
- The GDPR’s explanation requirement makes human interpretability an important consideration in algorithm design.
2 Background
The GDPR differs materially from the former Directive in legal force, penalties, and geographic scope. Its profiling provisions also impose constraints on automated evaluation and raise practical challenges for machine-learning deployment.
- The GDPR was scheduled to replace the 1995 Data Protection Directive in April 2018 and retains restrictions on automated decision-making and a right to explanation.
- Unlike the former Directive, the GDPR applies directly across EU member states without enabling national legislation.
- The GDPR introduces EU-wide maximum penalties of 20 million euro or 4% of global revenue, whichever is greater.
- Its requirements apply globally to companies processing EU residents’ personal data, regardless of where processing occurs.
- Profiling is automated processing used to evaluate personal aspects, including economic situation, health, preferences, reliability, behavior, or location.
- Article 22 prohibits solely automated decisions that significantly affect data subjects, while allowing specified exceptions and requiring safeguards such as human intervention.
3 Non-discrimination
The paper argues that GDPR non-discrimination requirements face a choice between ineffectiveness and infeasibility. It develops uncertainty bias as an example of how representation and risk-averse decision rules can produce unequal outcomes even when groups have equal repayment probabilities.
- Non-discrimination foundations: GDPR discrimination concerns arise because profiling groups people by variables can produce decisions based on group membership.
- Interpretations of sensitive data: The regulation can be read minimally as restricting explicit sensitive variables or maximally as also restricting variables correlated with sensitive categories.
- Practical complications: Removing correlated variables may make predictors useless, while large datasets make exhaustive detection of indirect correlations increasingly difficult.
- Uncertainty bias: Uncertainty bias occurs when underrepresented groups receive less certain predictions and risk-averse algorithms favor predictions with smaller confidence intervals.
- Uncertainty bias: Under 30% non-white population share, the risk-averse loan rule denies non-white applicants credit despite equal 95% repayment probabilities.
- Uncertainty bias: Complex feature combinations and active learning can intensify representation disparities as algorithms acquire more examples from better-represented groups.
- Implications: The paper characterizes GDPR non-discrimination as ineffective under a minimal interpretation and infeasible under a maximal interpretation, motivating human-intelligible explanations.
4 Right to explanation
The GDPR’s right to explanation raises questions about what meaningful algorithmic explanation requires, especially for complex machine-learning models. The paper frames human-understandable accounts of feature influence as a central design challenge.
- Legal requirement: The GDPR distinguishes access and notification rights from safeguards for people subject to profiling, including meaningful information about the logic involved.Article 22 also mentions a right to obtain human intervention, though the intended intervention is unspecified.
- Why explanation is difficult: Machine-learning systems typically discover predictive correlations rather than provide causal explanations of why outcomes occur.Statistical variance explained by predictors does not by itself reveal why the prediction holds.
- Why explanation is difficult: Transparency barriers include corporate concealment, limited technical literacy, and a mismatch between high-dimensional optimization and human reasoning.The GDPR addresses disclosure and accessible communication more directly than the interpretability challenge posed by algorithmic selection and design.
- What explanations should provide: An adequate explanation should make the trained model understandable to humans and show how input features relate to predictions.Relevant questions include which features most influence a prediction and whether a recommendation changes across applicant characteristics.
- Model design: Model interpretability trades off against representational capacity, with linear models easier to interpret than richer nonparametric, ensemble, and neural-network models.Neural networks are presented as posing an especially difficult explanation problem.
- Research opportunities: One research avenue develops methods that quantify input-variable influence on outputs using black-box access to trained prediction algorithms.This approach addresses explanation without requiring disclosure of the model’s internal implementation.
5 Conclusion
The paper treats GDPR-related challenges as a reason to pursue algorithms that are transparent and fair, while emphasizing that its overview is not comprehensive. It concludes that socially deployed algorithms require coordination between technical and philosophical expertise.
- Scope and open questions: The paper covers non-discrimination and explanation but is not a comprehensive account of the challenges engineers may face under the GDPR.Human intervention and data-subject input may impose additional algorithm-design requirements requiring further investigation.
- Implications: Research is pursuing algorithms that support inspection and tools that identify or rectify unwanted bias, although adoption in practice remains uncertain.For certain profiling tasks, the paper says discrimination can be identified and interventions implemented to correct it.
- Implications: The GDPR acknowledges that socially deployed algorithmic decisions are rarely purely technical and require coordination between technical and philosophical resources.The paper presents ethical algorithm design as an interdisciplinary task with substantial remaining work.