Source-linked AI summary
Saving Face: Investigating the Ethical Concerns of Facial Recognition Auditing
Inioluwa Deborah Raji, Timnit Gebru, Margaret Mitchell, Joy Buolamwini, Joonseok Lee, Emily Denton
TL;DR
Facial recognition auditing can reveal biased performance while also harming the populations it aims to protect, especially when sensitive biometric data and ethically questionable uses are involved. The paper develops CelebSET, evaluates commercial facial processing APIs, and examines five ethical concerns in the audit process. It finds that audit results and practices have important limitations, including disparities that persist across tasks and tensions involving representation, privacy, and harmful applications.
Problem
Algorithmic audits of facial processing systems need to reveal biased performance without exacerbating harms involving sensitive data, marginalized groups, and ethically questionable uses.
Method
The paper develops CelebSET, evaluates commercially available facial processing APIs, and analyzes the benchmark and audit development process as a case study of ethical concerns and tensions.
Results
29.47% and 29.10% discrepancy respectively between the error rates for the best-performing and worst-performing subgroups on Microsoft and Amazon age classification.
Takeaways & Limitations
Audit design must account for privacy, representation, scope, transparency, and whether improving performance on a task could normalize harms against marginalized communities.
Takeaways & Limitations
CelebSET uses groupings with inherent limitations, including ethnicity labels that are correlated with but not deterministically linked to racial categories and exclusionary binary gender labels.
Abstract
from arXiv · showhide
Although essential to revealing biased performance, well intentioned attempts at algorithmic auditing can have effects that may harm the very populations these measures are meant to protect. This concern is even more salient while auditing biometric systems such as facial recognition, where the data is sensitive and the technology is often used in ethically questionable manners. We demonstrate a set of five ethical concerns in the particular case of auditing commercial facial processing technology, highlighting additional design considerations and ethical tensions the auditor needs to be aware of so as not exacerbate or complement the harms propagated by the audited system. We go further to provide tangible illustrations of these concerns, and conclude by reflecting on what these concerns mean for the role of the algorithmic audit and the fundamental product limitations they reveal.
1 INTRODUCTION
Facial processing technology spans detection, analysis, and identification tasks, but its deployment can enable surveillance, predatory data collection, and discrimination. This paper develops CelebSET and uses the audit process to identify ethical concerns and tensions that limit how audits should inform evaluation and policy.
- Facial processing technology includes face detection, facial analysis, and face verification or identification.These tasks range from locating faces to estimating characteristics and differentiating one face from others.
- Commercial facial processing deployments are vulnerable to abuse, particularly when used for surveillance and predatory data collection that discriminates against marginalized groups.The stakes are especially high when systems support policing or hiring decisions.
- Policy proposals have recommended using facial processing audits as conditions for accreditation or moratoria, framing them as mechanisms for certifying systems as safe and reliable.
- The paper cautions that ethical concerns in audit development and use inherently restrict audit utility within broader system evaluation.It proposes explicitly acknowledging these limitations in practice and policy.
- The authors develop CelebSET, evaluate commercially available facial processing APIs, and use the benchmark and audit development process as a case study of ethical considerations and tensions.The contributions combine benchmark construction, API evaluation, and analysis of auditing practices.
2 CELEBSET: A TYPICAL FPT BENCHMARK
CelebSET is an intersectional benchmark derived from IMDB-WIKI that evaluates commercial facial processing APIs across demographic subgroups and prediction tasks. The results show strongest performance generally on gender classification and weakest performance on age classification, with darker and female subgroups often performing worse.
- 2 CELEBSET: A TYPICAL FPT BENCHMARK: Prior benchmarks have helped identify classification bias in face detection, verification, identification, identity clustering, and gender recognition.
- 2 CELEBSET: A TYPICAL FPT BENCHMARK: CelebSET contains 80 celebrity identities divided equally among darker male, darker female, lighter male, and lighter female subgroups.The benchmark is a subset of IMDB-WIKI with ethnicity metadata gathered from celebrity fan websites.
- 2 CELEBSET: A TYPICAL FPT BENCHMARK: The audit uses uncropped images for detection and cropped face images for facial analysis tasks.
- 2 CELEBSET: A TYPICAL FPT BENCHMARK: The study evaluates Microsoft, Amazon, and Clarifai APIs on gender recognition, smile detection, name identification, age prediction, and face detection.Age prediction uses an 8-year acceptance margin, while detection uses AP50 at a 0.50 IoU threshold.
- 2.1 API Evaluation on CelebSET: All APIs generally perform best on gender classification and worst on age classification, with Amazon and Clarifai only slightly better than chance on age.
- 2.1.3 Performance on Intersectional Subgroups: Across tasks, APIs generally perform worst on darker and female subgroups, while darker females are most commonly least accurate and lighter males most commonly most accurate.These intersectional patterns include exceptions, and Clarifai shows notably higher gender-classification disparities across unitary groups.
3.1 Design Considerations
The paper identifies design and deployment considerations that constrain what facial-processing audits can reveal and how their results should be used. Audits may expose disparities while overlooking development practices, underrepresented groups, privacy and consent risks, or harms from the tasks being optimized.
- Selecting Scope of Impact: Narrow audit targets can focus remediation but also limit impact and encourage institutions to overfit improvements to audited tasks.The paper reports large disparities on age classification despite smaller gender-classification disparities for audited APIs.
- Selecting Scope of Impact: Audits focused on previously studied subgroups can ignore other marginalized populations, including transgender users affected by facial-analysis failures.The paper notes that prior audits commonly emphasize binary gender and skin type, while transgender Uber drivers were locked out of their accounts.
- Selecting Scope of Impact: Equalizing performance can normalize tasks that harm communities, such as gender classification, which may reinforce stereotypes and exclude transgender, non-binary, and gender-nonconforming people.The paper emphasizes that both correct and incorrect gender classifications can produce harm.
- Auditing for Procedural Fairness: Performance disparities do not necessarily reveal the integrity of engineering processes, including data collection, testing, documentation, and deployment guardrails.The paper describes procedural fairness as broader than interpretability or checking whether image features encode protected attributes.
- Privacy and Representation: Benchmark curation can expose sensitive biometric data and disproportionately affect marginalized communities through tokenism, exploitation, monitoring, or targeted violence.The paper gives the example of facial data harvested from millions of Zimbabwean citizens without active consent or mutual benefit.
- Privacy and Representation: CelebSET inherits demographic skew from IMDB-WIKI, making some groups and intersectional identities less represented and harder to include in a balanced benchmark.The paper specifically identifies Black women as especially starkly underrepresented and links the skew to media and social biases.
- Privacy and Representation: Increasing representation can disproportionately raise privacy risks, while public availability of images does not establish consent for inclusion in an FPT benchmark.In CelebSET, an image from the Black subgroup was twice as likely to be included as one from the White subgroup, and public figures had not specifically consented to benchmarking.
- Intersectionality and Group-Based Fairness: Balancing CelebSET across gender and coarse ethnicity left age highly unbalanced, so apparent race- or gender-related disparities may instead correlate with age.The paper describes this as fairness gerrymandering: optimizing fairness on one axis can compromise fairness on another.
4 RECONSIDERING THE ROLE OF ALGORITHMIC AUDITS
The paper argues that audits must examine their own ethical risks rather than treating benchmark results as definitive validation. Audits should expose blind spots within a broader evaluation framework and question the product’s ethical use case.
- Audits can reproduce the ethical concerns they are intended to evaluate, so auditors must apply comparable ethical standards to their own processes.
- Auditors should acknowledge evaluation limitations and interpret benchmark results as one component of a larger, more qualitative audit framework.
- The audit’s primary goal is to expose blind spots rather than validate performance.
- CelebSET audits are necessary but insufficient: they may help stall or halt deployment but cannot alone justify deployment or a moratorium.
5 CONCLUSION
Designing CelebSET exposed ethical concerns in algorithmic auditing that intersect with harms in facial processing systems. The paper argues that audits must apply the same privacy and anti-exploitation standards expected of the products they evaluate.
- Auditing facial processing systems can create a false sense of progress when unethical audit processes appear to align products with desired principles.
- Both audits and audited systems require careful privacy protections and must avoid exploiting marginalized groups while pursuing greater representation.