PaperlayerView exact revision on arXiv

Source-linked AI summary

Hard Choices in Artificial Intelligence: Addressing Normative Uncertainty through Sociotechnical Commitments

Roel Dobbe, Thomas Krendl Gilbert, Yonatan Mintz

arXiv:1911.09005v1cs.AIcs.CYcs.SYeess.SY

TL;DR

AI safety debates face unsettled definitions and uncertainty about harms to stakeholders in complex social contexts. The paper analyzes safety dilemmas across AI development through intuitive comparability and proposes sociotechnical commitments; it concludes that informed dissent and deliberation should be cultivated throughout deployment.

  • Problem

    Definitions of AI safety and the normative foundations for addressing stakeholder harms remain unsettled, including how safety should be defined across development stages.

    Source: Introduction · p. 1, Prominent Approaches to Vagueness · p. 8

  • Method

    The paper applies intuitive comparability to dilemmas across design, training, and deployment, using the Rekognition case to examine sociotechnical commitments.

    Source: Resolving Vagueness via Hard Choices · p. 3, Sociotechnical Commitments in… · p. 4

  • Results

    The paper outlines commitments addressing formal, substantive, and discursive challenges, including channels for dissent before, during, and after value commitments.

    Source: Introduction · p. 2

  • Takeaways & Limitations

    The authors argue that engineers, data scientists, designers, and the public should be prepared to deliberate about system assumptions, abilities, and limitations as systems are deployed.

    Source: Conclusion · p. 6

    The Rekognition confidence-threshold metric may be insufficient to validate consensus among relevant stakeholders, and some thresholds appear arbitrary.

    Source: Training · p. 6

Abstract

from arXiv · show

As AI systems become prevalent in high stakes domains such as surveillance and healthcare, researchers now examine how to design and implement them in a safe manner. However, the potential harms caused by systems to stakeholders in complex social contexts and how to address these remains unclear. In this paper, we explain the inherent normative uncertainty in debates about the safety of AI systems. We then address this as a problem of vagueness by examining its place in the design, training, and deployment stages of AI system development. We adopt Ruth Chang's theory of intuitive comparability to illustrate the dilemmas that manifest at each stage. We then discuss how stakeholders can navigate these dilemmas by incorporating distinct forms of dissent into the development pipeline, drawing on Elizabeth Anderson's work on the epistemic powers of democratic institutions. We outline a framework of sociotechnical commitments to formal, substantive and discursive challenges that address normative uncertainty across stakeholders, and propose the cultivation of related virtues by those responsible for development.

Introduction

AI safety has become more consequential while its core definitions and governance criteria remain unsettled. The paper treats safety as sociotechnical and proposes commitments for deliberating normative issues across development.

Introduction

  • AI safety has grown more urgent as systems enter critical infrastructure and social domains, while definitions of safety and appropriate governance remain unsettled.The paper defines AI safety broadly as designing independently acting machines to avoid harm to people and the environment.Source: Introduction · p. 1
  • Technical approaches to formalizing safety and fairness face deployment uncertainty and externalities, leaving criteria for evaluating and resolving harms vague.The paper notes that formalizing fairness can involve traps of abstraction and trade-offs requiring moral deliberation.Source: Introduction · p. 1
  • The authors frame AI systems as sociotechnical and use a safety case study, responses to uncertainty, vagueness, and intuitive comparability to develop their proposal.Their commitments address formal, substantive, and discursive challenges and establish channels for dissent before, during, and after value commitments.Source: Introduction · pp. 1–2
  • The paper’s core contribution is to treat technological systems as political and to call for normative deliberation and ongoing citizen participation.The stated aim is to ensure safety for stakeholders affected by a system’s actions.Source: Introduction · p. 2

The Vagueness of Safety - ACLU vs. AWS

The Rekognition dispute illustrates how safety judgments depend on contested definitions of protection, robustness, and resiliency. The competing accounts expose disagreements over evidence, governance, responsibility, and acceptable uses.

The Vagueness of Safety - ACLU vs. AWS

  • Safety is interpreted through protection from harm, robustness under adverse conditions, and resiliency under stress, but stakeholder-specific meanings make deliberation difficult.Source: The Vagueness of Safety - ACLU vs. AWS · p. 2
  • 40% of members of Congress falsely matched by the ACLU’s Rekognition test were people of color; AWS said the test used a below-recommended threshold and an insufficiently documented dataset.The ACLU used publicly available arrest photos, and AWS argued the system was used outside its design specifications.Source: The Vagueness of Safety - ACLU vs. AWS · p. 2
  • The protection dispute pits Amazon’s private-contract validation against the ACLU’s demand for public assurances that shield vulnerable communities from harm.The text says the parties’ practical definitions of protection arise in orthogonal legal contexts and at different points in the development pipeline.Source: The Vagueness of Safety - ACLU vs. AWS · p. 2
  • Robustness lacks agreed criteria and authority: responsibility may fall on public officials preparing cities for the tools or engineers refining the tools for cities.The paper calls this allocation of moral and legal responsibility the “moral crumple zone”; it says liability mechanisms cannot be justified until governance notions are clarified.Source: The Vagueness of Safety - ACLU vs. AWS · pp. 2–3
  • Resiliency depends on a contested optimality metric and on whether facial-recognition dynamics and uses are judged legitimate against broader social risks and benefits.The paper connects this judgment to deliberation about what kind of society is wanted.Source: The Vagueness of Safety - ACLU vs. AWS · p. 3

Resolving Vagueness via Hard Choices

The paper uses intuitive comparability to frame AI safety decisions as hard choices among values that may resist direct measurement. It argues that stakeholder conflicts and moral overload call for deliberation that preserves shared agency.

Resolving Vagueness via Hard Choices

  • At deployment scales, the meanings of privacy, security, and social choice can become unclear as the stakes of value commitments are difficult to determine.Source: Resolving Vagueness via Hard Choices · p. 3
  • Intuitive comparability treats qualitatively different values as practically comparable, allowing informed tradeoffs even when values are incommensurable.Hard choices arise when alternatives are on a par; deliberators can create will-based reasons by standing behind a feature of one option.Source: Resolving Vagueness via Hard Choices · p. 3

The Case for Intuitive Comparability in AI Safety

  • The authors use intuitive comparability as a lens for identifying AI safety dilemmas, where quantitative thresholds alone cannot settle hard choices among stakeholder communities.They propose iteratively redrawing formal boundaries and design parameters through qualitative feedback, specifying new criteria when communities clash.Source: The Case for Intuitive Comparability… · p. 3
  • AI hard choices involve encoding divergent stakeholder values and addressing moral overload when complex needs resist solutions within the traditional design space.The authors extend the argument that reward-function misspecification must be addressed in relation to wider social institutions.Source: The Case for Intuitive Comparability… · p. 4
  • Developers and governors must mediate stakeholder value conflicts through quantitative and qualitative evaluation while safeguarding pluralist value hierarchies and shared moral agency.The paper identifies shared agency with the ability to engage in design, training, and deployment.Source: The Case for Intuitive Comparability… · p. 4

Sociotechnical Commitments in Developing AI

The framework addresses vagueness across AI development through formal, substantive, and discursive commitments, illustrated across design, training, and deployment in the AWS–ACLU case. It calls for stakeholder dissent and context-sensitive choices throughout that pipeline.

Sociotechnical Commitments in Developing AI

  • Formal, substantive, and discursive commitments address unavoidable tradeoffs, broaden the design space for plural values, and require stakeholder communication; the framework is illustrated through the AWS–ACLU case.The authors also emphasize tracking dissent during design, testing, and implementation.Source: Sociotechnical Commitments in… · p. 4

Design

  • Designers must negotiate what can be modeled, account for context and potential externalities, and leave non-design viable when dissent persists.The Rekognition example contrasts hardcoded use limits with maximizing predictive accuracy; design constraints can also exclude stakeholders’ values.Source: Design · p. 5

Training

  • Training requires balancing verification with validation and building team consensus on testing; accuracy in well-specified settings can leave systems brittle in poorly defined ones.AWS’s confidence-threshold slider and documentation may support verification, but the passages question whether they establish consensus among stakeholders for validation.Source: Training · pp. 5–6

Deployment

  • Deployment commitments must preserve user exit and voice, with public accountability; whether agreements are treated as private contracts or public assurances implies different regulatory approaches.The framework calls for an open feedback channel and a public commitment that makes it trustworthy.Source: Deployment · p. 6

Conclusion

The conclusion calls for proactive dissent channels and sociotechnical commitments in AI development. It presents democratic dissent and public understanding as necessary to preserve users’ ability to shape their values as systems are deployed.

Conclusion

  • Developers should engage distinct dissent channels proactively, before AI risks become technically or politically insurmountable.The authors anticipate integrating related commitments into the training of engineers, data scientists, and designers.Source: Conclusion · p. 6
  • Public education about systems’ assumptions, abilities, and limitations is needed to make informed dissent attainable during deployment.The authors state that deliberation is the goal of AI Safety, not the procedure by which it is ensured.Source: Conclusion · p. 6
  • The authors argue that democratic dissent is necessary to preserve intuitive comparability and let users define the contours of their own values.They ground this claim in value hierarchies described as computationally underdetermined, semantically indeterminate, and politically obfuscated.Source: Conclusion · p. 6

Prominent Approaches to Vagueness

The paper contrasts approaches that resolve normative uncertainty by ranking values, treating them as indeterminate, or accepting pluralism. It presents intuitive comparability as a way to reason through hard choices without requiring directly measurable values.

Epistemicism

  • Epistemicism requires ranking values; metanormativism seeks second-order norms for choices among moral doctrines, but still requires a clear positive relation between actions.The paper also describes proposals to learn values or replicate external normative orders, while noting that safety definitions remain unclear across the development pipeline.Source: Prominent Approaches to Vagueness · p. 8

Semantic Indeterminacy

  • Semantic indeterminacy treats value relations as fundamentally ambiguous; the paper warns that arbitrary design choices can neglect value hierarchies and moralize existing inequalities.The discussion connects this concern to public-interest costs of society-wide technological innovations.Source: Prominent Approaches to Vagueness · pp. 8–9

Value Pluralism

  • Value pluralism rejects an ultimate ordering of values and highlights how formal specifications can exclude groups, while agonistic approaches foreground conflict and dissent in design.The paper also notes critiques that fairness approaches may leave the normative conditions favoring advantaged subjects unexamined.Source: Prominent Approaches to Vagueness · p. 9

Intuitive Comparability

  • Intuitive comparability holds that people can articulate evaluative differences between values that are not directly measurable, and that agency can create reasons when alternatives are on a par.The authors present it as an intermediary lens for mediating different definitions of safety across the development pipeline, not as a superior position.Source: Prominent Approaches to Vagueness · p. 9
Loading 1911.09005v1…