Hard Choices in Artificial Intelligence

1 Introduction

• Normative standards and technical approaches leave unresolved how to reconcile diverse user needs and systematically prioritize safety or fairness criteria.The paper identifies a missing sustained inquiry into how systems restructure values and practices, how governance and technical criteria meet in design, and where gaps emerge across development.Source: §1 · p. 2
• Existing safety criteria lack unifying categories for determining whether a system’s capabilities are safe.Proposals often treat safety as a technical design criterion, an operational condition, or an end-user experience.Source: §1 · p. 2
• The authors propose reconceiving AI development around encounters between system capabilities and sociotechnical gaps, using a new vocabulary and framework.They also call for communicative channels that let stakeholders help shape design criteria for managing those gaps.Source: §1 · pp. 2–3
• Hard Choices in Artificial Intelligence maps possible gaps to feedback channels for designers and stakeholders.The paper also examines existing normative and technical responses as piecemeal treatments of vagueness and discusses implications for evaluating system performance and governance standards.Source: §1 · p. 3
• The paper connects its approach to situated and critical technical practice, arguing that computation should support human values and concerns.It characterizes technological systems as political and calls for normative deliberation and ongoing collective participation by those affected.Source: §1 · p. 3

2 Towards a Sociotechnical Lexicon for AI

• The preliminary lexicon maps overlap and divergence between AI Safety, cybernetics, and governance, enabling comparison of current development assumptions with alternative paradigms.The paper frames this lexicon as a response to the lack of a robust sociotechnical vocabulary for AI.Source: §2 · p. 3
• A gap separates value alignment from managing deployed systems’ consequences, because values such as safety and fairness emerge from interactions among technical and human components, processes, and infrastructure.This system-level view treats values as emergent properties rather than outcomes guaranteed by a precise specification.Source: §2 · p. 5
• Cybernetic feedback treats norms as enacted and elaborated through outcomes, implying indeterminacy that cannot be resolved solely by learning a reward function for human preferences.This contrasts feedback-based adjustment with passively learning norms represented as preferences.Source: §2 · p. 6
• AI governance research points to institutions and domain expertise for articulating metrics, standards, and acceptable performance thresholds for particular systems.Proposed approaches include governance coordination committees and oversight bodies.Source: §2 · p. 6
• The lexicon defines sociotechnics as the relationship between a system and real-world conditions, specified through active engagement with stakeholder concerns.It also defines an AI system lifecycle spanning design, verification and validation, deployment, and operation and monitoring.Source: §2 · pp. 4, 7

• The lexicon treats sociotechnical specification as system normativity across featurization, optimization, and integration, interpreted against situated norms and real-world performance thresholds.Source: §2 · p. 7
• Hard choices arise when AI-development values are at parity, requiring normative deliberation to make options technically commensurable; stakeholder weighing is inherently political.Source: §2 · p. 8
• Developers should build diagnostic practices and commitments into development, anticipating choices through stakeholder feedback rather than deciding on stakeholders’ behalf; dissent can prompt amended specifications and proportional democratic governance.Source: §2 · pp. 8–9
• AI governance and Safety need sociotechnical reframing of classic AI problems, shared language for indeterminacy across intended and actual behavior and stakeholder communities, and feedback modalities to address operational indeterminacy.Source: §2 · p. 9

3 The Problem of Vagueness

• Vagueness names cases where technical uncertainty fails to capture specific indeterminacies, and the section maps three approaches to corresponding standards and prioritized feedback.The approaches are epistemicism, ontic incomparabilism, and semantic indeterminacy; their associated standards are metanormativism, value pluralism, and fuzziness, with preference learning, refusal, and equitable outcomes as feedback forms.Source: §3 · p. 10

3.1 Epistemicism - resolving vagueness through model uncertainty

• Epistemicism treats unclear boundaries as discoverable facts, but preference learning is inadequate when human behavior is difficult to observe.Behavioral signals may be sparse, or observation may raise concerns about surveillance or behavioral manipulation.Source: §3.1 · pp. 10–12

3.2 Ontic incomparabilism - respecting value pluralism

• Ontic incomparabilism holds that no amount of information can yield an objective basis for choosing among definitions, so finite representations cannot exhaust the world’s richness.Its expression in value pluralism rejects an ultimate scheme for delineating human values because people cannot be exhaustively represented.Source: §3.2 · p. 12
• System design makes political choices among stakeholder values, producing hierarchies that can distribute benefits and harms unequally.Refusal can reject an unsuitable specification, but the text leaves open whether refusal alone can articulate a more just and equitable society without alternative feedback.Source: §3.2 · p. 13

3.3 Semantic indeterminism - declaring things fuzzy by nature

• The conflict between eliminating bias and winning social trust reveals inconsistent determinations of safety across the lifecycle, including which norms should guide design and use.Source: §3.3 · p. 14
• Addressing semantic differences would require feedback such as preference learning and refusal, but semantic indeterminism defers rather than resolves normative indeterminacies.These indeterminacies are left for designers or stakeholders to handle.Source: §3.3 · p. 14

4 A Framework of Commitments for AI Development

• HCAI frames development as a dynamic, reflexive process of sociotechnical specification, featurization, optimization, and integration, with stakeholder feedback and democratic dissent shaping governance.The framework is a conceptual depiction of deliberating about normative indeterminacy, not a prescription of particular law or policy interpretations.Source: §4 · pp. 15–16
• The framework is conceptual rather than a fixed policy prescription; regulatory measures may constrain development or be informed by it, while legal and design choices remain contextual.The authors describe translation into particular laws or policies as a natural extension, not an interpretation advocated by this framework.Source: §4 · p. 16

4.1 Sociotechnical Commitments

• Developers should diagnose normative indeterminacy beyond technical logic, attend to social, affective, corporeal, and political factors, and adapt specifications through practical judgment responsive to context.Specifications and safety thresholds may need to differ across contexts and applications.Source: §4.1 · p. 17
• Stakeholder feedback channels and dissent mechanisms should surface the value hierarchies and parity of design options, enabling communities to reimagine moral boundaries.These channels make decision-making deliberative rather than simply imposed.Source: §4.1 · p. 17

4.2 Sociotechnical Specification (engaging the “stakes” and forms of agency)

• Sociotechnical specification clarifies whose agency a system serves, who administers it, and how safety is defined, formalized, and enacted.It facilitates different interests relevant to a situation that may benefit from technological intervention.Source: §4.2 · p. 18
• The specification hard choice is to include sufficient perspectives and distribute decision-making power broadly enough for trust and legitimate consensus, while keeping requirements and responsibilities feasible.The framework also proposes diagnostic and procedural questions for AI applications broadly, focusing here on safety-critical or public-infrastructure contexts.Source: §4.2 · p. 19
• Solidarity means creating channels for stakeholders to actively determine system specifications, resolving operationally blocking indeterminacies in advance and deferring others for stakeholder judgment.The three forms of hard choice are situational alerts throughout iterative development, not a linear checklist.Source: §4.2 · p. 19

4.3 Featurization (epistemic uncertainty)

• Unanticipated model and data choices can deny stakeholders a chance to evaluate alternatives or force harmful choices, so they should negotiate what can be modeled and validate designs against context.This includes challenging modeling assumptions, considering value conflicts between model and world, and being prepared to revisit tools and methodology.Source: §4.3 · p. 20
• The criterion for when a model must remain technically ignorant or intentionally suboptimal should be grounded in stakeholders’ self-determination.Modeling constraints must also balance representing stakeholder interests with tractability, performance, and privacy, while recognizing that technical bias can limit whose values are expressed.Source: §4.3 · pp. 20–21
• Featurization should anticipate deployment, misuse, training bias, effects on vulnerable groups, and externalities; if dissent persists without consensus, not designing the system should remain an option.Model parameters should be both computationally tractable and normatively defensible, although limited resources and stakeholder interests can lead to under- or mis-specification.Source: §4.3 · pp. 20–21

4.4 Optimization (semantic indeterminacy)

• Optimization objectives, constraints, and parameters intervene in how identified objects relate to meanings active in stakeholders’ behavior.Developers should assess whether optimization translates and respects specifications, establish empirical validation procedures responsive to stakeholder concerns, and revise verification and validation strategies as indeterminacies emerge.Source: §4.4 · pp. 21–22
• Safety assessment requires verification that the system meets specifications and validation that it performs safely and reliably in its deployment context.These assessments cover both the artifact and its integration, including whether prospective users’ needs are met and how it interacts with systems and people.Source: §4.4 · p. 22
• Laboratory optimization cannot fully capture integration context, and simulation can overfit features that do not reflect real-world interactions.Interactions with other systems and human agents also complicate optimization.Source: §4.4 · p. 22
• Insufficient validation and safeguards can produce disparate impacts, especially for underrepresented groups, and these concerns intensify with online learning.The text notes that underrepresented groups may be poorly represented on design teams.Source: §4.4 · p. 22
• Optimization stewardship should prioritize qualitative assurances for domain stakes and allocate limited team resources to stakeholder concerns rather than abstract accuracy or efficiency.Quality management must also adjudicate how potentially plural values are operationalized without compromising parity.Source: §4.4 · pp. 22–23

4.5 Integration (ontic incomparabilism)

• Integration must address new harms, varied stakeholder expectations and jurisdictions, and multiple value priorities while recognizing affected stakeholders’ agency.The section frames integration as a challenge of handling diverse objectives and values, including those of subpopulations outside the intended user base.Source: §4.5 · pp. 23–24
• Developers and managers should specify mechanisms to identify, contest, and mitigate risks across affected communities, with responsibilities and safeguards for accidents.These conditions include technical mechanisms as well as processes, organizational measures, responsibilities, and cultural norms.Source: §4.5 · p. 23
• Trustworthy integration requires meaningful exit and voice, open feedback channels, and public updates that support stakeholders’ ability to express concerns or dissent.The section asks whether stakeholders can withdraw and whether they can regularly propose concerns and be kept informed.Source: §4.5 · p. 23

5 Implications and Discussion

• HCAI’s practical recommendations go beyond governance and performance standards, encouraging policymakers, designers, and STS scholars to coordinate development decisions.The paper identifies opportunities for these groups to learn from one another’s insights.Source: §5 · p. 24

Confront the choices and assumptions behind the AI system

• Formal abstractions can decontextualize and depoliticize problems, so modeling choices require an account of their normative assumptions and political boundaries.The paper describes capturing everything in an AI model and objective function as a political choice.Source: §5 · p. 25, Confront the choices and assumptions… · p. 25

5 Implications and Discussion

• High-stakes AI needs a sociotechnical lens spanning human, organizational, and cultural contexts; technical vulnerabilities alone cannot secure systems.The paper presents this broader systems perspective as a more comprehensive starting point for safety.Source: §5 · p. 25

Confront the choices and assumptions behind the AI system

• Actors may bring different lived realities and conflicting interests, making a multi-actor approach relevant to development and governance.The paper notes that how closely actor and system perspectives should be integrated is itself normatively indeterminate.Source: Confront the choices and assumptions… · p. 26
• Addressing normative indeterminacy requires feedback at system and actor levels, iterative reconsideration of stakes, and a reciprocal relationship between development and governance.The paper presents this feedback as a way to revisit the stakes and consensus reached among actors.Source: Confront the choices and assumptions… · p. 26
• Safety-critical contexts require an honest account of problem and solution spaces that reflects different actors’ perspectives.Such accounts provide a basis for seeking broad consensus.Source: Confront the choices and assumptions… · p. 26

6 Conclusion

• AI practitioners’ technical work is necessary but insufficient; stakeholders need to participate actively through tools and techniques suited to their situations.The conclusion calls for a critical practice that includes the forms of feedback required by the application domain.Source: §6 · p. 27
• The framework is a preliminary account of what cybernetic practices might look like in particular development domains; the authors leave that work for the future.The authors describe pursuing this effort in future work.Source: §6 · p. 28
• Clarifying sociotechnical safety requirements is presented as groundwork for proactive dissent channels before AI risks become technically or politically insurmountable.The conclusion anticipates cybernetic practices in training engineers, data scientists, and designers.Source: §6 · p. 28
• The conclusion makes deliberation the goal of AI Safety and argues that democratic dissent is needed to safeguard parity and let users define their values.It also calls for the public to understand systems’ assumptions, abilities, and limitations so informed dissent is attainable during deployment.Source: §6 · p. 28