Source-linked AI summary
Society-in-the-Loop: Programming the Algorithmic Social Contract
Iyad Rahwan
TL;DR
Rapid AI advances have created governance questions about transparency, fairness, and accountability. The paper proposes society-in-the-loop, a framework that combines human oversight with an algorithmic social contract. It concludes by calling for institutions and tools to program, debug, and monitor that contract.
Problem
AI systems increasingly govern important aspects of life, raising concerns about accountability, transparency, fairness, and societal oversight.
Method
The paper synthesizes human-in-the-loop control with social-contract theory into a society-in-the-loop framework for algorithmic governance.
Results
The paper proposes an algorithmic social contract between stakeholders, mediated by machines, with society involved in oversight and compliance monitoring.
Takeaways & Limitations
The paper calls for institutions and tools that put society in the loop and enable programming, debugging, and monitoring of algorithmic social contracts.
Takeaways & Limitations
HITL does not sufficiently emphasize society-wide oversight, while societal values may be difficult to articulate operationally and tradeoffs difficult to quantify.
Abstract
from arXiv · showhide
Recent rapid advances in Artificial Intelligence (AI) and Machine Learning have raised many questions about the regulatory and governance mechanisms for autonomous machines. Many commentators, scholars, and policy-makers now call for ensuring that algorithms governing our lives are transparent, fair, and accountable. Here, I propose a conceptual framework for the regulation of AI and algorithmic systems. I argue that we need tools to program, debug and maintain an algorithmic social contract, a pact between various human stakeholders, mediated by machines. To achieve this, we can adapt the concept of human-in-the-loop (HITL) from the fields of modeling and simulation, and interactive machine learning. In particular, I propose an agenda I call society-in-the-loop (SITL), which combines the HITL control paradigm with mechanisms for negotiating the values of various stakeholders affected by AI systems, and monitoring compliance with the agreement. In short, `SITL = HITL + Social Contract.'
1 Introduction
Recent AI advances are proliferating in everyday life and delivering societal benefits, while raising concerns about transparency, accountability, filter bubbles, and bias. The paper proposes society-in-the-loop as a conceptual framework for regulating AI and data-driven systems through an algorithmic social contract.
- AI advances: Recent theoretical and practical advances have accelerated AI’s proliferation in everyday life.Examples include deep neural networks, reinforcement learning from evaluative feedback, and economic reasoning in multi-agent systems.
- Societal benefits: AI technologies are producing benefits in supply chains, matchmaking, medical diagnosis, and drug discovery.
- Governance concerns: Algorithmic systems raise governance concerns because they may be opaque, create filter bubbles, or perpetuate bias and injustice.
- Framework: The paper proposes an algorithmic social contract between stakeholders, mediated by machines, using society-in-the-loop oversight.SITL adapts human-in-the-loop approaches but extends oversight to society as a whole.
2 Human-in-the-Loop
Human-in-the-loop systems place people within automated processes to supervise, correct, optimize, and maintain them. The paper argues that expert or individual oversight does not sufficiently address regulation when society as a whole is affected.
- HITL foundations: HITL makes a human operator a crucial component of automated control, handling supervision, exception control, optimization, and maintenance.
- HITL machine learning: HITL machine learning includes human data labeling and interactive feedback that can improve learning and system performance.Applications include benchmarks for computer vision and systems that learn from user behavior.
- Applications: HITL has been applied to crisis counseling and human-robot interaction through real-time visualization, adjustable autonomy, interactive teaching, and flexible teams.
- Regulation: Human oversight can identify misbehavior, take corrective action, and provide an accountable entity when an AI system misbehaves.
- Limits: The paper argues that HITL does not sufficiently emphasize society-wide oversight and may wrongly suggest that expert oversight solves regulation.
3 Society-in-the-Loop
Society-in-the-loop is proposed for AI systems with broad societal implications, where both inputs and outputs have wide scope. It embeds societal values in algorithmic governance and requires balancing competing stakeholder interests.
- Scope: SITL addresses AI systems serving broad functions with wide societal implications, such as self-driving cars, news filtering, and economic allocation.
- Conceptual shift: SITL is presented as a qualitative shift from individual or expert oversight toward society-wide oversight for broadly scoped AI systems.
- Societal values: SITL embeds society’s values in the algorithmic governance of societal outcomes with broad implications.
- Social contract: The shift from HITL to SITL creates a fundamentally different problem: balancing competing interests among stakeholders, including algorithmic governors.The paper connects this balancing problem to defining a social contract.
4 Detour: The Social Contract
Social contract theory explains legitimate governance as a mechanism for coordinating strangers through mutual consent, enforcement, and accountability. Its evolution aimed to preserve cooperation while constraining sovereign power through the general will and fundamental rights.
- Complex social institutions emerged because kin selection and reciprocal altruism could not adequately coordinate increasingly large groups.
- Social contract theory holds that centralized government legitimates its power by enabling cooperation among strangers through third-party enforcement.
- The social contract evolved through political thinkers who refined how it emerges and how it can be prevented from collapsing.
- Modern political institutions combine institutional innovation with learning, allowing societies to borrow governance ideas and institutions from one another.
- A mature social contract combines sovereign efficiency and stability with implementation of the people’s general will and accountability for violations of fundamental rights.
5 The Algorithmic Social Contract
The algorithmic social contract extends human-in-the-loop oversight into society-in-the-loop governance for AI systems with broad societal impact. It requires collective agreement on values, stakeholder tradeoffs, and channels connecting public expectations to algorithmic behavior.
- SITL extends HITL by embedding the general will, due process, and accountability into an algorithmic social contract mediated by machines.
- Unlike HITL’s oversight of uncontested common goals, SITL requires society to resolve value conflicts such as security versus privacy and competing notions of fairness.
- SITL also requires agreement about which stakeholders receive benefits and bear costs, including safety tradeoffs between autonomous-vehicle passengers and pedestrians.
- As governance functions become encoded in AI algorithms, societies need channels connecting human values with governance algorithms.
- Implementing SITL requires tools to elicit expectations, translate goals and norms into machine-operational forms, quantify human values, and program, debug, and monitor the social contract.
6 The SITL Gap
The SITL gap is the lack of comprehensive, operational mechanisms for translating societal values into AI governance. Key difficulties include the engineering–humanities divide, opaque externalities, complex tradeoffs, and co-evolving technical capabilities and norms.
- Existing treaties and scholarship illuminate social and legal challenges from opaque algorithms but fall short of comprehensive solutions.
- 6.1 Articulating Societal Values: The engineering–humanities divide makes it difficult to translate moral hazards, ethical principles, and constitutional rights into requirements engineers can operationalize.
- 6.2 Quantifying Externalities & Negotiating Tradeoffs: AI systems can impose negative externalities on uninvolved third parties, such as pedestrians facing increased risk when vehicle algorithms prioritize passengers.
- 6.2 Quantifying Externalities & Negotiating Tradeoffs: Externalities are difficult to quantify when they arise through long indirect causal chains or opaque machine code.
- 6.2 Quantifying Externalities & Negotiating Tradeoffs: Negotiating AI tradeoffs is harder than setting a speed limit because complex systems offer more design choices and learned behavior can shift beyond programmers’ intentions.
- 6.1 Articulating Societal Values: Engineers may struggle to quantify behavior in forms accessible to ethicists and legal theorists, while concepts such as fairness admit multiple mathematical formalizations.
- 6.1 Articulating Societal Values: Human values and AI capabilities co-evolve, and technical advances can alter what society considers acceptable, including privacy norms.
7 Bridging the Gap
The paper surveys mechanisms for negotiating societal values and scrutinizing algorithmic behavior, while emphasizing that simulated audits can be subverted and public input has limits.
- Negotiating values: Value-sensitive design, crowdsourcing, observational data, and computational social choice offer ways to identify, measure, or aggregate societal preferences.These approaches address value tradeoffs through design methods, preference elicitation, social-media reactions, and algorithmic aggregation.
- Negotiating values: Social contract theory offers normative tools for identifying enforceable outcomes that rational actors would accept.One example uses Rawls’ original position and veil of ignorance to program autonomous vehicles for unavoidable-harm dilemmas.
- Scrutinizing behavior: Algorithmic accountability should scrutinize external behavior against standards rather than rely on source-code inspection.Behavioral scrutiny includes investigative reporting, professional audits using real or synthetic data, and oversight programs monitoring operational AI.
- Limits and safeguards: Simulated audits can be defeated when an audited algorithm detects testing conditions and behaves differently from its real-world behavior.The paper compares this adversarial behavior with emissions-control defeat devices and argues that simulation alone is insufficient.
- Scrutinizing behavior: Real-time monitoring can quantify bias in deployed systems and raise alarms when it exceeds a threshold.The paper presents automated oversight as a way to monitor operational behavior continuously rather than inspect code alone.
- Limits and safeguards: Public participation can help shape societal values, but laypeople cannot be fully informed about every specialized policy question.The paper therefore distinguishes public influence over norms from expert provision of relevant facts and policy assessments.
8 Discussion
The discussion frames algorithmic regulation as a social process requiring desired outcomes, measurement, adjustment, and periodic evaluation. It concludes with a call to place society inside the governance loop of algorithmic systems.
- Algorithmic regulation: Algorithmic regulation requires defined outcomes, real-time measurement, data-driven adjustment, and periodic analysis of whether the rules remain correct.These four properties are presented as O’Reilly’s characterization, which the paper endorses.
- Algorithmic regulation: Identifying and negotiating desired outcomes is non-trivial, and evaluating algorithmic performance is also a social challenge.The paper argues that the social-contract framework is useful because correctness depends on negotiated goals as well as technical performance.
- SITL and HITL: SITL operates on longer time-scales than HITL, resembling public feedback on regulations and legislation rather than feedback on frequent microlevel decisions.The framework still attends to every component of the loop as data science shortens the time between diagnosis and policy adjustment.
- The challenge ahead: The paper synthesizes HITL and social-contract paradigms into a call to build institutions and tools that program, debug, and monitor the algorithmic social contract.This agenda addresses algorithmic systems governing social and economic life and the need to tame the new Techno-Leviathan.