Source-linked AI summary
OmouAI: Argumentative Human-AI Policy Deliberation with Simulated Personas
Stylianos Loukas Vasileiou, Antonio Rago, William Yeoh, Georgina Curto
TL;DR
High-stakes policy deliberation with LLMs can involve sycophancy and reasoning that is difficult to explain or contest. OmouAI combines LLMs with computational argumentation and simulated personas; arguments are evaluated against goals to produce verdicts, but experts must review outputs before real-world use.
Problem
LLM reasoning in high-stakes policy deliberations may be difficult to explain or contest, and models may behave sycophantically.
Source: Introduction · p. 1Method
OmouAI combines LLM and argumentation layers so users and simulated personas contribute arguments to a shared framework evaluated with gradual semantics.
Source: Introduction · p. 1, System Overview · pp. 1–2Results
The framework computes verdicts for the policy claim and selected goals from the shared arguments.
Source: System Overview · p. 2Takeaways & Limitations
Users can contest and revise arguments, while goal verdicts provide indicators for policy recommendations.
Source: Introduction · p. 1, System Overview · p. 2OmouAI is not prescriptive, its arguments reflect model biases and stakeholder values, and domain experts must evaluate outputs before real-world use.
Source: System Overview · p. 2
Abstract
from arXiv · showhide
Debates amongst agents driven by large language models (LLMs) have demonstrated vast potential in various applications, but when these interactions include humans and take place in high-stakes environments, e.g., in public policy deliberations, they are beset with issues such as sycophancy and a lack of faithful explanations. To tackle these issues, we present OmouAI, an interactive and inclusive deliberation system that uses LLMs in combination with computational argumentation, a field which excels in representing and reasoning within debates. OmouAI allows a human user to deliberate policy claims for real-world challenges with simulated personas, e.g., representing stakeholders, domain experts or devil's advocates, towards reducing sycophancy. Each persona generates its own arguments, and the arguments of all parties form a shared argumentation framework. Users can then contest, add and revise arguments, providing crucial human oversight. Then, arguments are evaluated using deterministic argumentative semantics against external goals, such as the UN Sustainable Development Goals, guaranteeing faithful explanations. The advancement or worsening of the goals thus serve as indicators for the policy recommendations.
Introduction
LLM-supported policy deliberation offers promise, but its reasoning can be difficult to explain or contest, and models may be sycophantic. OmouAI combines LLMs with computational argumentation for public policy deliberation.
Introduction
- In high-stakes deliberations, opaque LLM reasoning is difficult to explain, evaluate, or contest, while sycophancy is also a concern.Policy questions involve parties who hold different values and weigh the same evidence differently.Source: Introduction · p. 1
- Prior argumentative LLM systems generate and weigh arguments before computing verdicts, but were not built specifically for public policy deliberation.Related deliberation systems aggregate human views while leaving the reasoning behind their statements unexposed.Source: Introduction · p. 1
- OmouAI combines LLMs and argumentation in a collaborative human-AI system for public policy deliberation.Users deliberate with stakeholder, expert, or devil’s-advocate personas in a contestable argumentative thread, with proposed arguments evaluated against goals such as the UN Sustainable Development Goals.Source: Introduction · p. 1
System Overview
OmouAI combines LLM and argumentation layers to support policy deliberation with selected personas and indicators. Arguments from personas and users enter a shared framework, where gradual semantics evaluate the claim and goals; domain experts must review outputs before real-world use.
System Overview
- The pipeline starts with a user-entered policy claim, selected personas, and goal indicators, then generates persona arguments and mines user messages into arguments.Figure 1 depicts the OmouAI pipeline for a climate change policy example.Source: System Overview · pp. 1–2, Figure 1 · p. 2
- A shared quantitative bipolar argumentation framework combines the claim, persona and user arguments, and indicators through attack and support relations.Gradual semantics assign argument strengths from base scores and attackers’ and supporters’ strengths.Source: System Overview · p. 2
- Personas add one attacker or supporter to a target argument, or decline; base scores reflect evidence for empirical or predictive arguments and persona commitments for normative ones.The claim and indicator arguments begin with a neutral base score of 0.5.Source: System Overview · p. 2
- User messages classified as arguments are restated and linked by a proposed attack or support relation, which the user confirms or redirects before entry into the framework.Requests prompt new persona arguments; questions are answered from the computed deliberation record.Source: System Overview · p. 2
- Gradual semantics compute strengths for the claim and indicators, producing a verdict for each goal alongside the claim verdict.The implemented semantics satisfy contestability properties intended to make interventions move verdicts in the user’s intended direction.Source: System Overview · p. 2
- OmouAI is an argumentation-support tool, not a prescriptive decision system; domain experts must evaluate its outputs before real-world decision-making use.Generated arguments reflect underlying model biases and the represented stakeholders’ value systems.Source: System Overview · p. 2