PaperlayerView exact revision on arXiv

Source-linked AI summary

ARGUS: Theory-of-Mind Guided Argument Generation with Strategy-Aware Planning and Knowledge Grounding

Zhe Hu

arXiv:2608.20405v1cs.CL

TL;DR

Persuasive argument generation needs audience modeling, rhetorical strategy, and factual grounding, which existing methods often treat incompletely. ARGUS combines a Theory-of-Mind reasoner, component-aware planning with strategy-guided retrieval, and iterative refinement; across benchmarks and backbones, it outperforms strong baselines and shifts resistant stances.

  • Problem

    Existing argument-generation methods remain audience-agnostic and largely separate rhetorical strategy selection from dynamically grounded evidence retrieval.

  • Method

    ARGUS constructs a dual Theory-of-Mind audience model, uses it to plan rhetorical components and subtopic-level evidence retrieval, then iteratively refines multi-dimensional weaknesses.

  • Results

    Across three benchmarks and multiple backbone models, ARGUS consistently outperforms strong baselines, while targeted simulations show stance shifts among resistant audiences.

  • Takeaways & Limitations

    The results support explicit audience profiles, strategy-aware planning, and planning-time evidence grounding as central components of ARGUS’s persuasive-generation framework.

  • Takeaways & Limitations

    The LLM-generated audience model may not faithfully represent real mental states, and simulated LLM-judge persuasion remains an approximation requiring validation against human stance shifts.

Abstract

from arXiv · show

Persuasive argument generation requires modeling audience beliefs, rhetorical strategies, and factual grounding. Despite recent advancements, existing methods remain largely audience-agnostic and fail to integrate strategy selection to improve persuasiveness. To bridge this gap, we propose Argus, an agent-based framework that operationalizes classical rhetoric for persuasive writing. At its core, a Theory-of-Mind (ToM) Reasoner constructs an explicit dual mental model of the audience's beliefs and values to guide downstream decisions. This representation conditions a component-aware planner that decomposes the argument into subtopics, assigns fine-grained rhetorical functions (logos, pathos, ethos, kairos), and triggers strategy-guided evidence retrieval at planning time. Finally, a refinement module iteratively targets and resolves multi-dimensional weaknesses without quality regression. We evaluate Argus across three diverse benchmarks using both automated pairwise Elo and LLM-as-judge metrics. Results show that Argus consistently outperforms strong baselines across multiple backbone models, achieving top rankings and the highest overall scores. Targeted simulation experiments further validate its effectiveness in shifting resistant audience stances.

1 Introduction

ARGUS addresses audience-agnostic and strategy-agnostic argument generation by explicitly modeling audience mental states, planning rhetorical functions, and retrieving evidence during planning. Across three datasets and multiple backbone models, it consistently outperforms competitive baselines and shifts resistant audience stances.

  • Research gap: Existing systems improve coherence but often omit explicit audience modeling, rhetorical strategy selection, and dynamically adapted evidence retrieval.These limitations constrain genuine persuasive efficacy by separating what to say from how to persuade and when to retrieve supporting evidence.
  • Framework: ARGUS uses a Theory-of-Mind Reasoner to construct an explicit dual mental model of audience argumentative profiles, emotional triggers, and values.The model guides downstream planning before writing begins.
  • Evaluation: ARGUS is evaluated on ChangeMyView, iDebate, and ExplaGraphs using pairwise Elo and absolute LLM-judge evaluations across three backbone LLM families.The evaluation spans diverse discourse styles and uses both ranking and absolute scoring protocols.
  • Findings: ARGUS consistently outperforms competitive baselines and excels in targeted simulations designed to model stance shifts among resistant counterparts.The empirical validation links explicit audience modeling with rhetorical influence in skeptical environments.
  • Framework: Its component-aware planner decomposes arguments into subtopics, assigns logos, pathos, ethos, or kairos, and triggers strategy-guided evidence retrieval during planning.This couples rhetorical decisions with localized evidence grounding rather than relying on undifferentiated retrieval.

2 Related Work

Prior argument-generation research has moved from end-to-end neural models toward structured planning and agentic workflows, while ToM research has begun examining audience-aware persuasion. ARGUS builds on these lines by targeting planning ToM for deliberate stance change.

  • Argument Generation: Earlier argument-generation systems used end-to-end neural generation, while later methods introduced explicit planning and multi-agent workflows to improve argument quality.These approaches include decomposition pipelines, prompting-based agents, and debate-driven generation.
  • Synthesis: ARGUS is positioned at the intersection of structured argument generation and audience-aware rhetorical planning.Its contribution is framed against both agentic generation workflows and emerging ToM-for-persuasion research.
  • Theory of Mind and Audience Modeling: Argumentation theory treats persuasion as jointly dependent on the argument, its source, and the audience.This motivates modeling the audience as more than an implicit generation context.
  • Theory of Mind and Audience Modeling: Recent ToM evaluations often test belief prediction, whereas persuasion requires planning ToM that deliberately models how to shift a stance.Related work has therefore begun developing persuasive-dialogue benchmarks and opponent-aware persuaders.

3 Method

ARGUS decomposes argument generation into interpretable stages that analyze the input, model the audience, plan rhetorical components with localized evidence, write the argument, and iteratively refine weaknesses. The pipeline keeps audience alignment, rhetorical organization, grounding, and coherence explicit throughout generation.

  • 3.1 Task Formulation: The task maps a proposition x and stance s to an argumentative article y that persuasively defends s.ARGUS treats this as conditional generation rather than a single prompting step.
  • Framework: ARGUS uses five sequential modules: Input Analyzer, ToM Reasoner, Argument Planner, Argument Writer, and Iterative Refiner.Each module operates on structured representations passed to downstream components.
  • 3.3 Input Analysis: The Input Analyzer retrieves targeted external knowledge when needed and produces claims, synthesized background knowledge, and optional logical structure.This retrieval-first design grounds analysis in evidence and exposes assumptions or logical gaps.
  • Audience Modeling: The ToM Reasoner represents the audience through a dual model of argumentative profile and value landscape.The profile includes emotional themes, held positions, and underlying motivations, while the value model uses open-ended phrases.
  • 3.5 Argument Planning: The planner creates an ordered subtopic blueprint whose components include rhetorical mode, key claims, retrieved evidence, and anticipated rebuttals.Assignments are conditioned on audience modeling and may use logos, pathos, ethos, kairos, or evidence.
  • 3.5 Argument Planning: Evidence retrieval occurs at subtopic level, and the planner self-evaluates rhetorical diversity, ToM alignment, logical ordering, and stance consistency before writing.Modified subtopics can trigger renewed evidence retrieval during iterative plan revision.
  • 3.6 Argument Writing and Refinement: The Writer preserves planned rhetorical composition, local evidence grounding, and audience value framing, while the Refiner targets weaknesses across persuasiveness, coherence, factual accuracy, value alignment, and rhetorical balance.A best-of-n buffer returns the highest-scoring draft to prevent quality regression.

4 Experimental Setup

The experiments evaluate ARGUS across diverse benchmarks and backbone models against increasingly structured generation baselines. Evaluation combines pairwise comparisons with absolute LLM-based scoring across argument quality dimensions.

  • Datasets: ARGUS is evaluated on ChangeMyView and iDebate, covering politics, policy, and controversial propositions with refute as the target stance.The benchmarks span diverse domains and discourse styles.
  • Baselines: The comparison includes Direct, Plan-and-Write, Self-Refine, and Debate baselines using the same backbone LLMs.These baselines represent single-pass, planning, self-critique, and opposing-agent generation strategies.
  • Backbones: Three backbone models are included: DeepSeek-V3.2, Qwen3.5-Flash, and GPT-5-mini.Using multiple backbones tests whether results extend beyond one generation model.
  • Evaluation Protocol: The evaluation combines round-robin pairwise Elo comparisons with absolute LLM judging of persuasiveness, coherence, and factual accuracy.Pairwise judgments swap order to mitigate position bias, and absolute scoring assesses each argument independently.
  • Evaluation Protocol: The judge is fixed to GPT-5.4, distinct from the generation backbone.This separates the evaluation model from the models producing the arguments.

5 Results and Analysis

ARGUS consistently outperforms baselines across datasets, backbones, evaluation protocols, and persuasion simulations. Its strongest advantages center on persuasive effectiveness and factual grounding, while ablations support the complementary value of audience modeling, structured planning, and targeted refinement.

  • Main Results: ARGUS consistently outperforms all baselines across pairwise Elo rankings and absolute LLM-as-judge scores on three datasets.The absolute evaluation particularly favors ARGUS in persuasiveness and factual accuracy, while coherence is uniformly high across methods.
  • Pairwise Evaluation: ARGUS achieves its most significant pairwise Elo gains on iDebate, where short open-domain propositions require broad knowledge, reasoning, and flexible rhetorical construction.Plan&Write is the strongest competing baseline, while Self-Refine and Multi-Agent Debate lag behind because of instability or insufficient coordination.
  • Pairwise Evaluation: ARGUS shows strong CMV improvements and the best ExplaGraphs Elo scores across settings while maintaining factual consistency and coherent reasoning.CMV contains opinion-rich statements, whereas ExplaGraphs presents a more structured, fact-oriented reasoning scenario.
  • Absolute Evaluation: ARGUS wins pairwise comparisons even when another baseline leads an isolated rubric dimension, indicating that holistic quality includes global balance, organization, and rhetorical effectiveness.On ExplaGraphs with GPT-5-mini, Plan&Write has the highest persuasiveness score and Self-Refine the best factual accuracy, yet ARGUS still wins pairwise comparisons.
  • Target Simulation: ARGUS achieves the highest simulated persuasion scores, with especially pronounced improvement on CMV, where judges role-play skeptical audiences and rate stance change from 1 to 10.Each argument receives two independent ratings whose average reduces stochasticity.
  • Ablation Study: Ablations show that removing any component degrades performance; omitting the planner hurts more than omitting ToM, while removing the refiner causes the largest drop.The findings indicate that audience profiles require structured rhetorical plans, and targeted multi-dimensional revision complements planning rather than replacing it.
  • Rhetorical Analysis: Rhetorical strategy effectiveness depends on audience and task: CMV is more logos-oriented, while iDebate and ExplaGraphs emphasize ethos, and pathos-heavy CMV arguments correlate negatively with persuasion.Stronger ethos correlates positively with CMV effectiveness, whereas these correlations largely disappear for iDebate and ExplaGraphs.
  • Case Study: A successful example shows the ToM audience model guiding plan ordering so the argument resolves the audience’s central objection before presenting supporting claims.The argument reframes “safety algorithm” as a concrete engineering concept and supports the definition with examples.

6 Conclusion

ARGUS combines explicit audience modeling, rhetorical planning with evidence grounding, and multi-dimensional refinement in an agent-based framework. Across three benchmarks and multiple backbones, it outperforms strong baselines, with targeted simulations indicating stance shifts beyond fluency.

  • Conclusion: ARGUS externalizes audience beliefs and values, plans rhetorical functions and evidence for each subtopic, and refines weaknesses without quality regression.Its three connected components are the ToM reasoner, component-aware planner, and multi-dimensional refiner.
  • Conclusion: Across three benchmarks and multiple backbones, ARGUS consistently outperforms strong baselines, while targeted simulations show gains translating into audience stance shifts.The conclusion distinguishes these simulated persuasion effects from mere fluency.

Limitations

ARGUS has limitations in audience modeling, factual grounding, latency, and persuasion evaluation. The paper identifies these as boundaries for interpreting its results and as directions for future work.

  • The Theory-of-Mind model may plausibly represent audience mental states without faithfully capturing verified real audience states.
  • Individual sections can still contain inaccurate claims despite planning-time evidence retrieval, especially when arguing against empirical consensus.The authors identify a dedicated fact-checking agent as a natural next step.
  • The multi-stage pipeline adds latency compared with single-pass generation, although parallelizing independent agents could mitigate it.
  • The targeted simulation provides a controlled and reproducible proxy for persuasion, but validating simulated stance shifts against human evaluation remains future work.LLM judges may carry biases and favor model-generated text.

Ethical Considerations

ARGUS's audience modeling creates dual-use risks because techniques that increase resonance can also support manipulation, disinformation, or influence operations.

  • Audience modeling can enable manipulation, disinformation, or influence operations alongside legitimate persuasive applications.The paper advocates disclosure, safeguards against deceptive use, and research on provenance and detection.

A Implementation Details

The implementation evaluates ARGUS and its baselines across three datasets using specified backbone and judge models, while supplementary figures document system prompts and an input-analysis example.

  • Three instruction-tuned LLMs serve as backbone models, while GPT-5.4 is fixed as the evaluation judge across conditions.The listed backbones are DeepSeek-v3.2, Qwen3.5-Flash-2026-02-23, and gpt-5-mini-2025-08-07.
  • Generation uses up to three JSON-output retries with exponential backoff, and both planning iterations and refinement rounds are capped at two.WebSearch uses ddgs for URLs and snippets and trafilatura for webpage parsing.
  • Evaluation samples 30 inputs from each of Reddit/CMV, iDebate, and ExplaGraph.
  • Figure 3 presents a structured Input Analyzer output for the proposition “Hate speech is free speech.”The accompanying background states that U.S. First Amendment doctrine has no formal hate-speech exception, while narrow exclusions and psychological harms are noted.
  • All four baselines use the same backbone LLM as ARGUS with zero-shot prompting and matched inference parameters.

B ARGUS Module Prompts

Figures 5–11 document the system prompts used by ARGUS modules and evaluators, highlighting the three novel components and showing informative user-message templates.

  • Figures 5–11 report system prompts for all ARGUS modules and evaluators, with red-bordered boxes marking the three novel components.User-message templates are included where informative, and brackets denote dynamically inserted content.

C Evaluation Protocol Details

ARGUS evaluates arguments with absolute quality scores and pairwise ELO comparisons, while its prompts specify how retrieval, analysis, audience modeling, planning, writing, and refinement are coordinated.

  • Absolute scoring: Each argument receives an independent GPT-5.4 quality score using the Quality Evaluator prompt.The evaluator assesses arguments across multiple dimensions and provides actionable feedback.
  • Pairwise ELO ranking: Pairwise ELO evaluation compares each argument pair in both orders and declares wins only when the score gap exceeds 0.5.A gap between -0.5 and 0.5 is treated as a tie, while averaging forward and reverse matrices cancels position bias.
  • Input analysis and retrieval: The Input Analyzer decides whether retrieval is needed, generates 2–4 targeted queries, and grounds unified analysis in retrieved content before downstream reasoning.The resulting structured outputs K, L, and B are passed verbatim to the ToM Reasoner and Argument Planner.
  • Theory-of-Mind modeling: The ToM Reasoner models the audience descriptively from the proposition and both stances without generating arguments or persuasive text.Its output is a dual mental model containing opponent and value representations.
  • Pipeline prompts: ARGUS passes structured representations through five sequential modules: Input Analyzer, ToM Reasoner, Argument Planner, Argument Writer, and Iterative Refiner.Each module operates on structured representations whose outputs are passed to downstream components.
Loading 2608.20405v1…