PaperlayerView exact revision on arXiv

Source-linked AI summary

Adaptive Multi-Agent Response Refinement in Conversational Systems

Soyeong Jeong, Aparna Elangovan, Emine Yilmaz, Oleg Rokhlenko

arXiv:2511.08319v1cs.CLcs.AIcs.MA

TL;DR

Personalized multi-turn conversations challenge LLMs to maintain factual accuracy, persona alignment, and coherence, while single-agent refinement can be suboptimal. MARA uses dynamically allocated specialized agents and significantly outperforms existing baselines on challenging conversational datasets.

  • Problem

    In complex multi-turn conversations, LLM responses may miss user profiles or specific knowledge, lose coherence, and single-agent refinement can be suboptimal.

    Source: §1 · p. 1, §3.1 · p. 3

  • Method

    MARA uses specialized agents for fact-checking, persona alignment, and coherence, dynamically allocating them to each query.

    Source: §6 · p. 9

  • Results

    MARA significantly outperforms existing baselines on challenging conversational datasets.

    Source: §6 · p. 9

  • Takeaways & Limitations

    MARA’s superior performance over single-agent refinement demonstrates the benefit of distributed specialization across multiple agents.

    Source: §5.2 · p. 8

    The unsupervised LLM-based planner has a performance gap from the ideal planner.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Large Language Models (LLMs) have demonstrated remarkable success in conversational systems by generating human-like responses. However, they can fall short, especially when required to account for personalization or specific knowledge. In real-life settings, it is impractical to rely on users to detect these errors and request a new response. One way to address this problem is to refine the response before returning it to the user. While existing approaches focus on refining responses within a single LLM, this method struggles to consider diverse aspects needed for effective conversations. In this work, we propose refining responses through a multi-agent framework, where each agent is assigned a specific role for each aspect. We focus on three key aspects crucial to conversational quality: factuality, personalization, and coherence. Each agent is responsible for reviewing and refining one of these aspects, and their feedback is then merged to improve the overall response. To enhance collaboration among them, we introduce a dynamic communication strategy. Instead of following a fixed sequence of agents, our approach adaptively selects and coordinates the most relevant agents based on the specific requirements of each query. We validate our framework on challenging conversational datasets, demonstrating that ours significantly outperforms relevant baselines, particularly in tasks involving knowledge or user's persona, or both.

1 Introduction

LLMs can produce inaccurate or poorly personalized responses in multi-turn conversations, making user-requested corrections disruptive. The paper proposes specialized refining agents coordinated dynamically for each query, and reports that MARA outperforms relevant baselines.

1 Introduction

  • In multi-turn conversations, errors and misalignments can prompt users to request corrections, disrupting conversational flow and diminishing user experience.Single-agent refinement may also amplify bias when an agent becomes overconfident in its initial output.Source: §1 · p. 1
  • The framework assigns distinct agents to persona alignment, factual grounding, and coherence with earlier conversational turns.These roles target different aspects of response quality.Source: §1 · p. 2
  • A planner selects the refining agents and their sequence for each query, providing justifications that the agents use while refining responses in order.The selected agents are unsupervised LLMs prompted for specific roles.Source: §1 · p. 2
  • MARA significantly outperforms relevant baselines on challenging conversations, and analyses highlight the importance of dynamically tailoring refinement strategies to each query.The evaluated settings involve persona integration, specific factual knowledge, or both.Source: §1 · p. 2

2 Related Work

Prior work addresses unsatisfactory conversational responses through verification, self-refinement, and multi-agent collaboration. However, the use of multiple agents to refine conversational turns remains underexplored.

LLMs in Conversation Systems

  • LLMs can miss user profiles, specific knowledge, or coherence with previous turns, leaving conversational responses unsatisfactory.Prior work connects these issues to failures in personalization, knowledge use, and turn-to-turn coherence.Source: §2 · p. 3

Verification and Refinement with LLMs

  • Self-refinement methods let an LLM generate feedback on its response, but single-agent capacity and confidence can limit novel feedback and refinement.The cited work raises these concerns: Liang et al. (2023); Xu et al. (2024b); Huang et al. (2024).Source: §2 · p. 3

Multi-Agent LLMs

3 Method

MARA refines conversational responses through role-specialized agents and a planner that adapts agent selection and order to each query. The method targets factual accuracy, persona alignment, and coherence across turns.

3 Method

  • MARA dynamically refines conversational responses using a multi-agent framework.This is the method's central approach.Source: §3 · p. 3

3.1 Preliminaries

  • A responding LLM generates each turn from the current query and preceding conversational context; initial responses may miss persona, knowledge, or coherence requirements.Errors in earlier turns can cumulatively affect subsequent turns.Source: §3.1 · p. 3

3.2 Multi-Agent Response Refinement

  • Three specialized agents refine responses for factual accuracy, user-profile alignment, and coherence across conversational turns.Each agent uses a role-specific prompt template; this contrasts with a single agent handling multiple aspects.Source: §3.2 · pp. 3–4

3.3 Communication Strategy among Agents

  • A planner selects a query-specific sequence of refining agents and supplies justifications; each agent then refines the preceding response using those justifications.The paper contrasts this dynamic approach with simultaneous refinement, which uses a finalizer, and static sequences whose optimal order can vary with context.Source: §3.3 · p. 4

4 Experimental Setups

The experiments evaluate MARA on conversational datasets covering persona alignment, factual grounding, or both, against single- and multi-agent baselines. They use G-Eval metrics and several LLM configurations.

4.1 Datasets

  • PersonaChat tests persona alignment, INSCIT tests Wikipedia-grounded information seeking, and FoCus requires both; PRODIGy and Ubuntu Dialogue Corpus add role-play and domain-specific settings.The evaluation also includes these latter two datasets.Source: §4.1 · p. 4

4.2 Baselines and Our Model

  • MARA is compared with no-refinement, single-agent, and multi-agent baselines, including Self-Refine, SPP, LLMvLLM, MADR, and MultiDebate.The baselines cover different refinement strategies; MARA uses a planner to adaptively generate the agent order.Source: §4.2 · p. 5

4.3 Evaluation Metrics

  • G-Eval measures coherence, groundedness, naturalness, engagingness, and a scale-normalized average overall score.Coherence, groundedness, and engagingness correspond to the coherence-, fact-, and persona-refining agents, respectively.Source: §4.3 · p. 5

4.4 Implementation Details

  • Claude Sonnet 3 is the base model; the fact-refining agent uses Sonnet 3.5, while MARA* uses Sonnet 3 for that role and GPT-4o-mini and LLaMA 3.1 test robustness.The study samples 100 conversations per dataset, totaling 673 PersonaChat, 506 INSCIT, and 563 FoCus queries.Source: §4.4 · p. 5

5 Experimental Results and Analyses

MARA outperforms other models across diverse metrics, while the analyses report benefits from query-adaptive allocation and distributed refinement.

5.1 Main Results

  • Refining-agent allocation varies by dataset: persona refinement is common for preference-focused queries, fact refinement for knowledge-focused queries, and both are used similarly when both are needed.The figure compares distributions across three datasets.Source: Figure 2 · p. 6, §5.1 · p. 6
  • MARA consistently outperforms other models across diverse metrics, while single-agent refinement is less effective on factual tasks and multi-agent performance depends on agent design.The reported scenarios include persona alignment, factual grounding, and both together.Source: §5.1 · p. 6
  • The planner-driven sequence beats random assignment; the ideal planner achieves the highest performance while requiring fewer refining-agent accesses.The ideal planner selects the best sequence by testing possible combinations for each query.Source: Table 2 · p. 6, §5.1 · pp. 6–7
  • MARA significantly outperforms baselines on the role-playing PRODIGy dataset and the Ubuntu Dialogue Corpus.The evaluations use GPT as the base LLM for PRODIGy and Claude for Ubuntu.Source: Table 3 · p. 7, Table 4 · p. 7, §5.1 · p. 7

5.2 Ablations and Discussion

  • Human raters preferred MARA; G-Eval aligned most strongly with human rankings on engagingness and least on naturalness.The human evaluation covers 288 FoCus queries rated by eight English-fluent participants.Source: Table 5 · p. 7, §5.2 · p. 7
  • All three perspectives help, but distributed agents outperform a single agent combining them; repeated single-agent rounds instead reduce performance.The ablation study reports each agent as important and all three perspectives together as more beneficial.Source: Table 6 · p. 7, §5.2 · p. 8
  • Performance varies substantially with static refinement order, whereas dynamic selection yields more consistent and improved results; simultaneous communication uses more agent accesses per query.The simultaneous strategy uses all three refining agents plus a finalizer.Source: Table 7 · p. 8, §5.2 · p. 8
  • A different fact-refining model yields better outcomes than same-model refinement, and MARA also improves responses across other tested LLMs.The paper reports improvement even when the responding agent is already powerful.Source: §5.2 · p. 8

6 Conclusion

MARA targets customized conversations requiring persona alignment and specific knowledge through specialized agents assigned dynamically to each query. On challenging conversational datasets, it significantly outperforms existing baselines.

6 Conclusion

  • The target setting is customized conversation requiring both persona alignment and specific knowledge.Source: §6 · pp. 8–9
  • MARA significantly outperforms existing baselines on challenging conversational datasets.Its agents specialize in fact-checking, persona alignment, and coherence, with allocation tailored to each query.Source: §6 · p. 9
  • The framework dynamically allocates specialized agents for fact-checking, persona alignment, and coherence to address each query’s needs.Source: §6 · p. 9

Limitations

The authors identify the planner as an area for improvement and flag scalability and resource efficiency as concerns for multi-agent refinement. They point to planner optimization, smaller models, and external tools as directions for further work.

Limitations

  • The current unsupervised-LLM planner trails an ideal planner, motivating labeled training data for its future fine-tuning.Source: Limitations · p. 9
  • Scalability and resource efficiency remain concerns, though an optimized planner reduces LLM accesses and MARA performs well with smaller models.The authors also suggest exploring lightweight agent models and identify further scalability exploration as future work.Source: Limitations · p. 9
  • The framework’s flexibility could support external tools such as RAG systems to further enhance performance.This is proposed as a future research direction.Source: Limitations · p. 9

Ethics Statement

The authors validate MARA in realistic conversational scenarios with diverse queries, while noting that harmful or offensive inputs and generated responses require additional safeguards. Monitoring both is identified as future work.

Ethics Statement

  • User inputs and generated responses may be harmful or offensive, so the authors recommend safeguards including monitoring both.Source: Ethics Statement · p. 9
  • The authors identify improving MARA’s robustness and ethical alignment in real-world applications as future work.Source: Ethics Statement · p. 9

A.1 Datasets

The evaluation uses three conversational datasets spanning persona alignment, factual grounding, or both. The appendix also describes comparison models and prompt templates for MARA and G-Eval.

A.1 Datasets

  • FoCus, PersonaChat, and INSCIT cover queries requiring persona alignment, factual grounding, or both.FoCus uses Wikipedia-sourced knowledge; PersonaChat provides persona sentences, and INSCIT grounds information-seeking conversations in Wikipedia.Source: §A.1 · p. 14

A.2 Models

  • The comparison includes no refinement, two single-agent refinement baselines, and multiple multi-agent approaches, including MARA’s planner-selected sequence.The appendix describes the baselines’ refinement procedures and MARA’s dynamic agent ordering.Source: §A.2 · p. 14

A.3 Implementation Details

  • The appendix provides separate prompt templates for MARA’s responding, planning, and three refining agents, as well as G-Eval.Source: §A.3 · p. 14

B.1 Significance Test on FoCus (Table 13)

On FoCus, MARA significantly outperformed the other models on coherence, groundedness, naturalness, and engagingness.

  • 0.7456 mean difference over MADR: MARA significantly led coherence, also beating No Refine by 0.2838 and Self-Refine by 0.5691.Source: §B.1 · p. 14
  • 0.3337 mean difference over MADR: MARA significantly led groundedness, also beating No Refine by 0.1648 and Self-Refine by 0.2806.Source: §B.1 · p. 15
  • 0.4772 mean difference over MADR: MARA significantly led naturalness, also beating No Refine by 0.1339 and Self-Refine by 0.2851.Source: §B.1 · p. 15
  • 1.2939 mean difference over MADR: MARA significantly led engagingness, also beating No Refine by 0.6771 and Self-Refine by 0.7801.Source: §B.1 · p. 15

B.2 Significance Test on PersonaChat (Table 14)

On PersonaChat, MARA significantly outperformed the other models on coherence, groundedness, naturalness, and engagingness.

  • 0.7359 mean difference over MADR: MARA significantly led coherence, also beating No Refine by 0.4572 and Self-Refine by 0.0611.Source: §B.2 · p. 15
  • 0.2731 mean difference over MADR: MARA significantly led groundedness, also beating No Refine by 0.1406 and Self-Refine by 0.0369.Source: §B.2 · p. 15
  • 0.6000 mean difference over MADR: MARA significantly led naturalness, also beating No Refine by 0.4273 and SPP by 0.2599.Source: §B.2 · p. 15
  • 1.2213 mean difference over MADR: MARA significantly led engagingness, also beating No Refine by 0.9953 and SPP by 0.4745.Source: §B.2 · p. 15

B.3 Significance Test on INSCIT (Table 15)

On INSCIT, MARA led several model comparisons across four response-quality metrics; ablations and case examples also describe the effects of agent coordination and query-specific refinement.

B.4 Ablation Studies on Design Choices

  • Removing the planner’s sequence and justifications from agent prompts lowered every metric, particularly groundedness; removing step-by-step verification and justification lowered coherence, naturalness, and engagingness.Source: Table 10 · p. 16, §B.4 · p. 16

B.5 Case Studies

  • In the Chaos Crags case, MARA supplied the mountain’s exact elevation and geological context, then tailored a lake response to the user’s interests while SPP missed the query.Source: §B.5 · pp. 16–17, Table 11 · p. 17

B.6 Illustrative Example of Agent Roles

  • In the Galusha House example, the planner selects coherence then persona refinement: the first adds context, and the second adds personalized context and an engaging follow-up.Source: Table 12 · p. 17, §B.6 · p. 18

B.3 Significance Test on INSCIT (Table 15)

  • 0.3264 mean difference over MADR: MARA significantly led coherence, also beating No Refine by 0.0844 and SPP by 0.0946.Source: §B.3 · p. 16, Table 15 · p. 20
  • 0.129 mean difference over MADR: MARA significantly led groundedness, also beating No Refine by 0.0765 and SPP by 0.101.Source: §B.3 · p. 16, Table 15 · p. 20
  • 0.1382 mean difference over MADR: MARA significantly led naturalness and beat No Refine by 0.1052, but its comparison with Self-Refine was not significant.Source: §B.3 · p. 16, Table 15 · p. 20
  • 0.8387 mean difference over MADR: MARA significantly led engagingness, also beating No Refine by 0.4544 and SPP by 0.2664.Source: §B.3 · p. 16, Table 15 · p. 20

Related papers

Loading 2511.08319v1…