PaperlayerView exact revision on arXiv

Source-linked AI summary

Persona-Guided LLM Agents for Task-Oriented Dialogue

Maryam Shoaeinaeini, Brent Harrison, A. B. Siddique

arXiv:2608.18085v1cs.CL

TL;DR

It remains unclear whether LLMs can express personality in goal-directed dialogue without compromising task completion, and whether personality adaptation improves interaction quality. The paper tests training-free adaptation in simulated task-oriented dialogues and finds that cue-based adaptation best balances personalization gains with reduced truthfulness.

  • Problem

    It remains unclear whether LLMs can express personality in task-oriented dialogue while completing tasks, and whether adapting to users’ personalities improves interaction quality.

  • Method

    A training-free framework simulates matched schema-guided dialogues between LLM-based user and system agents under Neutral, Try, and Oracle personality-access conditions.

  • Results

    Personality adaptation improves constraint satisfaction, inform rate, and user satisfaction but reduces truthfulness; Try is less sensitive to trait realization strength than Oracle.

  • Takeaways & Limitations

    Cue-based Try adaptation offers the most reliable overall balance for personality-aware task-oriented dialogue without fine-tuning.

  • Takeaways & Limitations

    The study uses simulated interactions between LLM-based agents, so its findings require validation with real users and human-centered interaction studies.

Abstract

from arXiv · show

Prior work has shown that large language models (LLMs) can express diverse personality traits in open-ended text generation. However, it remains unclear whether they can do so in a goal-directed dialogue without compromising task completion, and whether adapting to the user's personality improves the interaction quality. We study these questions in task-oriented dialogue (TOD), where a system helps a user accomplish a goal via multi-turn interaction. We build a training-free framework that simulates a TOD interaction between two LLMs: a user agent that exhibits a target personality and a system agent that adapts to the user while completing the task. To isolate the effect of adaptation, we vary how much the system knows about the user's personality across three conditions. In Neutral, the system receives no personality information. In Try, it infers the personality from dialogue cues. In Oracle, it is given the personality explicitly. We evaluate GPT-4o, Qwen3-Next-80B, and Gemini 2.0 Flash on Hotel and Restaurant dialogues from the Schema-Guided Dialogue (SGD) dataset, across the Big Five traits and their opposite poles. We find that the user agent can express personality while the system maintains strong task performance, although some traits are realized far less reliably than others. Adapting to the user's personality improves constraint satisfaction, inform rate, and user satisfaction, but lowers truthfulness, revealing a trade-off between personalization and task-grounding. Oracle's gains grow when the target trait is strongly expressed, whereas Try's gains are largely insensitive to realization strength. Overall, cue-based adaptation in Try best resolves this trade-off and offers a more reliable route to personality-aware TOD without fine-tuning.

1 Introduction

This section motivates personality-aware task-oriented dialogue as a balance between personalized interaction and task-grounded behavior. It frames three questions about personality expression, personality access, and trait-realization strength, then reports gains in several task metrics alongside reduced truthfulness.

  • Motivation: Personalization matters because users want assistants that complete tasks while communicating in appropriate, responsive, and personally aligned ways.The motivation spans customer support, booking, and assistant scenarios.
  • Motivation: Task-oriented dialogue requires personality-aware systems to balance responsiveness to communication style with schema compliance, valid API calls, task completion, and grounding.Strong adaptation may reduce grounding, distort recommendations, or interfere with completion.
  • Research questions: The study asks whether training-free LLM agents can express personality while satisfying schema constraints, how personality access affects quality and satisfaction, and how trait realization shapes Oracle-versus-Try gains.These questions define RQ1, RQ2, and RQ3.
  • Findings: LLM-based user agents express personality in schema-guided dialogues while systems maintain strong task performance, although some traits are realized much less reliably than others.This finding addresses whether personality and task performance can coexist without fine-tuning.
  • Findings: Personality adaptation improves constraint satisfaction, inform rate, and pairwise user satisfaction, but reduces truthfulness, exposing a personalization–grounding trade-off.The result addresses the effect of personality access on system quality and user satisfaction.

2 Related Work

Prior work has advanced structured task-oriented dialogue, personality expression, and personalized interaction largely in separate settings. This section positions the framework as connecting these directions through personality-conditioned user simulation and evaluation of schema-grounded behavior.

  • Task-Oriented Dialogue: Traditional TOD systems track dialogue state, fill slots, predict actions, and ground responses in structured APIs or knowledge sources.Datasets such as MultiWOZ and SGD support cross-domain benchmarking while exposing the cost and brittleness of manual annotation.
  • Task-Oriented Dialogue: LLM-based TOD improves flexibility through in-context learning and reduced annotation, but prior work mainly targets task success, slot accuracy, and domain-shift robustness.User-specific adaptation remains comparatively underexplored.
  • Personality and Personas: Personality studies show that LLMs can imitate recognizable trait patterns and exhibit measurable variation across personality dimensions.Other work induces or edits personality through prompting, persona conditioning, and model-level control, mostly in open-ended or non-task settings.
  • Research Gap: Personality research does not establish how trait conditioning interacts with schema constraints, API decisions, or multi-turn task completion.Persona-aware dialogue generation improves persona consistency but is not designed for schema-guided TOD requiring valid API calls and completed user goals.
  • Contribution: The framework connects TOD and personality research by conditioning simulated users on Big Five traits and evaluating schema-grounded behavior, satisfaction, and trait realization.It examines personality access across multiple models and evaluation metrics.

3 Methods and Experimental Setup

The study uses a controlled, training-free simulation of personality-aware task-oriented dialogue on SGD, varying only the system’s access to user-personality information across Neutral, Try, and Oracle conditions. It evaluates implicit Big Five personas and system, task, personality, and satisfaction outcomes across Restaurant and Hotel dialogues.

  • Framework: The framework simulates matched interactions between LLM-based user and system agents, holding task schema, domain, demonstrations, and dialogue rules fixed while varying personality access.The system conditions are Neutral, Try, and Oracle; both agents use one-shot prompting without fine-tuning.
  • Personas: Each user agent receives exactly one implicitly expressed target from ten Big Five traits or opposite poles, testing personality realization under schema and API constraints.The targets include Extraversion/introversive, Agreeableness/disagreeable, Conscientiousness/unconscientious, Neuroticism/stable, and Openness/closed.
  • Experimental settings: Experiments cover Restaurant and Hotel SGD search and reservation tasks using GPT-4o, Qwen3-Next-80B, and Gemini 2.0 Flash, with dialogues capped at 70 turns or 5 API calls.The user always initiates, and dialogues stop upon scripted completion after task completion, at 70 turns, or after 5 API calls.
  • Evaluation: Evaluation combines task outcomes, personality realization, system quality, and pairwise user satisfaction across matched domain, model, and target-trait dialogues.Metrics include Dialogue Completion, Task Success, TOD Trait Score, BFI prompt-level alignment, Constraint Satisfaction, Inform Rate, Truthfulness, and pairwise wins for Oracle–Neutral, Try–Neutral, and Try–Oracle.
  • Evaluation: Gemma-4-31B judges system quality and dialogue-level personality realization consistently across conditions, while the three generation models separately evaluate BFI trait alignment.The BFI score is a controlled prompt-level check rather than a score computed from generated TOD dialogues.

4 Results and Analysis

Training-free LLM agents jointly support personality-aware interaction and schema-guided task completion, although dialogue-level trait realization is uneven. Personality access improves system quality and satisfaction, while cue-based adaptation in Try offers the most reliable balance between personalization and grounding, with gains less dependent on trait realization strength.

  • RQ1: Personality-aware TOD is feasible: 89.94% Dialogue Completion and 85.27% Task Success show that personality-conditioned user behavior generally preserves coherent, schema-guided task completion.The framework jointly supports personality-aware interaction and task completion without generally preventing dialogues from reaching an endpoint.
  • RQ1: Personality-aware TOD is feasible: User Trait Score averages 3.620, with stronger realization for Extraversion, Agreeableness, Neuroticism, Openness, and disagreeable than for introversive and closed.The separate BFI probe reaches a high average BFI Score of 4.888, despite weaker dialogue-level realization for introversive and closed.
  • RQ2: Personality access improves quality and satisfaction: Try achieves the highest average Constraint Satisfaction (4.646) and Inform Rate (4.849), while Neutral remains the most truthful condition.Try is preferred over Neutral at 63.40% versus 35.20% and over Oracle at 56.10% versus 43.00%.
  • RQ2: Personality access improves quality and satisfaction: Try preserves significantly higher Truthfulness than Oracle (W=119937, p<.001), revealing a trade-off between explicit personalization and task-grounding.The trade-off is especially visible for Extraversion and Openness, where explicit conditioning lowers Truthfulness despite improving or maintaining other quality dimensions.
  • RQ3: Explicit conditioning depends on realization: Oracle gains correlate positively with trait realization (ρ=+0.685, p=.029, n=10), whereas Try gains do not (ρ=−0.103, p=.777).Try adapts to dialogue cues rather than a predefined trait label, making its benefits less sensitive to whether the intended trait is strongly expressed.

5 Conclusion

The study shows that LLM agents can express personality in goal-directed dialogue while maintaining strong task performance, though trait realization is uneven. Personality adaptation improves several task and satisfaction outcomes but reduces truthfulness, with cue-based Try providing the most reliable overall balance without fine-tuning.

  • Framework: A controlled, training-free framework simulates personality-conditioned users and task-oriented systems across three LLMs, two SGD domains, and ten Big Five traits with opposite-pole variants.The framework isolates personality adaptation in multi-turn task-oriented dialogue.
  • Personality realization: LLM agents maintain strong task performance while expressing personality, but realization is uneven, with closed and introversive especially difficult to realize in multi-turn TOD.Some traits are expressed clearly, whereas others are much harder to realize reliably.
  • Adaptation effects: Personality adaptation improves Constraint Satisfaction, Inform Rate, and pairwise user satisfaction, but reduces Truthfulness.Wilcoxon signed-rank tests confirm that these differences are reliable.
  • Implications: Cue-based adaptation in Try offers the most reliable overall balance and a robust route to personality-aware TOD without fine-tuning.Future work should validate these findings with human users and explore hybrid strategies combining explicit trait information with dialogue cues.
  • Adaptation conditions: Oracle gains increase with trait realization (ρ=+0.685, p=.029), whereas Try gains do not (ρ=−0.103, p=.777).Explicit personality labels help most when they match observable behavior, while Try is less sensitive to realization strength.

6 Limitations

The study’s conclusions are limited by simulated interactions, narrow domains, single static traits, LLM-based evaluation, and prompting-only adaptation. Future work should test real users, broader and riskier domains, dynamic personas, human judgments, and alternative implementations.

  • Limitations: Simulated LLM user–system interactions isolate personality access under matched tasks but require validation with real users and human-centered studies.The controlled design supports causal comparison while limiting ecological validity.
  • Limitations: Experiments cover only Hotel and Restaurant dialogues, leaving unclear whether the personalization–grounding trade-off extends to education, healthcare, or customer support.These domains provide structured search and reservation tasks with clear API grounding.
  • Limitations: Assigning one Big Five trait or opposite pole per dialogue omits users who express mixed traits or shift interaction styles across turns.Future studies could evaluate mixed-trait personas and dynamic personality expression.
  • Limitations: LLM-based judges enable scalable evaluation but may not replace human judgments of system quality, satisfaction, or personality alignment.Additional human evaluation would strengthen interpretations of user satisfaction and trait realization.
  • Limitations: Because the study uses prompting-based adaptation without fine-tuning, different models, decoding settings, prompts, or tool-use implementations may produce different outcomes.The findings therefore apply specifically to training-free prompting in controlled schema-guided dialogue settings.

A Supplementary Experimental Results … A.4 Pairwise Satisfaction and Trait Realization

The supplementary results cover task success, dialogue completion, system-quality scores, and pairwise satisfaction alongside trait-realization metrics. These analyses span models, domains, personality-access conditions, and the Hotel and Restaurant settings.

  • A.1 Task Success Rates: Table 4 reports task success rates across models, domains, and personality-access conditions.
  • A.1 Task Success Rates: The task-success analysis compares performance across the evaluated models and domains.
  • A.2 Dialogue Completion Rates: Table 5 reports dialogue-completion rates across all models, traits, domains, and personality-access conditions.
  • A.3 System-Quality Scores: Tables 6 and 7 report trait-level system-quality scores for the Restaurant and Hotel domains, respectively.
  • A.4 Pairwise Satisfaction and Trait Realization: Tables 8 and 9 report pairwise user-satisfaction deltas for the Hotel and Restaurant domains.
  • A.4 Pairwise Satisfaction and Trait Realization: Tables 8 and 9 also report averaged trait-realization metrics for the Hotel and Restaurant domains.

A.5 Representative Model- and Domain-Specific Score-Delta Figure · B Prompts · C Realization–Satisfaction Correlation: Full Details

The appendix supplies a representative GPT-4o Restaurant score-delta visualization, full quantitative tables, and prompt templates for personality adaptation and evaluation. It also specifies evidence-based procedures for measuring trait realization, satisfaction, system quality, and alignment with Big Five directions.

  • A.5 Representative Model- and Domain-Specific Score-Delta Figure: Tables 5–9 provide model-, domain-, trait-, condition-, and evaluation-specific breakdowns for completion, system quality, satisfaction, and realization metrics.They cover GPT-4o, Qwen3, and Gemini in Hotel and Restaurant settings under Neutral, Try, and Oracle conditions.
  • A.5 Representative Model- and Domain-Specific Score-Delta Figure: Figure 3 visualizes GPT-4o Restaurant-domain score changes from Neutral to Try and from Neutral to Oracle.The figure illustrates condition-wise variation across traits for one model-domain setting, complementing aggregated main-paper results and appendix tables.
  • B.1 Dialogue-Agent Prompt Templates: The Restaurant Oracle system prompt explicitly provides the user’s target personality trait and definition while requiring adaptive, helpful, grounded, schema-compliant responses.The prompt restricts generation to the next system utterance and specifies Restaurant-domain schemas and required and optional slots.
  • B.1 Dialogue-Agent Prompt Templates: The Restaurant prompt grounds information in API calls, required slots, schema-supported fields, and database results, prohibiting invented reviews, menus, ratings, or unsupported details.It also requires explaining empty search results and confirming key reservation slots before finalization.
  • B.2 User Trait Expression Evaluator Prompt: The trait evaluator identifies the three most evident user traits across the full dialogue and assigns each a 1–5 expression score using observable, evidence-based behavior.It evaluates only USER behavior and avoids inferring characteristics beyond what the user’s utterances support.
  • B.3 Pairwise User Satisfaction Evaluator Prompt: Pairwise satisfaction evaluation compares dialogues sharing the same domain, model, and intended trait, while excluding task success from the satisfaction judgment.The evaluator focuses on helpfulness, tone, responsiveness, patience, clarity, naturalness, emotional responsiveness, and communication-style fit.
  • B.4 System Quality Evaluation Prompt: System quality evaluation rates Constraint Satisfaction, Inform Rate, and Dialogue Truthfulness on a 1–5 scale using full-dialogue, evidence-based judgments.The criteria assess constraint fulfillment, correctly supported information, and avoidance of unsupported claims or fabricated information.
  • B.5 BFI Questionnaire Probe: The fixed BFI questionnaire probe separately tests whether each trait prompt aligns with its intended Big Five direction using trait-specific items and a 1–5 Likert scale.The probe is separate from generated task-oriented dialogues and returns structured trait responses.

C.1 Statistical Methodology · C.2 Per-Trait Summary

The analysis aggregates trait × model × domain cells and compares personality-aware conditions with Neutral using satisfaction deltas. It summarizes per-trait intended expression and satisfaction effects across six model–domain combinations, with trait-level Spearman correlations reported for n=10.

  • C.1 Statistical Methodology: Each data point represents one trait × model × domain cell summarizing 50 pairwise comparisons per condition.The unit combines a target trait, model, and domain.
  • C.1 Statistical Methodology: Satisfaction deltas measure win-rate differences between personality-aware and Neutral conditions across 50 binary outcomes.The deltas are denoted ∆O-N or ∆T-N.
  • C.1 Statistical Methodology: Avg. Intended Score is the target trait’s mean expression score averaged across Oracle, Try, and Neutral conditions.The score is computed for each trait × model × domain cell.
  • C.1 Statistical Methodology: Spearman rank correlation is used because the analysis is aggregated, need not assume linearity, and involves bounded, non-normal realization scores.Two traits, introversive and closed, have near-zero scores, while the other eight cluster near 4–5.
  • C.1 Statistical Methodology: Results are reported at the trait-averaged level with n=10 traits, averaging all six model–domain combinations.This averaging defines the trait-level analysis used for correlation results.
  • C.2 Per-Trait Summary: Table 10 reports per-trait Avg. Intended Score and average satisfaction deltas across all six model–domain combinations.The table provides the per-trait summary for the preceding methodology.

C.3 Disagreeable: System Mirroring in Qwen3 and Gemini

Under Oracle, Qwen3 and Gemini mirror disagreeable users’ antagonistic style, causing uncooperative responses and sharply lower satisfaction, whereas GPT-4o remains cooperative and task-focused. Pairwise comparisons are more informative than absolute ratings near the satisfaction floor.

  • System Mirroring: Under Oracle, Qwen3 and Gemini mirror disagreeable users’ antagonistic style, producing uncooperative responses and satisfaction drops of ∆O-N −56 to −92 pp across Hotel and Restaurant.GPT-4o avoids this failure, maintaining cooperative, task-focused behavior while acknowledging the user’s difficult style.
  • System Mirroring: Absolute User Satisfaction scores can appear slightly higher for Oracle than Neutral in some Qwen3 and Gemini cells despite pairwise preferences for Neutral.Near the rating floor, 1–5 absolute scales may not resolve relative differences that pairwise comparisons detect more clearly.

D Reproducibility Details

The experiments use a balanced design spanning models, domains, personality traits, and personality-access conditions, with 9,000 dialogue-level observations and 3,000 paired differences. Duplicate service entries are removed without adding records or modifying task schemas.

  • Experimental design: The balanced design covers three LLMs, two SGD domains, ten personality traits, and three personality-access conditions: Neutral, Try, and Oracle.Each model generates 50 dialogues per domain–trait–condition cell.
  • Experimental design: 9,000 dialogue-level observations are generated across all domain–trait–condition cells.The total comes from 50 dialogues per cell across 3 LLMs, 2 domains, 10 traits, and 3 conditions.
  • Experimental design: 3,000 paired differences support each pairwise condition comparison across model–domain–trait cells and both domains.Scores are paired within model–domain–trait cells using 50 × 3 × 2 × 10 paired observations.
  • Data processing: Duplicate or inconsistent service entries are removed using identifying fields without introducing new records or modifying task schemas.The deduplication fields include cuisine, restaurant_name, city, has_live_music, price_range, serves_alcohol, street_address, and phone_number.
Loading 2608.18085v1…