Source-linked AI summary
Conversation Coach: A Voice-enabled AI System that Helps Practice Difficult Workplace Conversations
Fanyou Wu, Suraj Maharjan, Ainur Yessenalina, Dennis Xu Chen, Rahul Srivastava, Srinivasan H. Sengamedu
TL;DR
Managers need scalable ways to rehearse difficult workplace conversations aloud, because traditional training is costly and text chat lacks realistic spoken practice. Conversation Coach compares end-to-end and cascaded voice architectures and deploys the cascaded system at organizational scale. The end-to-end approach provides lower latency and cost, while the cascaded approach provides stronger persona consistency and reasoning for complex coaching.
Problem
Traditional manager training is costly and difficult to scale, while text chatbots do not provide realistic spoken rehearsal for difficult conversations.
Method
Conversation Coach combines configurable employee personas, voice-first role-play, personalized content and policy feedback, and a comparison of end-to-end with cascaded speech architectures.
Results
3× lower median latency and 8× lower cost were achieved by NS2, while the cascaded pipeline offered superior persona consistency for complex coaching scenarios.
Takeaways & Limitations
Deployment to 40,000+ managers showed selective use for the most challenging conversations, supporting voice-first coaching as a just-in-time rehearsal format within this deployment.
Takeaways & Limitations
The study does not establish downstream coaching effectiveness and relies on 148 LLM-generated conversations scored by LLM judges, with human validation only on a subset.
Abstract
from arXiv · showhide
Effective manager-employee communication is critical for retaining high performers and developing underperformers, yet training managers in these skills remains costly. Text-based chatbots offer a scalable approach but cannot provide realistic rehearsal: managers need to practice speaking aloud to build confidence before high-stakes conversations. In this paper, we propose Conversation Coach, a voice-first AI system that enables managers to rehearse difficult workplace conversations in a realistic spoken format. The system addresses three challenges: achieving low-latency interactions with strong language understanding, enabling adaptive conversations through configurable bot personalities that simulate different employee types, and generating personalized feedback on content and policy compliance. We compare an end-to-end speech-to-speech model with a cascaded approach combining automatic speech recognition, a large language model, and text-to-speech synthesis. The end-to-end approach achieves 3$\times$ lower median (P50) latency with native barge-in capability at an estimated 8$\times$ lower cost, while the cascaded approach offers superior reasoning essential for coaching quality. We deployed the cascaded architecture in production, where 40,000+ managers used it over six months, with adoption patterns indicating selective use for difficult conversations.
1 Introduction
Conversation Coach is a voice-first system for realistic spoken rehearsal of difficult workplace conversations. It frames deployment and architecture choices around latency, reasoning, configurable simulations, and personalized feedback.
- Voice interaction lets managers practice speaking aloud in real time, unlike text chat, while introducing tradeoffs among latency, language understanding, and naturalness.The system targets difficult feedback, conflict navigation, and sensitive conversations that traditional training approaches make expensive and difficult to scale.
- The paper organizes coaching-relevant architectural tradeoffs across three dimensions to guide practitioners choosing between speech-to-speech and cascaded approaches.
- The production comparison evaluates Nova Sonic 2 against a cascaded Transcribe + Claude Sonnet 4.5 + Polly pipeline for latency, cost, and coaching quality.
- The organization-wide deployment reached 40,000+ managers, whose usage patterns indicated selective, just-in-time practice for difficult conversations.
- Conversation Coach combines configurable employee simulations, spoken conversation alternatives, and feedback on content, framing, and policy compliance.
2 Related Work
Prior work spans cascaded and end-to-end spoken dialogue systems, AI coaching, persona simulation, and real-time conversational dynamics. This paper contributes an empirical comparison of the two speech architectures in a deployed coaching application.
- Spoken dialogue systems use either cascaded ASR, LLM, and TTS pipelines or end-to-end models that process speech directly.
- End-to-end approaches promise lower latency and preserved paralinguistic cues, while optimized cascaded systems can retain stronger text-based reasoning with competitive latency.
- This work compares the two speech architectures within a deployed coaching application.
- Prior AI coaching and tutoring research includes spoken tutoring, executive coaching, mental-health coaching, and workplace communication training.
- Predictive turn-taking and emotional responsiveness are active research areas relevant to coaching applications, alongside typical spoken-system latency of 700–1000ms.
3 Design Principles
The design principles focus on making spoken coaching responsive, persona-consistent, and useful for post-conversation content feedback. They also frame architecture choice as a balance among these requirements.
- Building voice coaching requires addressing three interconnected challenges: real-time spoken interaction, adaptive persona simulation, and feedback generation.
- TTFA measures the delay from the end of a user’s speech until the system begins responding.
- End-to-end models avoid inter-component serialization and support native barge-in, whereas cascaded pipelines accumulate stage latency and require explicit interruption detection.
- Adaptive simulation requires instruction following and contextual consistency so the bot maintains its assigned employee persona across surprising user statements.
- Table 1 compares E2E speech-to-speech with a cascaded ASR + LLM + TTS pipeline based on production experience.
- Feedback analyzes content such as word choices, framing, policy compliance, and talking points; delivery features including tone and empathy remain a future extension.
4 System Architecture
Conversation Coach configures realistic employee role-play from scenario and employee context, supports two spoken conversation architectures, and generates structured content-focused feedback.
- The production system has pre-conversation, conversation, and feedback modules aligned with the paper’s design spaces.
- Managers configure sessions with scenarios, employee profiles, and behavioral parameters, while prompts combine organizational scenario context with employee history and feedback.
- Scenario Configuration: The typology defines three employee personality types: Direct / Assertive, Passive / Conflict Avoidant, and Analytical / Detail Oriented.
- Scenario Configuration: Crossing the personality types with Defensive / Deflective, Withdrawn, and Distressed reactions yields nine personality-reaction combinations.
- Nova Sonic 2 processes audio input to audio output in one model, avoiding inter-component serialization while supporting faster TTFA, native barge-in, and paralinguistic preservation.
- The cascaded implementation streams Transcribe, Claude Sonnet 4.5, and Polly stages, prioritizing reasoning quality while mitigating latency and handling interruptions through ASR monitoring.
- The feedback module analyzes word choices, framing, policy compliance, and talking points, then produces summaries, strengths, improvement areas, and actionable rephrasings.
5 Evaluation
The evaluation compares end-to-end NS2 with a cascaded ASR–LLM–TTS pipeline across latency, speech quality, reasoning, persona consistency, and coaching feedback. NS2 is faster and cheaper, while the cascaded approach performs better on complex persona behavior and supports reliable policy-violation detection.
- Evaluation design: The evaluation covers latency, persona consistency, speech quality, and coaching feedback across 148 synthetic scenarios spanning 27 configurations.The scenarios represent three employee profiles and nine personality–reaction types; 14 of 162 initial sessions were excluded for drift or premature termination.
- Latency: 1.4s versus 4.2s median TTFA (P50) makes NS2 3× faster than the cascaded approach.At P99, latency is 2.8s for NS2 versus 7.6s for the cascaded pipeline.
- Latency: 1.5s versus 7.0s TM99 latency confirms the NS2 advantage is consistent rather than driven by outliers.Both systems use streaming, but sequential processing in the cascaded pipeline produces higher first-audio latency.
- Persona consistency and reasoning: The cascaded approach has substantial persona-consistency advantages, especially for role adherence and authenticity, while differences are marginal on simpler conversational dimensions.The cascaded advantage is unanimous across four judges for role adherence and authenticity; NS2 remains effective for straightforward scenarios.
- Speech quality: Both architectures produce high-quality speech, while Transcribe achieves 5.1% WER on the evaluated manager turns.A matched per-turn NS2 WER measurement was not possible because its streaming transcript is not segmented by turn.
- Cost: $0.05 versus approximately $0.39 per typical 10-turn session gives NS2 an estimated 8× cost reduction.Generative TTS and the LLM’s growing context window are the primary cascaded cost drivers.
- Coaching feedback: F1 scores of 0.92–0.97 show high confidentiality-violation detection quality across four diverse classifiers, all with perfect recall.False positives mainly flagged indirect references as a compliance-sensitive over-caution.
6 Deployment and Usage Patterns
The cascaded coaching system was launched organization-wide for six months, and usage indicates substantive but selective rehearsal. Managers used it most around review deadlines and for employees involved in more challenging performance conversations.
- Deployment: The deployment was available to all managers during an annual review cycle over six months.The cascaded architecture was selected because persona consistency and reasoning quality were prioritized over latency.
- Adoption patterns: Over 3× higher adoption among newer frontline managers and a 22% return rate indicate strongest uptake among newer managers and repeat use.Median session duration was 135 seconds, supporting substantive engagement rather than superficial exploration.
- Adoption patterns: 4× above-average usage in the two weeks before review deadlines indicates just-in-time preparation rather than general skill development.Among sessions that began practice, 36% completed the full voice conversation and 23% of completers requested feedback.
- Selective use for difficult conversations: 43.3% of coached manager–employee pairs involved employees in the lowest performance tiers, nearly 2× their baseline rate.Managers prepared for only 17.6% of conversations on average, and 84% prepared for fewer than 25% of direct reports.
- Selective use for difficult conversations: Coached employees had 10.9 percentage points lower career-growth sentiment than noncoached teammates before the conversation.This within-team difference reflects pre-existing development challenges associated with managers’ selection of conversations for practice.
- Outcome analysis: The observed 4.3 percentage-point job-satisfaction difference between coached and control groups was not statistically significant.The sample was underpowered, with a minimum detectable effect of approximately 13 percentage points.
7 Discussion
The discussion frames architecture choice as application-dependent: end-to-end systems favor latency and interruptions, whereas cascaded systems favor complex reasoning and persona consistency. The authors also identify streaming requirements and longer-conversation persona drift as practical design considerations.
- Engineering lessons: Without end-to-end streaming, cascaded latency exceeds 8 seconds; full-pipeline streaming is required for acceptable conversational flow.TTS begins synthesizing as soon as the first LLM tokens arrive.
- Engineering lessons: NS2 may simplify complex persona instructions, whereas Sonnet follows detailed prompts but needs response-length and SSML engineering for natural speech.The systems therefore present different prompt-sensitivity and speech-generation challenges.
- Limitations: After 15+ turns, both architectures gradually weaken adherence to the assigned personality, although the cascaded approach is more robust.The authors attribute the greater robustness likely to stronger context-window utilization.
- Practitioner guidance: Choose end-to-end when sub-2-second latency or frequent natural interruptions is essential; choose cascaded when complex instruction following and reasoning are critical.The authors also suggest hybrid designs when applications need both properties and can tolerate added streaming-mitigated latency.
8 Conclusion
Conversation Coach enables managers to rehearse difficult workplace conversations in a realistic spoken format. The evaluated systems reveal a tradeoff between lower latency and cost versus superior persona consistency, while deployment suggests selective use for challenging conversations.
- Conversation Coach enables managers to rehearse difficult conversations through a voice-first spoken interaction.
- 3× lower latency and 8× lower cost characterize NS2 versus the cascaded pipeline, with native barge-in support.
- The cascaded pipeline offers superior persona consistency for complex coaching scenarios.
- 40,000+ managers used the deployed system, with adoption patterns consistent with selective use for the most challenging conversations.
Limitations
The paper’s evaluation has important limits involving downstream effectiveness, synthetic evaluation data, proprietary models, and the scope of feedback analyzed.
- The study does not establish that simulated practice improves real-world conversations.
- The architecture comparison relies on 148 LLM-generated conversations scored by LLM judges, with human validation only on a subset.
- The current system analyzes content but does not provide audio-based delivery feedback on tone, pace, or empathy.
- The evaluation covers only Claude Sonnet 4.5 and NS2, leaving generalization to open-source alternatives unexplored.
Ethical Considerations
The system addresses ethical risks through configurable personas, privacy protections, and positioning as a complement to human coaching, while simulated personas remain limited.
- Configurable persona parameters mitigate the risk of reinforcing particular management styles.
- Practice sessions are visible only to the individual manager, and audio recordings are not stored.
- Transcripts are retained for a maximum of 30 days before permanent deletion to comply with privacy regulations.
- The system is positioned as a complement to, rather than a replacement for, human coaching.
- Simulated personas do not capture the full range of human responses, particularly culturally or identity-informed communication norms.
B Employee Reaction Typology
The appendix defines prompts and evaluation procedures for simulating employee reactions, generating coaching feedback, detecting confidentiality violations, and judging role-play quality.
- The role-playing prompt assigns an employee identity, personality type, and reaction mode for an annual review conversation.
- The prompt requires consistent character behavior while guiding the conversation through review, feedback, development, and career-planning stages.
- The feedback prompt instructs the assistant to provide concise, actionable coaching focused on developmental feedback, goal alignment, and career planning.
- Feedback is structured around an overall summary, strengths, improvement areas, and an insufficient-context response for short conversations.
- The policy detector evaluates whether managers disclosed internal calibration ratings and whether generated feedback flagged the violation.
- The role-play judge scores nine evaluation categories from 1–5, including persona adherence, authenticity, directive compliance, and language patterns.