PaperlayerView exact revision on arXiv

Source-linked AI summary

CounselPlex: A Stressor-Aware Full-Duplex Counseling Model

Yeonju Kim, Bella Godiva, Yong Man Ro

arXiv:2610.32702v1cs.CLcs.SD

TL;DR

Full-duplex models can respond in real time, but counseling-oriented systems may not retain a seeker’s underlying stressor across turns. CounselPlex updates a concise stressor state during silence and injects it while listening, with RPSA supporting longer-turn performance. Evaluations report improved counseling quality and stressor grounding over strong baselines, alongside interaction trade-offs.

  • Problem

    Full-duplex models can focus on the latest utterance without maintaining and using a seeker’s underlying stressor across turns, a gap that matters in counseling.

    Source: §1 · p. 1

  • Method

    CounselPlex generates or updates a concise stressor state and injects it into the text stream while listening, with RPSA broadening positional coverage during training.

    Source: §1 · pp. 1–2

  • Results

    CounselPlex improves all five counseling dimensions over the in-domain PersonaPlex† baseline on ESConv and ESCoT, and narrows the stressor-verbalization gap to reference responses.

    Source: §6 · p. 7, §7 · p. 7

  • Takeaways & Limitations

    The results motivate further study of explicit, persistent stressor grounding in counseling-oriented conversational agents.

    Source: §7 · p. 7

    The evaluation does not establish clinical effectiveness: speech is synthesized from text-based datasets and quality is judged primarily from text outputs.

    Source: §6 · p. 7

Abstract

from arXiv · show

Full-duplex speech models can listen and speak at the same time, enabling natural, low-latency interaction. In counseling, however, these models often focus on the latest utterance and lose track of the seeker's underlying problem. We call this failure Stressor Blindness. We propose CounselPlex, a stressor-aware full-duplex framework that uses two operations. CounselPlex updates a short stressor state during seeker silence and injects this state into <PAD> positions while listening. We also introduce RoPE Position Shift Augmentation (RPSA), which broadens positional coverage during training and improves performance at longer turn depths in our evaluation. LLM-judge evaluations show improved counseling quality and stressor grounding over strong baselines.

1 Introduction

Role-conditioned full-duplex models can sustain natural interaction but do not specify which seeker-specific information to retain, leaving counseling models vulnerable to Stressor Blindness. CounselPlex maintains a concise stressor state, updates and injects it during listening, and is evaluated for counseling quality and duplex behavior.

1 Introduction

  • Stressor Blindness occurs when a model fails to maintain and use a seeker’s active stressor across turns, responding generically to a later symptom.In the example, the baseline treats feeling constantly on edge as general anxiety eight turns after trauma was disclosed, while CounselPlex links it to the trauma.Source: §1 · p. 1, Figure 1 · p. 1
  • CounselPlex detects changes in the seeker’s situation, generates a new stressor state when needed, and injects the active state into the text stream while listening.The stressor state is a concise 1–7 word noun phrase describing the seeker’s underlying problem.Source: §1 · pp. 1–2
  • The paper evaluates counseling quality with GPT-4.1 ratings across five dimensions, alongside turn-taking, latency, overlap handling, and stressor-state analyses.The rated dimensions are empathy, coherence, context recalling, strategy appropriateness, and stressor awareness.Source: §1 · p. 2
  • LLM-judge evaluations, duplex benchmarks, and stressor-state analyses show improved counseling quality and stressor grounding over strong baselines, with a modest latency increase over the fine-tuned baseline.The paper also introduces RPSA to broaden positional coverage and improve long-context stability.Source: §1 · p. 2

2 Related Work

Emotional-support research has moved from rule-based and neural dialogue agents toward explicit support strategies and LLM-based multi-turn support. Most such systems remain text-based and turn-taking, while full-duplex speech models support simultaneous listening and speaking but face declining performance in longer conversations.

2.1 Emotional Support Conversation

  • Emotional-support systems progressed from rule-based agents to neural empathetic dialogue, explicit strategies in ESConv, and LLM systems for multi-turn support.Recent work also uses data augmentation and explicit reasoning to improve strategy use.Source: §2.1 · p. 2
  • Most emotional-support systems are text-based and assume turn taking, so they do not model real-time overlap, interruptions, or backchannels.CounselPlex instead focuses on counseling-context tracking in a full-duplex spoken setting.Source: §2.1 · p. 2

2.2 Full-Duplex Speech Language Models

  • Full-duplex speech language models listen and speak simultaneously, with work addressing streaming, interaction control, emotional prosody, and interaction benchmarks.Multi-round evaluations report declining performance as conversations lengthen.Source: §2.2 · p. 2
  • CounselPlex remains end-to-end by injecting a compact stressor state directly into the text stream to retain counseling context.This contrasts with cascaded systems such as FireRedChat, which support external modules but may add latency.Source: §2.2 · p. 2

3 CounselPlex: Stressor-State Augmented Full-Duplex Model

CounselPlex is trained on spoken emotional-support dialogues labeled with core stressors and uses a full-duplex backbone to generate and inject stressor states. Its training objective weights text and audio losses, while RPSA broadens positional exposure for long conversations.

3.1 Data Construction & Stressor Labeling

  • The spoken corpus converts ESConv and ESCoT text into 80,034 utterance-level WAV files, covering 3,009 conversations and 158.3 hours of speech.GPT-4.1 extracts 1–7 word core-stressor phrases from factual disclosures; 127 conversations (4.22%) without an external stressor are left unlabeled.Source: §3.1 · p. 2

3.2 Overall Architecture

  • CounselPlex uses PersonaPlex as its backbone and generates or updates a stressor state before injecting the active state while listening to a later seeker turn.PersonaPlex is chosen because its separate streams support simultaneous listening and speaking.Source: §3.2 · p. 2

Generating New Stressor State

  • After 0.5 seconds of seeker silence, CounselPlex uses dialogue history to decide whether to generate a new stressor state, retaining the current state otherwise.The silence threshold is intended to avoid treating short within-sentence pauses as turn endings.Source: Generating New Stressor State · p. 2

Injecting Stressor State

  • When a later seeker turn is confirmed, CounselPlex injects the active stressor into the agent’s <PAD> positions rather than regenerating it.The system requires sufficient speech activity within a 0.5-second window to avoid treating brief backchannels or noise as full turns.Source: Injecting Stressor State · p. 2

Total Loss

  • The overall training objective sums text and audio losses, while text loss separately weights think, pad, and speech tokens.Think tokens receive a lower weight than speech tokens because equal weights cause stressor states to be generated too often; the system excludes the injection window from text loss.Source: Total Loss · pp. 2–3

RoPE Position Shift Augmentation (RPSA) for Long-Context Stability

  • RPSA broadens positional exposure by randomly shifting the absolute RoPE indices of cropped long-conversation segments that begin after 750 frames, approximately 60 seconds.With probability 0.7, it applies the same offset to every position, preserving dialogue order and relative spacing.Source: RoPE Position Shift Augmentation… · p. 3

3.4 Real-Time Inference

  • At inference, the model updates stressor state after a seeker turn and injects the stored state when sufficient activity confirms the next turn.The agent’s generated acoustic tokens are continuously decoded into a 24 kHz waveform.Source: §3.4 · pp. 3–4

4 Experimental Setup

The experiments fine-tune a full-duplex backbone on synthesized emotional-support dialogues and compare CounselPlex with end-to-end streaming baselines. Evaluation covers counseling quality and real-time interaction behavior.

4.1 Implementation Details

  • The synthesized ESConv and ESCoT corpus contains 158.3 hours across 3,009 sessions, with 2,107 training, 367 validation, and 535 test dialogues.Dialogues average 3.1 minutes and reach 13.0 minutes; turns average 16.5 seconds including silence.Source: §4.1 · p. 4
  • CounselPlex inherits the approximately 8.4B-parameter PersonaPlex/Moshi backbone and is fine-tuned for four epochs in a single training run.Training costs approximately 45 GPU-hours; RPSA randomly shifts long-context crops starting after 750 frames, and random 0–5-second pauses follow supporter turns.Source: §4.1 · p. 4

4.2 Baseline Models & Evaluation Setup

  • Comparisons include Moshi, zero-shot PersonaPlex, and fine-tuned PersonaPlex†; evaluation uses GPT-4.1 counseling ratings and Full-Duplex-Bench interaction tests.The counseling rubric covers five dimensions and Response Rate; Full-Duplex-Bench tests pause handling, backchanneling, turn-taking, and interruption management.Source: §4.2 · p. 4

5 Results and Analysis

CounselPlex leads the counseling-quality comparisons and maintains stressor grounding, with RPSA supporting performance through the first 30 turns. Its interaction profile is conservative, slower, and lacks detected backchannels; long-range grounding remains incomplete.

5.1 Counseling Quality Assessment

  • CounselPlex scores highest on all five counseling-quality dimensions on both datasets, exceeding PersonaPlex† by +0.34 Avg5 on ESConv and +0.17 on ESCoT.The matched-turn analysis also ranks CounselPlex first; zero-shot PersonaPlex is statistically indistinguishable on those turns, while the overall lead mainly reflects answering more turns. In a human study, CounselPlex was preferred over PersonaPlex† in 61.2% of non-tie judgments.Source: §5.1 · pp. 4–5, Table 1 · p. 5

5.2 Duplex Interaction Trade-offs

  • CounselPlex has low Take-Over Rates during pauses and responds in 99.2% of smooth turn-taking examples, but with 0.988 s average latency.It responds after nearly all interruptions but is slower than zero-shot PersonaPlex, and no backchannels are detected; adding backchannel cues raises Take-Over Rates.Source: Table 2 · p. 5, §5.2 · p. 5

5.3 Ablation Studies

  • Removing either RPSA or stressor injection lowers Response Rate and counseling quality below the in-domain fine-tuned baseline.Generating a think trace without reinjecting it is insufficient in this configuration; the complete model achieves the best scores.Source: §5.3 · p. 5, Table 5 · p. 6

5.4 Analysis of Stressor Reasoning

  • CounselPlex generates new stressor states on 11.3% of engaged turns, and a judge finds 92.5% of its 666 generated states correct.On reused-stressor turns, its Avg5 gap to ground-truth responses is 0.46 versus 0.99–1.00 for baselines; stressor verbalization is 36.2% versus 45.9% for ground truth.Source: §5.4 · p. 6

5.5 Long-Context Stability & RPSA Effect

  • RPSA broadens training RoPE-position coverage, raising normalized entropy from 0.66 to 0.85; on combined ESConv and ESCoT test sets, it maintains higher counseling quality and Response Rate than without RPSA over the first 30 turns.Those turns contain 87.1% of test turns, and a residual gap remains beyond turn 30.Source: Figure 3 · p. 6, §5.5 · p. 6, Figure 4 · p. 6

5.6 Partial Long-Range Grounding

  • After 16 turns, CounselPlex reconnects a response to an earlier family conflict, but asks about already disclosed details while the reference follows the immediate meditation discussion.The example illustrates topic-level stressor persistence without complete context recall or an unambiguously better response.Source: Table 7 · p. 7, §5.6 · p. 7

6 Discussion

CounselPlex’s counseling gains extend beyond Stressor Awareness, but its component contributions, duplex trade-offs, and intended scope remain qualified. The analyses also show that multiple-stressor references occur without establishing that the active state prioritizes them.

Counseling gains beyond explicit stressor mentions

  • CounselPlex improves all five counseling dimensions over PersonaPlex† on both datasets, with the same ranking under Gemini and human preferences favoring it in 61.2% of non-tie comparisons.The evaluation does not isolate why each dimension improves.Source: §6 · p. 7

What the ablations establish

  • Removing either RPSA or stressor injection lowers Response Rate and counseling quality below PersonaPlex†, while trace generation without reinjection is insufficient in this setup.RPSA’s gains cannot be explained by fine-tuning alone, but the ablations do not establish independent failure modes.Source: §6 · p. 7

Duplex interaction trade-offs

  • Low pause Take-Over Rates coexist with responses in 99.2% of smooth turns, slower responses, lower interruption-response scores than zero-shot PersonaPlex, and no detected backchannels.Whether this trade-off is preferable in counseling requires domain-specific evaluation.Source: §6 · p. 7

Behavior with multiple stressors

  • Among 80 conversations with a second stressor, 52 (65.0%) reference an earlier stressor and 38 (47.5%) reference both, but the descriptive analysis has no controlled baseline.It therefore does not establish that the active state functions as a priority mechanism.Source: §6 · p. 7

Scope

  • The experiments assess stressor grounding and interaction behavior, not clinical effectiveness or therapeutic outcomes, so CounselPlex is positioned as emotional support rather than a clinically validated counseling system.Speech is synthesized from text-based emotional-support datasets, and quality is evaluated primarily from text-stream outputs using LLM judges.Source: §6 · p. 7

7 Conclusion

CounselPlex combines persistent stressor grounding with RPSA and improves counseling and stressor verbalization in the reported evaluation. The conclusion and accompanying analyses qualify those gains through interaction trade-offs, dataset constraints, and case-study limitations.

7 Conclusion

  • CounselPlex outperforms strong baselines across five counseling dimensions and narrows the stressor-verbalization gap to reference responses, while RPSA improves performance at longer turn depths.Full-Duplex-Bench shows low pause Take-Over Rates alongside slower responses and no detected backchannels.Source: §7 · p. 7

A Stressor Labeling Prompt and Examples

  • The stressor-labeling prompt represents each state as a 1–7 word noun phrase and replaces the prior state when the seeker’s main focus changes.It distinguishes concrete situations from temporary emotional symptoms and does not accumulate multiple stressors.Source: A Stressor Labeling Prompt and Examples · p. 10

B Spoken-Corpus Construction

  • The study uses ESConv and ESCoT and synthesizes their text-only conversations into fixed two-voice speech, a design that does not represent real counseling speech diversity.The derived audio, annotations, and full preprocessing pipeline are not currently distributed.Source: B Spoken-Corpus Construction · pp. 10–11

C Strategy Selection and Diversity Analysis

  • Under the eight-class ESConv taxonomy, CounselPlex uses Question most often (37.3%) and is more diverse than zero-shot baselines, while GT is most balanced (Hnorm = 0.899).The strategy distributions are descriptive and do not isolate stressor reasoning as the sole cause of model differences.Source: C Strategy Selection and Diversity… · p. 11, Table 9 · p. 11

D Latency Breakdown of Stressor Reasoning

  • New-update turns have higher observed mean onset latency than turns with no active stressor: 1.668 s versus 1.320 s with Silero and 1.192 s versus 0.828 s with Whisper.Turn types occur naturally rather than being randomly assigned, and this protocol’s absolute latencies are not directly comparable with Full-Duplex-Bench; the 666 traces average 2.62 words and 4.98 tokens including <think> tokens.Source: D Latency Breakdown of Stressor… · pp. 11–12

E Case Studies

  • Across five case studies, CounselPlex names underlying tensions, follows emotional direction, and links current feelings to earlier stressors, but examples also show it can miss emotional shifts or details.The cases compare CounselPlex with baselines and reference responses after dialogue shifts toward symptoms or coping.Source: E Case Studies · p. 12, Table 10 · p. 12, Table 11 · p. 12

F Failure Case Analysis

Case analyses show that CounselPlex can retain a stressor while still responding poorly to emotional shifts, losing details, or preserving an incorrect state. The accompanying evaluation prompts assess counseling quality, stressor engagement, and trace accuracy using distinct criteria.

F Failure Case Analysis

  • The four failure cases show that correct stressor retention does not ensure an appropriate strategy, compressed traces lose details, and an incorrectly extracted stressor can persist without automatic correction.The examples include returning to caregiving after a hopeful shift, analyzing financial impact instead of distress, redundant questioning, and a resource-recommendation loop.Source: F Failure Case Analysis · p. 14, Table 15 · p. 14, Table 16 · p. 15

G LLM-as-a-Judge Evaluation Prompt

  • The GPT-4.1 prompt scores five counseling dimensions on a 1–5 rubric and asks the judge to infer the core stressor from the full dialogue before scoring Stressor Awareness.The prompt also requests a strategy label from the 14-class ESCoT taxonomy, mapped to eight classes.Source: Table 18 · p. 16, G LLM-as-a-Judge Evaluation Prompt · p. 16

H Stressor Verbalization Analysis Prompt

I Stressor Accuracy Prompt

J Cross-Judge Validation

A second judge preserves CounselPlex’s leading model ranking, though its scores are higher overall and shared LLM-judge biases remain possible. Response-rate checks qualify matched-subset comparisons, while evaluation dimensions overlap without being identical.

J Cross-Judge Validation

  • CounselPlex ranks first under both GPT-4.1 and Gemini and leads PersonaPlex† by +0.23 under each judge, although Gemini’s model-level Avg5 scores are 0.23–0.32 higher overall.The unchanged ordering provides evidence that the main ranking is not unique to GPT-4.1, but does not remove biases shared by LLM judges.Source: J Cross-Judge Validation · pp. 17–18, Table 21 · p. 18

K Response-Rate Normalization and Matched-Response Subset

  • CounselPlex ranks first on engaged-only and matched turns, but the matched subset favors low-response-rate baselines; even there, it leads PersonaPlex† 2.812 to 2.537.The matched comparison uses 3,472 turns answered by all four models, and zero-shot baselines score worse outside that subset.Source: K Response-Rate Normalization and… · p. 18, Table 22 · p. 18

L Correlation Across Evaluation Dimensions

  • Across reference responses, the five evaluation dimensions correlate from 0.69 to 0.90, indicating overlap without identity; Stressor Awareness remains separately reported for its focus on engagement with the underlying situation.Empathy and Strategy correlate most strongly (0.90), while Context Recalling and Stressor Awareness correlate at 0.88.Source: L Correlation Across Evaluation… · p. 18, Table 23 · p. 18

M Per-Turn-Bin RPSA Results

RPSA improves Response Rate and Avg5 across turn-depth bins, with larger Avg5 gains in the deepest bins. In a small exploratory preference study, raters favored CounselPlex, but the result was not statistically conclusive.

M Per-Turn-Bin RPSA Results

  • Avg5 gain rises from +0.12 at turns 1–10 to +0.37/+0.38 at 31+; RPSA improves Response Rate and Avg5 in every bin.The two variants use one seed and are directly comparable.Source: M Per-Turn-Bin RPSA Results · p. 18
  • The two deepest bins contain only 470 and 127 turns, and both variants still degrade in that range.Turns 1–30 account for 87.1% of the data; reported standard deviations are across turns, not training seeds.Source: M Per-Turn-Bin RPSA Results · p. 18

N Exploratory Human Preference Study

  • CounselPlex was preferred in 61.2% of non-tie judgments over PersonaPlex† (52/85), but the participant-level sign test was not conclusive (p=0.0625).Agreement was slight (Fleiss’ κ=0.139), and the study used five participants and 20 response pairs.Source: N Exploratory Human Preference Study · pp. 18–19, Table 25 · p. 19
  • Each of five participants preferred CounselPlex more often, but the 20-pair study supports qualitative evidence on selected cases, not population-level preference claims.Written justifications most often associated preferences for CounselPlex with coherence and naturalness.Source: N Exploratory Human Preference Study · pp. 18–19

O Multi-Stressor Handling

Across multi-stressor conversations, CounselPlex sometimes refers to earlier stressors from dialogue context, even while maintaining one active stressor state. A backchannel-augmented variant produces backchannels and improves JSD, but increases unwanted takeovers and reduces turn-taking latency.

O Multi-Stressor Handling

  • CounselPlex keeps one active stressor, and combining factors expressed at the same update point does not test retention across stressors disclosed at different times.In one example, it addresses current disruptive roommates and earlier school-exam pressure in the same response.Source: O Multi-Stressor Handling · p. 19
  • In 52 of 80 conversations (65.0%), responses referred to a prior stressor; 38 of 80 (47.5%) included a response engaging both current and prior stressors.Across evaluated turns, 27.6% referred to a previous stressor; earlier stressors remain in dialogue context.Source: O Multi-Stressor Handling · p. 19, Table 26 · p. 19

P Backchannel-Augmented Variant

  • The backchannel variant inserts synthesized “mm-hmm” cues during listening in both directions with probability 0.5.The turn-structured training data contains no overlapping backchannel supervision.Source: P Backchannel-Augmented Variant · p. 19
  • CounselPlex + BC raises backchannel Freq from 0.000 to 0.083 and improves JSD from 1.000 to 0.738, but backchannel TOR rises to 0.982.Smooth turn-taking latency falls to 0.000, interruption appropriateness drops to 1.605, and the variant speaks over the seeker more often.Source: P Backchannel-Augmented Variant · p. 19

Q Reproducibility Details

Turn-level tests find CounselPlex’s Avg5 scores differ significantly from every baseline across all test turns, while matched-turn comparisons distinguish it from only some baselines. Reproducibility is constrained by a single training trajectory, incomplete development records, and unavailable artifacts.

Q Reproducibility Details

  • The reported checkpoint comes from one training trajectory, so seed variance is uncharacterized; the training seed was not archived, and the hyperparameter search log is incomplete.The derived corpus, full training and evaluation code, and weights are not publicly released, preventing end-to-end reproduction.Source: Q Reproducibility Details · pp. 19–20
  • Reproduction records list principal training and decoding settings and archived software versions, but omit the CPU model and NVIDIA driver; unlisted settings retain implementation defaults.The final settings are summarized in Table 28.Source: Q Reproducibility Details · p. 20, Table 28 · p. 20

R Statistical Significance

  • Across all 6,278 test turns, CounselPlex’s Avg5 differs significantly from every baseline (p<0.05); on 3,472 matched turns, it differs from PersonaPlex† and Moshi, but not zero-shot PersonaPlex (p=0.49).The comparisons use two-sided paired Wilcoxon signed-rank tests pairing responses by conversation and turn.Source: R Statistical Significance · p. 20
  • On matched turns, fine-tuning without stressor reasoning lowers quality versus zero-shot PersonaPlex (2.537 vs. 2.796), while CounselPlex recovers zero-shot-level quality and answers more turns overall.The matched-subset test does not distinguish CounselPlex from zero-shot PersonaPlex (p=0.49).Source: R Statistical Significance · p. 20
  • The tests treat turns as units despite within-conversation clustering, so their nominal p-values may overstate precision and serve as turn-level diagnostics, not sole evidence for model ranking.The signed-rank test is not applied to Full-Duplex-Bench metrics.Source: R Statistical Significance · p. 20

S Limitations

The paper identifies constraints in stressor representation and long-term grounding, latency, data realism, evaluation, reproducibility, and safety coverage. These boundaries limit what the reported results establish and where the model should be used.

Reasoning constraints and hallucinations

  • One active stressor limits representation of overlapping problems; the model cannot assign an unstated external stressor, and inaccurate traces can hallucinate details that bias later responses.No external stressor was stated in 127 conversations (4.22%) in the data.Source: S Limitations · pp. 20–21

Limits of long-term grounding

  • Continuous injection does not fully preserve grounding when stressors leave immediate focus; the quality gap to GT grows from −0.11 on new-stressor turns to −0.46 on reused turns.RPSA stabilizes only the earlier conversation, with performance still declining after the first 30 turns.Source: S Limitations · p. 21

Latency trade-offs

  • Latency is 0.988 s for smooth turn-taking and 1.056 s after interruption; this slower interaction profile reduces overlap and may not suit time-sensitive applications.The delay may partly reflect the 0.5-second silence window and stressor reasoning.Source: S Limitations · p. 21

Data realism and evaluation bias

  • Synthesized data uses only two English voices and lacks real-counseling disfluencies, prosodic variation, language coverage, and speaker diversity; GPT-family evaluation also risks self-preference bias.The paper calls for evaluation with clinicians and more realistic speech.Source: S Limitations · p. 21

Evaluation limitations

  • A single random-seed checkpoint limits performance-variance estimates; the 20-pair Prolific study cannot support population-level preference claims, and unavailable corpus and preprocessing artifacts may limit replication.Principal prompts and settings are documented.Source: S Limitations · p. 21

Safety-critical deployment

  • Evaluation does not cover suicidal ideation, self-harm, or active abuse; the authors advise against standalone deployment without rigorous, explicit crisis-detection protocols.Source: S Limitations · p. 21

Related papers

Loading 2610.32702v1…