When Less Is More: An Empirical Study of Minimal Responses in Counseling Dialogues and the Behavior of LLMs

Zhiyang Qi

Source-linked AI summary

When Less Is More: An Empirical Study of Minimal Responses in Counseling Dialogues and the Behavior of LLMs

Zhiyang Qi

arXiv:2608.24080v1Aug 25, 2026cs.CLcs.AI

TL;DR

Counseling dialogue research often overlooks minimal responses because datasets, models, and evaluators favor content-rich replies. This paper analyzes such responses across multilingual datasets and evaluates model generation in curated contexts. Minimal responses are common in human-collected data, explicit instructions improve generation, and current quality evaluation may undervalue interactionally appropriate brevity.

Problem
Counseling systems and evaluation frameworks often favor explicit, content-rich replies, overlooking brief responses that support attentive, empathic interaction.
Method
The paper analyzes minimal responses across seven multilingual counseling datasets using length/content filtering followed by LLM contextual verification, then evaluates multiple models in curated minimal-response contexts.
Results
Minimal responses are common in human-collected data but rare in synthetic data; explicit instruction enables generation, while LLM-based quality scores may undervalue appropriate minimal responses.
Takeaways & Limitations
Minimal responses should be incorporated into counseling data construction, model training, and response evaluation.
Takeaways & Limitations
Experiments use selected local contexts and do not test minimal responses in complete dynamic counseling sessions or establish therapeutic appropriateness through expert or client evaluation.

Abstract

from arXiv · show

In psychological counseling, effective support is not always delivered through long, information-rich responses. Minimal responses, such as backchannel cues and concise empathic statements, help convey attentive listening, express empathy, and encourage clients to continue expressing themselves. However, existing counseling dialogue systems and evaluation frameworks often favor explicit, content-rich replies, overlooking the interactional value of brief counselor utterances. This paper presents a systematic cross-lingual analysis of minimal responses across multiple counseling dialogue datasets. We develop a two-stage filtering method based on utterance length and content, followed by contextual verification using a large language model (LLM). Our analysis shows that minimal responses are common in human-collected datasets but substantially underrepresented in LLM-generated ones. We further evaluate current LLMs in manually curated dialogue contexts where human counselors used minimal responses. The results show that strong commercial LLMs are capable of generating minimal responses when explicitly instructed, but still struggle to determine when such responses are appropriate. Counseling-specific models trained on synthetic data perform particularly poorly, tending instead to produce longer and more information-rich responses. Moreover, LLM-based response-quality evaluation may undervalue minimal responses, even when they are interactionally appropriate.

1 Introduction

The introduction argues that counseling systems and evaluators often overlook minimal responses, despite their interactional value, and presents a cross-lingual analysis addressing this gap.

Real counseling commonly uses minimal responses, but current LLM-based evaluation may undervalue them because of their brevity.
Minimal responses such as “uh-huh” and “I see” convey empathy, acceptance, and attentiveness while encouraging clients to continue speaking.
Existing counseling datasets and evaluation methods often favor informative, structurally complete replies over brief interactionally meaningful utterances.
The paper analyzes minimal responses across seven Chinese, Japanese, and English counseling datasets using a cross-lingual, cross-dataset approach.
Minimal responses are substantially more common in human-collected datasets than in LLM-generated datasets.
The study evaluates multiple models in minimal-response contexts and finds that explicit instruction helps generation, while model selection and evaluation remain problematic.

2 Dataset Analysis

This section identifies minimal responses across seven multilingual counseling datasets with rule-based filtering followed by LLM contextual verification, finding them far more prevalent in human-collected data.

The analysis covers seven Chinese-, Japanese-, and English-language datasets, including four LLM-generated and three human-collected resources.
The first filtering stage screens counselor utterances by length, content, and the length of the immediately following client utterance.
Counselor utterances are limited to 15 Chinese or Japanese characters or 10 English words, with longer following client utterances required.
GPT-5.4-mini then classifies candidates as backchannel-like, brief empathic or reflective, or other short responses.
Validation rates exceeded 94% for every category across the three human-collected datasets.
Minimal responses are almost absent from LLM-generated datasets but occur substantially more often in all three human-collected datasets.

3 Experiment

The experiments test whether models generate minimal responses in manually selected multilingual contexts and whether evaluators recognize their interactional appropriateness. Models respond better to explicit instructions than to contextual demands, while quality scoring favors richer replies.

Experimental setup: The study evaluates models on manually selected Chinese, Japanese, and English contexts where human counselors used minimal responses.
Experimental setup: General prompting tests whether models naturally select minimal responses, whereas instructional prompting explicitly prioritizes them while clients continue expressing themselves.
Evaluation: The evaluation measures minimal-response rate, interruption likelihood on a 0–5 scale, and response quality across multiple dimensions.
Findings: Models trained on human counseling data generate minimal responses more often than models trained on synthetic data.
Findings: GPT-5.4’s MR rate rises from 0.00% to 97.18% in Chinese and from 0.00% to 82.00% in Japanese under explicit instruction.
Findings: For GPT-5.4 and Llama-3, instructional prompting improves interruption scores but reduces Quality Avg., indicating that evaluation favors content-rich responses.

4 Related Work

Prior work treats minimal responses as therapeutically meaningful but has not made them a primary focus of counseling dialogue research. This paper addresses that gap by analyzing their distribution and generation across datasets, models, and prompting strategies.

Backchannels signal attention, understanding, or agreement without taking the conversational floor.
Counseling minimal responses additionally express empathy, acceptance, and attentiveness while allowing clients to continue exploring their experiences.
Prior work includes related labels such as Grounding and Minimal Encouragement, but treats them as one of many dialogue-act or annotation categories.
Little is known about minimal-response representation in counseling datasets or whether counseling systems can generate these responses appropriately.

5 Conclusion

The paper finds that minimal responses are common in human-collected counseling data but rare in synthetic datasets, where trained models struggle to generate them. General-purpose LLMs can produce them when instructed, yet do not reliably judge when they are appropriate, while conventional evaluation tends to undervalue them.

Minimal responses are common in human-collected data but rare in synthetic datasets.
Models trained on synthetic data struggle to generate minimal responses.
General-purpose LLMs can produce minimal responses when explicitly instructed but do not reliably determine when they are appropriate.
Conventional response-quality evaluation tends to undervalue minimal responses.
These findings highlight the need to incorporate minimal responses more effectively into counseling data construction, model training, and evaluation.

Limitations

The study’s comparisons and evaluation are constrained by dataset heterogeneity, the preliminary status of LLM-based evaluation, and testing limited to selected local contexts rather than complete counseling processes.

Cross-dataset comparisons may reflect differences in collection modality, counselor characteristics, interaction styles, and therapeutic orientations.
LLM-based evaluation is a preliminary, scalable proxy rather than a substitute for expert or client evaluation.
Human counselor responses provide a reference but do not establish generated responses’ therapeutic appropriateness or client-perceived helpfulness.
Experiments examine minimal responses only in selected local contexts, without testing complete dynamic counseling processes or optimal timing and frequency.
Excessive or repetitive minimal responses may feel mechanical or irritating rather than supportive.

Ethical Considerations

The study uses licensed public counseling datasets that had undergone anonymization or privacy screening, analyzes them only in aggregate, and does not deploy systems for actual counseling. The authors caution that the findings do not establish that LLMs can provide professional counseling.

The study uses existing publicly available counseling datasets under their respective licenses.
The datasets had undergone anonymization or privacy screening before public release, to the authors’ knowledge.
The analysis is aggregate-level and does not identify or interpret individual clients.
The study does not collect new data from real clients or deploy systems for actual counseling.
The findings should not be interpreted as showing that LLMs can provide professional counseling.

A Details of Minimal-Response Identification

Minimal responses are identified through rule-based filtering followed by contextual LLM classification. The pipeline uses utterance length, content exclusions, and surrounding dialogue to distinguish backchannels and brief empathic responses from other short utterances.

Adjacent utterances from the same speaker are merged before candidate identification.
Rule-based filtering retains short counselor turns when the following client turn exceeds a language-specific minimum length.The following client utterance provides a heuristic signal that the client continued elaborating after the brief response.
Content rules exclude short questions, prompts, directives, suggestions, thanks, apologies, greetings, closings, logistical expressions, and other formulaic utterances.These responses do not primarily acknowledge the client’s ongoing narrative or leave the conversational floor open.
Table 3 specifies language-specific maximum counselor-utterance and minimum next-client-utterance lengths for rule-based filtering.
GPT-5.4-mini classifies retained candidates using dialogue history, the candidate utterance, and the next client utterance.It assigns backchannel-like, brief empathic or reflective, or other short-response labels.

B.1 Models and Inference Prompts

The experiments compare general, commercial, counseling-oriented, and human-data-fine-tuned models under inference-only settings. General models receive either a history-only prompt or an instructional prompt encouraging concise minimal responses, while evaluation uses multiple LLM judges and established quality dimensions.

B.1.1 Models.: The compared systems include general open-source models, GPT-5.4, counseling-domain fine-tuned models, and human counselor responses.All model-based experiments use inference only, without parameter updates or additional fine-tuning.
B.1.1 Models.: Qwen3-8B serves as the Chinese general open-source baseline without counseling-specific fine-tuning.
B.1.1 Models.: The Japanese model set includes Llama 3.1 Swallow as a general-purpose baseline and PsyDial-Pi4 as a PsyDial-D4-fine-tuned model.Source dialogues overlapping evaluation examples were removed before retraining PsyDial-Pi4.
B.1.1 Models.: The English baseline is Meta-Llama-3-8B-Instruct, evaluated on counseling texts extracted from AnnoMI.
B.1.2 Retraining of human-data fine-tuned models.: KokoroChat-Full is retrained after removing overlapping source dialogues, leaving 6,213 dialogues split into 5,592 training and 621 validation dialogues.This produced 192,598 training instances and 21,668 validation instances.
B.1.2 Retraining of human-data fine-tuned models.: PsyDial-Pi4 is retrained after removing 273 evaluation-source dialogues, with 2,109 remaining dialogues divided into 1,898 training and 211 validation dialogues.
B.1.3 Inference prompts.: General models and GPT-5.4 are tested with a general prompt and an instructional prompt encouraging concise minimal responses during ongoing client expression.Figure 4 presents the response-generation templates.
B.2 LLM-based Evaluation Prompts: GPT-5.4-mini and Gemini-3.1-Flash-Lite evaluate minimal-response function, interruption risk from 0 to 5, and response quality across comprehensiveness, professionalism, authenticity, and safety.Responses are scored together within each dialogue history and averaged across the two evaluators.

C Relationship Between LLM Quality Scores and Minimal Responses

General response-quality scores are strongly associated with interruption risk and minimal-response frequency in the opposite direction expected for interactionally appropriate minimal responses. The evaluation rubric may therefore undervalue brief responses because it emphasizes comprehensiveness and elaboration.

Pearson r = 0.89 and Spearman ρ = 0.89 link higher quality scores with higher interruption scores.Conditions judged more likely to interrupt the client tend to receive higher general quality ratings.
Pearson r = −0.86 and Spearman ρ = −0.92 link higher quality scores with lower minimal-response rates.Conditions producing minimal responses more frequently tend to receive lower ratings under the general quality rubric.
General-purpose LLM-as-a-judge evaluation may undervalue contextually appropriate minimal responses.Minimal responses contain little explicit informational content and may be penalized by rubrics emphasizing comprehensiveness or professional elaboration.
Each Figure 5 point represents one dataset–model condition, with rows for datasets and columns for interruption-score versus quality and minimal-response-rate versus quality analyses.Marker shapes indicate model conditions, dashed lines show within-dataset trends, and column-title correlations use all conditions.