Source-linked AI summary
HiThink Turn: An Intent-Aware Turn-Taking Control Module for Full-Duplex Dialogue
Feiyang Chen, Wenhan Yang, Bohan Wang, Xinjian Gao, Rongjunchen Zhang, Jun Wang, Xinhui Hu
TL;DR
Full-duplex dialogue needs selective, timely interruptions because complete utterances may not warrant a response, while unfinished requests may. HiThink Turn combines playback-conditioned intent-aware state prediction with intent-prefix supervision and boundary-aware training; it leads the reported benchmark comparisons, and intent-prefix triggering improves interruption outcomes.
Problem
End-of-turn prediction alone does not capture whether a complete utterance warrants a response or whether an unfinished request already merits interruption.
Source: §1 · p. 1Method
HiThink Turn separates response intent from semantic completeness, conditions turn states on playback state, and trains on minimal intent-sufficient prefixes and boundary-truncated audio.
Source: §1 · p. 1, §2.3 · p. 3Results
HiThink Turn leads the compared methods in Easy Turn macro accuracy and Full-Duplex-Bench aggregate interaction score (0.933); intent-prefix triggering raises response success from 89% to 98% and cuts mean decision latency 60.9% among successful cases.
Source: §1 · p. 1, §3.2 · p. 3, §3.3 · p. 4Takeaways & Limitations
Abstract
from arXiv · showhide
Full-duplex dialogue requires timely yet selective interruption handling, which end-of-turn prediction alone cannot achieve: complete utterances may need no response, while unfinished requests may warrant interruption. To address this challenge, we propose HiThink Turn, an intent-aware streaming turn-state predictor that separates response intent from semantic completeness and conditions decisions on system playback state. A key contribution is minimal intent-sufficient prefix supervision, constructed through LLM judgments and speech alignment, while training on audio truncated at chunk boundaries improves robustness to partial speech. These components support streaming inference with 240-ms audio chunks, enabling low-latency, accurate full-duplex turn control. Experiments show that HiThink Turn leads the compared methods in Easy Turn macro accuracy, Full-Duplex-Bench average interaction rate score (0.933), and non-target-speech average playback resume rate (0.735). Additionally, intent-prefix triggering raises interruption success from 89\% to 98\% and reduces mean stop latency by 60.9\%.
1. INTRODUCTION
Full-duplex turn control must decide not only whether an utterance is complete, but whether it merits a response and when enough evidence is available to act. HiThink Turn separates response intent from semantic completeness and uses playback state, intent-prefix supervision, and boundary-aware training for streaming prediction.
1. INTRODUCTION
- Full-duplex systems must choose whether to listen, respond, backchannel, or stop playback while handling simultaneous speech.Cascaded systems remain attractive for modularity, reuse of mature components, and preservation of text-based LLM reasoning.Source: §1 · p. 1
- Completion alone is insufficient: a complete statement may not warrant a response, while an unfinished request may justify interrupting playback.Streaming partial inputs can support earlier decisions but also risk premature interruptions.Source: §1 · p. 1
- Earlier predictors focused on utterance completion, while later systems separated completion from acceptance or rejection decisions [14] [15].Smart Turn and TEN Turn Detection infer completion from speech or ASR text; FlexDuo [11] and Easy Turn also model irrelevant input, incomplete speech, and backchannels.Source: §1 · p. 1
- HiThink Turn separates response intent from semantic completeness and conditions turn-state decisions on system playback state.Its supervision labels minimal intent-sufficient prefixes using LLM judgments and speech alignment, with boundary-aware truncation and non-target speech augmentation for 240-ms chunk-wise inference.Source: §1 · p. 1
- HiThink Turn is implemented by fine-tuning Qwen3-ASR-1.7B [17] with LoRA, and the reported results show accurate completeness prediction, effective turn control, and robust non-target-speech rejection.Triggering on intent prefixes improves response success and reduces interruption latency compared with triggering on utterance completion.Source: §1 · p. 1
2. HITHINK TURN
HiThink Turn organizes turn states around audio validity, user intent, and system playback state. It builds annotation and training procedures to support fine-grained, playback-conditioned turn control.
2.1. Overview
- The overview adapts Qwen3-ASR with LoRA and conditions turn-state prediction on whether the system is speaking or listening.The formulation defines eight state labels across audio validity, user interaction intent, and playback state.Source: §2.1 · p. 2
2.2. Intent-Aware Turn-State Design
- Turn states distinguish noise and incomplete or complete speech from response, backchannel, and interruption intent, with playback state conditioning duplex decisions.Backchannel and interruption are valid only during system playback.Source: §2.2 · p. 2, §2.1 · p. 2
2.3. Turn-State Annotation Workflow
- The annotation workflow labels completeness, creates partial examples at 240-ms chunk boundaries, and uses LLM judgments to identify response intent and minimal intent-sufficient prefixes.The earliest response-intent prefix is labeled interruption, with its aligned end time marking the barge-in point; noise and conflicting or repeated labels are filtered.Source: §2.3 · pp. 2–3
2.4. State-Conditioned Training
- Training conditions predictions on system speaking or listening state, excludes that conditioning input from the loss, and randomly assigns either state to other samples.Backchannel and interruption samples are conditioned exclusively on speaking.Source: §2.4 · p. 3
3. EXPERIMENTS
HiThink Turn is evaluated on completeness prediction, full-duplex interaction, non-target speech rejection, and ablations of its training data and playback-state context. The results show strong benchmark performance, while the ablations reveal distinct benefits from boundary-aware data and playback-state conditioning.
3.1. Experiment Details
- Training used 7,633 h of English and Chinese audio to produce 5.552 million samples; LoRA fine-tuning used two epochs on eight NVIDIA H100 GPUs.The training samples span ASR corpora, real-world QA recordings, synthesized user turns, and noise data.Source: §3.1 · p. 3
3.2. Easy Turn Results
- HiThink Turn achieves the highest Easy Turn macro accuracy in English (91.9%) and Chinese (99.4%), and the best accuracy on incomplete utterances.Appending 0.3 s of silence corrects 61.5% of misclassified complete English utterances that often have little or no trailing silence.Source: §3.2 · p. 3
3.3. Full-Duplex-Bench Results
- 0.933 aggregate interaction score leads the compared systems; interruption triggers raise success from 89% to 98% and cut successful-case mean decision latency by 60.9%.The reported comparison holds the model and its predictions fixed, contrasting interruption-or-response triggers with response-only triggers.Source: Table 3 · p. 4, Figure 3 · p. 4, §3.3 · p. 4
3.4. Non-Target Speech Rejection Evaluation
- HiThink Turn achieves the highest average RsR of 0.735 in FDB v1.5, leads on Background, but trails Gemini Live on Talk-to-Other.The authors identify contextual recognition of system-directed speech as an area for improvement.Source: Table 4 · p. 4, §3.4 · p. 4
3.5. Ablation Study
- Removing boundary-aware data lowers aggregate score despite slightly shorter latency; removing playback-state conditioning lowers turn-taking success, increases pause takeovers, and delays interruption despite better backchannel resume.Error analysis links boundary-split words to inappropriate waiting or early responses; playback context distinguishes responding while listening from interrupting during playback.Source: §3.5 · p. 4
4. CONCLUSION
HiThink Turn separates response intent from semantic completeness using playback context, minimal intent-sufficient prefix supervision, and boundary-aware training. On Full-Duplex-Bench and Easy Turn, it demonstrates strong turn control and completeness prediction, while future work targets dialogue context and speaker cues.
4. CONCLUSION
- HiThink Turn separates response intent from semantic completeness through playback context, minimal intent-sufficient prefix supervision, and boundary-aware training.Source: §4 · p. 4
- Intent-prefix triggering reduces mean decision latency among successful cases by 60.9% over response-only triggering.Source: §4 · p. 4
- Future work will incorporate dialogue context and speaker cues to distinguish response intent, turn yielding, and system-directed speech.Source: §4 · p. 4