PaperlayerView exact revision on arXiv

Source-linked AI summary

Thai Semantic End-of-Turn Detection for Real-Time Voice Agents

Thanapol Popit, Natthapath Rungseesiripak, Monthol Charattrakool, Saksorn Ruangtanusak

arXiv:2510.04016v1cs.CLcs.AI

TL;DR

Thai semantic end-of-turn detection for spoken dialogue remains largely unaddressed, despite its relevance to low-latency interaction. This paper defines a text-only Thai EOT task and compares prompting with fine-tuned models using YODAS subtitles. Fine-tuned small models achieve strong reported F1 scores with CPU latencies reported for leading options.

  • Problem

    Thai conversational turn-completion prediction remains largely unaddressed, while semantic EOT uses linguistic content to identify when a speaker has finished.

    Source: §I · p. 1

  • Method

    The paper defines a Thai text-only EOT benchmark from public subtitles and compares compact LLM prompting with fine-tuned encoders and decoders.

    Source: §I · p. 1

  • Results

    0.881 F1 for fine-tuned Llama3.2-Typhoon2-1B at 110ms CPU latency was the highest reported accuracy; fine-tuned Qwen3-0.6B reached 0.866 F1 at 90ms.

    Source: E. Results and Analysis · p. 4, §VI · p. 6

  • Takeaways & Limitations

    The study establishes a practical Thai baseline and reports that lightweight fine-tuned transformers can provide a fast semantic EOT signal beyond silence-based endpointing.

    Source: §VIII · p. 6, §VI · p. 6

    Labels inherit subtitle biases and timing drift, true conversational EOT may differ from subtitle line breaks, and acoustic cues are not modeled.

    Source: §VII · p. 6

Abstract

from arXiv · show

Fluid voice-to-voice interaction requires reliable and low-latency detection of when a user has finished speaking. Traditional audio-silence end-pointers add hundreds of milliseconds of delay and fail under hesitations or language-specific phenomena. We present, to our knowledge, the first systematic study of Thai text-only end-of-turn (EOT) detection for real-time agents. We compare zero-shot and few-shot prompting of compact LLMs to supervised fine-tuning of lightweight transformers. Using transcribed subtitles from the YODAS corpus and Thai-specific linguistic cues (e.g., sentence-final particles), we formulate EOT as a binary decision over token boundaries. We report a clear accuracy-latency tradeoff and provide a public-ready implementation plan. This work establishes a Thai baseline and demonstrates that small, fine-tuned models can deliver near-instant EOT decisions suitable for on-device agents.

I. INTRODUCTION

Semantic EOT predicts whether a speaker has finished using linguistic content, offering an alternative to silence-based detection for reducing latency and improving conversational naturalness. The paper addresses the largely unstudied problem of Thai conversational turn completion with a benchmark and a comparison of prompting and fine-tuning approaches.

I. INTRODUCTION

  • Thai conversational turn-completion prediction remains largely unaddressed, despite research on other Thai NLP tasks.The paper identifies this as a gap in semantic end-of-turn detection for Thai spoken dialogue.Source: §I · p. 1
  • The benchmark provides a Thai EOT task and public-subtitle dataset recipe with Thai-specific analysis.The paper also compares compact LLM prompting with fine-tuned encoders and decoders under a shared evaluation protocol emphasizing CPU latency.Source: §I · p. 1

A. End-of-Turn Detection

  • Prior EOT work covers silence gating,, audio- or context-based detectors –, and language-model turn prediction,,.Related work also reports that contextual language modeling benefits turn-taking tasks [5].Source: A. End-of-Turn Detection · pp. 1–2

B. Decoder-Only Based EOT Detection

Decoder-based EOT approaches use prompting or fine-tuning to estimate turn completion, while encoder models classify complete context windows. These approaches differ in whether they make incremental probabilistic assessments or holistic judgments.

B. Decoder-Only Based EOT Detection

  • Zero-shot and few-shot decoder prompting frames Thai text-only EOT as instruction-based binary classification, with demonstrations added for few-shot inference.Compact models are contrasted with larger instruction-tuned counterparts that generally offer higher performance at greater latency.Source: B. Decoder-Only Based EOT Detection · p. 2
  • Fine-tuned decoder models assign likelihood to a special EOT token at each step, treating turn completion as a predictable sequence event.The token is inserted at speaker-change boundaries during dialogue-corpus training.Source: B. Decoder-Only Based EOT Detection · p. 2
  • Fine-tuned decoder models trade performance against efficiency, with smaller variants suited to low-latency on-device use.The passage contrasts Qwen3-0.6B for low-latency use with Llama3.1-Typhoon2-8B as a more effective accuracy-resource balance.Source: B. Decoder-Only Based EOT Detection · p. 2

C. Encoder-Only Based EOT Detection

  • Encoder models use a lightweight classification head for a binary EOT decision based on a complete context window, unlike incremental decoder assessments.Figure 2 depicts a streaming transcript boundary classified as EOT or not-end, determining whether the AI speaks or continues listening.Source: C. Encoder-Only Based EOT Detection · p. 2, Figure 2 · p. 2

III. TASK AND DATA

The task labels each streaming-transcript boundary as EOT or not-end, using subtitle line ends and sentence-final punctuation as positive cues. The dataset is derived from Thai YODAS subtitles and cleaned and segmented into sentence-like units.

A. Problem definition

  • At each streaming-transcript boundary, the task predicts EOT or not-end; subtitle line ends and sentence-final punctuation are positive labels.Figure 2 presents the decision at position t in a streaming transcript.Source: A. Problem definition · p. 2

B. Dataset construction

  • Thai utterances from YODAS are filtered for Thai characters and reasonable length, then screened for irrelevant subtitle content and segmented into sentence-like units.An instruction-tuned LLM removes songs, advertisements, and other content judged irrelevant to conversational turn-taking; the same model performs sentence segmentation.Source: B. Dataset construction · pp. 2–3

C. Labeling strategy

Encoder training creates positive full-sentence and negative mid-sentence examples, while decoder inputs use the model’s existing EOT token. The paper also reports the dataset split and the model families selected for evaluation.

C. Labeling strategy

  • For encoder training, sufficiently long sentences are cut at the midpoint for a not-end example, while the full sentence is labeled end; decoder sentences use the existing EOT token.This produces a balanced classification dataset without explicit punctuation heuristics.Source: C. Labeling strategy · p. 3

D. Splits and statistics

  • The preprocessed dataset contains approximately 59k Thai sentences, divided into 80% training, 10% validation, and 10% test data.The test split is used for evaluation.Source: D. Splits and statistics · p. 3

IV. MODELS

  • The evaluation covers decoder prompting, zero-shot stop-token thresholding, fine-tuned encoder classification, and thresholding with fine-tuned decoders.The stated model paradigms include both prompted and fine-tuned approaches.Source: §IV · p. 3

A. Model Selection

  • The encoder comparison pairs Thai-pretrained WangchanBERTa with multilingual mDeBERTa, while decoder selections include Thai-specialist Typhoon2 and multilingual Qwen3 models at two scales.The decoder selections include compact and mid-size variants for assessing the scale-performance-efficiency trade-off.Source: A. Model Selection · p. 3

B. Decoder Zero-shot and Few-shot Prompting

The section evaluates decoder prompting for Thai EOT classification and describes encoder training with complete and midpoint-truncated examples.

B. Decoder Zero-shot and Few-shot Prompting

  • Zero-shot decoder prompting classifies each Thai text sequence as a completed or incomplete turn.The task is framed as instruction-based binary classification.Source: B. Decoder Zero-shot and Few-shot… · p. 3
  • Few-shot prompting adds a small number of demonstrations covering common Thai conversational patterns.The demonstrations are intended to assess the impact of in-context learning.Source: B. Decoder Zero-shot and Few-shot… · p. 3

C. Encoder Model Training: Binary Classification

D. Decoder Model Training: Supervised Fine-Tuning

The study fine-tunes decoder models on complete YODAS Thai utterances, compares model families and thresholding approaches, and evaluates performance on a held-out test split.

D. Decoder Model Training: Supervised Fine-Tuning

  • Decoder supervised fine-tuning uses complete “Finished” YODAS Thai utterances with a causal language modeling objective.Training ran for one epoch, with a maximum sequence length of 512 tokens.Source: D. Decoder Model Training: Supervised… · p. 3
  • Figure 3 organizes prompting, thresholding, and fine-tuning approaches across Thai-specific specialist and multilingual generalist models.Source: Figure 3 · p. 4
  • At inference, turn completion is estimated from the model’s native stop-token probability at each boundary and a threshold.Training uses 8-bit parameterization, a 2 × 10−5 learning rate, weight decay 0.01, and a cosine scheduler with 3% warmup.Source: D. Decoder Model Training: Supervised… · p. 4

E. Decoder Zero-Shot Thresholding

  • Zero-shot thresholding applies a validation-set threshold chosen to maximize Youden’s J-statistic on the ROC curve.The threshold is applied to the pretrained decoder’s native stop-token probability, without additional training.Source: E. Decoder Zero-Shot Thresholding · p. 4

A. Experimental Setup

  • All models are evaluated on the held-out test split, with positive-class F1 as the primary metric alongside accuracy, precision, and recall.Source: A. Experimental Setup · p. 4

B. Evaluation Methods

The evaluation compares fine-tuning, instruction prompting, and zero-shot thresholding, with fixed and validation-calibrated operating points for probability-scoring methods.

B. Evaluation Methods

  • The evaluation compares three paradigms: fine-tuning, instruction prompting, and zero-shot thresholding.Source: B. Evaluation Methods · p. 4
  • Fine-tuned encoders use binary classification, while fine-tuned decoders use causal language modeling to predict a stop token.Source: B. Evaluation Methods · p. 4
  • Instruction prompting tests pretrained decoders without weight updates, using either no examples or five examples.Full prompts are shown in Figures 5 and 6.Source: B. Evaluation Methods · p. 4
  • Zero-shot thresholding classifies turns using the pretrained decoder’s native stop-token probability as a score.Source: B. Evaluation Methods · p. 4

C. Operating Points and Calibration

  • Probability-scoring methods report fixed-0.5 and validation-optimized operating points, with the latter selected by maximizing Youden’s J-statistic on the validation ROC curve.The calibrated threshold simulates a lightweight calibration step.Source: C. Operating Points and Calibration · p. 4

D. Latency Measurement

Results show supervised fine-tuning achieves the highest reported performance, while decoder thresholding depends on validation calibration and instruction prompting is too slow for real-time use. Latency was measured per sample on a CPU with batch size one.

D. Latency Measurement

E. Results and Analysis

  • Fine-tuned Llama3.2-Typhoon2-1B leads at 0.881 F1, followed by Qwen3-0.6B at 0.866 and mDeBERTa-v3-base at 0.861.The findings identify supervised fine-tuning as the highest-performing approach and report that it outperforms all zero-shot methods.Source: E. Results and Analysis · pp. 4–5, Table I · p. 5
  • Prompting peaks at 0.706 F1 with zero-shot Qwen3-8B, while inference takes 1.5–2.6 seconds and grows slower with few-shot context.The evaluation concludes that zero- and few-shot instruction prompting is impractical for real-time EOT, regardless of example count.Source: E. Results and Analysis · p. 5, Table III · p. 5
  • At a fixed 0.5 threshold, decoder thresholding yields F1 ≈0.000; validation calibration lifts Llama3.1-Typhoon2-8B to 0.824 F1, while fine-tuned encoders need no such step.Calibration selects a threshold using validation-set ROC analysis and Youden’s J.Source: E. Results and Analysis · pp. 5–6, Table II · p. 5
  • Thai-specialist Typhoon models outperform generalist Qwen3 models zero-shot, but fine-tuned mDeBERTa-v3-base (0.861 F1) surpasses WangchanBERTa [26] (0.784 F1).Source: E. Results and Analysis · p. 6

VI. DISCUSSION

The discussion identifies fine-tuned models that balance Thai EOT accuracy and CPU latency, with deployment trade-offs depending on model type and fine-tuning availability. It also argues that semantic EOT can handle linguistically complete utterances containing pauses.

a) Identifying the Optimal Accuracy-Latency Trade-off

  • 0.881 F1 at 110ms CPU latency makes fine-tuned Llama3.2-Typhoon2-1B the top balance; Qwen3-0.6B offers 0.866 F1 at 90ms.The discussion presents Qwen3-0.6B as an alternative for highly constrained devices.Source: §VI · p. 6

b) Deployment Considerations: Encoders vs. Decoders

  • Fine-tuned Typhoon2-1B reaches peak F1 but needs validation-set threshold calibration, while mDeBERTa offers threshold-insensitive, out-of-the-box deployment.The encoder-versus-decoder choice trades deployment simplicity against maximum performance.Source: §VI · p. 6

c) The Role of Thai-Specific Pretraining

  • Thai-specific pretraining matters most without task fine-tuning; when fine-tuning data is available, model architecture becomes more dominant.The discussion names Typhoon2 as the viable zero-shot option and says a strong generalist model can excel when fine-tuned.Source: §VI · p. 6

d) Advancing Beyond Silence-Based Endpointing

  • A lightweight fine-tuned transformer can signal semantic EOT quickly, including for linguistically complete utterances that contain pauses.The discussion contrasts this with silence-based endpointing and its arbitrary delays.Source: §VI · p. 6

VII. LIMITATIONS AND ETHICS

The paper cautions that subtitle-derived labels may not match true conversational EOT and that acoustic cues and overlap are unmodeled in multi-party settings. It concludes with a Thai-focused, text-only formulation compared against prompted LLMs and proposes future extensions.

VII. LIMITATIONS AND ETHICS

  • Subtitle biases and timing drift mean line breaks may diverge from true conversational EOT; prosody and overlap are unmodeled in multi-party settings.Banking-dialog data may include sensitive content, so dataset handling must comply with privacy policies.Source: §VII · p. 6

VIII. CONCLUSION

  • The conclusion presents a Thai-focused, text-only EOT formulation and compares fine-tuned small transformers with zero- and few-shot LLMs for real-time agents.It identifies lightweight prosodic features and multi-party overlap as future extensions.Source: §VIII · p. 6

APPENDIX

The appendix provides the full zero-shot and few-shot prompts used in the experiments. The few-shot version extends the instruction-only prompt with examples.

APPENDIX

  • The appendix provides the full text of the prompts used in the experiments, including both zero-shot and few-shot variants.Source: APPENDIX · p. 8

a) Zero-Shot Prompt

  • Figure 5's zero-shot prompt gives task instructions and output constraints, emphasizing clear single-token outputs without examples.It serves as a baseline for classifying turns as complete or incomplete.Source: APPENDIX · p. 8, Figure 5 · p. 8

b) Few-Shot Prompt

  • Figure 6's few-shot prompt adds positive and negative examples covering common conversational patterns to reduce ambiguity in borderline cases.It extends the zero-shot version.Source: APPENDIX · p. 8, Figure 6 · p. 8
Loading 2510.04016v1…