TamilEOT: A Dataset and Model for Semantic End-of-Turn Detection in Tamil Telephone Speech

1 Introduction

• 18,485 labelled turn boundaries from 116 real Tamil narrowband telephone conversations form an open dataset, described as the first for Tamil semantic EOT detection.The dataset is released under CC BY 4.0.Source: §1 · p. 2
• Rule-derived labels were validated against a blind human pass, revealing a below-chance negative class that prompted replacement with an audio-LLM labeller.The paper reports both the diagnosis and the resulting label-noise ceiling.Source: §1 · p. 2
• 70.30% →86.13% on the held-out split summarizes the reported change alongside two fine-tuned models and five ineffective training levers.The paper also reports a run-to-run spread to establish a floor for interpreting small deltas.Source: §1 · p. 2
• Replaying the same labelled boundaries through the production VAD and streaming adapter measures a serving cost distinct from offline clip accuracy.Offline clips and production windows are cut differently.Source: §1 · p. 2
• The positive class is complete, so a false positive means the agent talks over the user; FP/N and standard FPR use different denominators.On the test set, 10.1% FP/N corresponds to 27.4% standard FPR.Source: §1 · p. 2

Turn-taking models

• Smart Turn v3 uses one user channel and the last 8 seconds for a binary verdict, but its 23-language release includes no South Indian language.It is based on a Whisper-tiny encoder [4], and its weights and training code are open.Source: §2 · p. 2

Datasets

• ETD [1] combines TTS-generated dialogue with real conversational audio, while Smart Turn’s training corpus is largely TTS; neither covers Tamil narrowband telephone speech.The passage identifies narrowband telephone speech as the acoustic condition most Indic voice agents run in.Source: §2 · p. 2

Language-specific work

• Thai end-of-turn detection [8] uses transcribed subtitles and token boundaries, whereas TamilEOT is audio-only and can run before a transcript exists.The passage characterizes the two approaches as complementary because transcript-based detection waits for ASR while audio-based detection cannot use lexical cues.Source: §2 · p. 2

3.1 Source, and why this corpus

• The 116-call SPRING-INX Tamil R1 corpus has separate human-transcribed audio files for each speaker, allowing each clip to use a single speaker leg.The paired legs were verified complete with no duration mismatch, and diarization is not needed.Source: §3.1 · p. 3

3.2 Boundaries come from the VAD, not the transcript

• Transcript segment edges misplace gaps because segments cover about 107% of call wall-clock time; Silero VAD supplies speech onsets and offsets instead.The transcript confirms VAD spans as real speech and supplies text for the backchannel filter; 50,532 candidate boundaries result.Source: §3.2 · p. 3

3.3 Clip geometry

• Each sample contains 8 seconds ending 200 ms after speech offset, giving both classes equal trailing audio to avoid silence-length label cues.A weak classifier on crude global features is used to test for such shortcuts.Source: §3.3 · p. 3

3.4 Composition and splits

• Boundary provenance is recorded but unavailable at inference; after validation, previously withheld disagreement classes and 2,272 additional rows were added.The shipped label is the labeller’s verdict, which overrules provenance on 53% of hold_intra.Source: §3.4 · p. 3, Table 1 · p. 4
• Splits are assigned by call, keeping test voices out of training; the test split was cut once before training and not revised.This avoids the source corpus’s utterance-level split pattern that can place the same recording on both sides.Source: §3.4 · pp. 3–4, Table 2 · p. 4

4.1 Validating the rule-derived labels

• 44.4% of rule-derived hold_intra negatives agreed with the listener, versus 95.9% for change positives; the negative class was below chance.The blind pass covered 197 clips, stratified by provenance and label, with no label shown to listeners.Source: §4.1 · p. 4

4.2 The rule answered a different question

• The rule-derived negatives answered whether a speaker intended to continue (Q2), not whether an utterance was syntactically finished (Q1), the fine-tune’s target.Prior pause-derived labelling is described as a reasonable construction [1], but mixing Q2 labels into a Q1-pretrained model would teach two questions.Source: §4.2 · p. 4

4.3 Relabelling with an audio LLM

• gemini-3.7-flash agreed with human listeners 97.5% of the time and relabelled the full pool for US$5.69.On the class the rules got wrong, agreement was 97.0%; two measurements found 53% of negatives flipped under relabelling versus 56% in the human pass.Source: §4.3 · p. 5

4.4 The confound this removed

• The hold_intra rule selected longer utterances by construction, leaking voiced fraction into the audio; after relabelling, leaked-feature effect sizes fell near zero.The shortcut-probe AUC rose from 0.601 to 0.622, with both values described as near chance.Source: §4.4 · p. 5

4.5 The label-noise ceiling

• Above 97.1% a model would fit labeller error, while the best model’s 86.13% leaves roughly 11 points of headroom.The passage states that label quality is not currently the limiting factor; the human labels were also a single listen to an isolated 8-second clip, not truth.Source: §4.5 · p. 6

• The architecture uses a Whisper encoder over an 8-second log-mel window, attention pooling, and a binary classification head; the whole stack is fine-tuned.Source: §5 · p. 6
• The two models differ only in encoder size; training uses six epochs, learning rate 5 × 10−5, batch size 32, and seed 0.The encoder is Whisper-tiny or Whisper-base; models are exported through fp32 ONNX to dynamic int8 and run on CPU hardware.Source: §5 · p. 6
• Both released ONNX files reproduce their training scores on the sealed test set, and re-exporting the base checkpoint yields a bit-identical graph.The paper’s numbers are scored from downloadable artifacts rather than training logs.Source: §5 · p. 6

6.1 The reproducibility floor

• Three identical-seed base runs span 0.87 accuracy points, so the authors treat deltas below about 1 point as noise.Source: §6.1 · p. 6

6.2 Main result

• On the held-out test set, fine-tuning raises accuracy from 70.30% to 86.13% (+15.83 points) and ROC-AUC from 0.751 to 0.921.Source: Table 4 · p. 7, §6.2 · p. 7
• At threshold 0.5, v3.2 marks 36.4% of unfinished turns complete despite ROC-AUC 0.751; v3.0 trades 4.66 accuracy points for fewer false completions.Source: §6.2 · p. 7

6.3 Where the gains land

• Gains concentrate on same-speaker pauses, where a majority-class policy is useless and timeouts cannot handle the turn boundary.Source: Table 5 · p. 7, §6.3 · p. 7

6.4 Training levers

• Encoder capacity adds +2.88 accuracy and +0.018 AUC; four times the data at fixed compute adds +3.32, whereas four times the compute on the same data adds −0.17.Source: §6.4 · p. 8
• Distillation’s +0.41 accuracy falls within noise and worsens FP/N by 1.41 points; class imbalance was already handled by per-batch weighting.Recovering 19.9% more train data added only +0.03 points; the relaxed gate mostly harvested positives.Source: §6.4 · pp. 8–9

6.5 The decision threshold

• A dev-set argmax loses 1.13 test accuracy points, while a 10% dev-FPR target roughly halves FP/N but transfers to 12–14% test FPR.Source: §6.5 · p. 9, Table 7 · p. 9

6.6 Quantisation

• Dynamic int8 stays within noise while shrinking models about 3.8× and speeding them about 38%; static int8 costs −4.03 tiny and −12.76 base accuracy.For tiny static int8, FP/N is best in Table 4 even though it is the second-worst model overall.Source: §6.6 · p. 9

7.1 Offline accuracy and served accuracy are different quantities

• 2.60 points of accuracy are lost in paired replay through the production VAD and streaming adapter, a difference consistent in direction but not significant at p < 0.05.The VAD closes 90 ms later than the dataset clips, and 39 of 500 boundaries never reach the model.Source: §7.1 · p. 10, §7.2 · p. 10

7.2 Coverage: the boundaries a VAD never surfaces

• 92.2% of 500 boundaries reached a VAD close; the remaining 39 were never surfaced, and pre-cut clips scored 86.12% versus 83.35% on the full split.The VAD-surfaced boundaries are the easier ones; coverage was 94% on complete and 89% on incomplete turns.Source: §7.2 · p. 10

7.3 Latency, and what the numbers are good for

• 83 ms for tiny int8 and 143 ms for base are single-thread model-only laptop latencies; the authors present these as build comparisons, not deployment specifications.Live adapter latency is 120–155 ms; server hardware, contention, and framework scheduling were not varied.Source: §7.3 · p. 10

7.4 Deployment, and what live evidence establishes

• In live recordings across two production agent frameworks, the detector, compared with fixed-timeout endpointing, incurred +120 ms median endpointing while reducing utterance fragmentation by 35%.This comparison was directional, with seven utterance blocks per arm, one speaker, and uncontrolled arms.Source: §7.4 · p. 10
• Live calls cannot establish accuracy without pause-level labels, and the framework’s false-interruption counter misses mid-sentence commits that fragment utterances.The live calls are an existence proof; the labelled paired replay carries the serving claim.Source: §7.4 · p. 11
• Paired replay also caught harness bugs that produced plausible but invalid serving numbers, including one that inflated the apparent accuracy cost by 26 points.The same rows were scored in the paired run, allowing failures to be checked against established offline results.Source: §7.4 · p. 11

• All 116 calls come from one narrowband Tamil telephone corpus; wideband, noisy, code-switched, and other demographic settings were not measured.The 30 test calls are unseen, but do not represent a different distribution.Source: §8 · p. 11
• Label validation covered 197 clips and one listener, so the 97.5% human agreement estimate rests on a small validation sample.The labels are machine-produced and human-validated, not human-produced.Source: §8 · p. 11
• A 0.87-point run-to-run spread makes several small reported deltas unfalsifiable; the proposed determinism fix is untested and expensive.Source: §8 · p. 11
• The live comparison is uncontrolled and should be read as directional evidence, not a benchmark; the labelled paired replay carries the serving claim.Source: §8 · p. 11
• Regularisation remains untried despite base reaching 98.99% train accuracy versus 87.18% dev, and the proposed short-buffer test has not been run.Only 58 of 16,216 training clips are shorter than eight seconds; three live calls were not considered a measurement.Source: §8 · p. 11

• Tamil semantic end-of-turn detection was not blocked by architecture; the paper identifies checked Tamil data as a contribution alongside the model.Source: §9 · p. 12
• The authors identify label-rule validation, measuring run-to-run spread, and replaying labelled boundaries through production serving as lessons that generalise beyond Tamil.Their examples include 0.87 points of run spread, a 2.60-point serving cost, and 7.8% of boundaries never reaching the model.Source: §9 · p. 12
• The authors propose that the pipeline should port to languages with a two-channel conversational corpus because its label oracle is structural rather than linguistic.Source: §9 · p. 12

• The code and experiments, an 18,485-boundary dataset, and an inference package are publicly linked.Source: Availability · p. 12
• The dataset derives from SPRING-INX Tamil R1 [5] and retains its CC BY 4.0 licence; code and fine-tuned models use BSD-2-Clause.Source: Availability · p. 12
• The labeller bake-off and label-quality analysis reproduce byte-identically from a bare clone using committed text-stripped label tables, without the corpus, GPU, or cloud account.Source: Availability · p. 12