Leveraging Fine-grained Error Correction in Korean Speech Recognition for Consultation Services

• Call-center ASR remains error-prone because of pronunciation variance, background noise, overlapping speech, and domain-specific jargon.Source: §1 · p. 1
• Privacy restrictions often block raw-audio access, making text-only transcript post-editing particularly valuable for public-service call logs.Source: §1 · p. 1
• DasanCallDial contains approximately 1,974 dialogue pairs and 115,460 utterance lines, with a 17.95% error rate across distinct utterances.Source: §1 · p. 2
• DCSC detects token-level errors, routes utterances to a corrector, edits fine-grained spans, and augments targets with preceding dialogue context.Source: §1 · pp. 2–3
• pkoT5-based DCSC reaches 90.43 detection accuracy, reduces Bal-WER from 14.67 to 13.30, and lowers E-WER from 29.35 to 26.27.Source: §1 · p. 3

• KsponSpeech and ClovaCall provide Korean speech resources but omit raw ASR outputs, limiting their suitability for text-based error-correction research.Source: §2.1 · p. 3
• Existing Korean post-editing resources are limited by sentence-level independence, artificial noise or speech generation, and erroneous-only source samples.Source: §2.1 · p. 3
• DasanCallDial provides 115k parallel STT-and-ground-truth samples from actual call-center complaints while preserving a clean-majority, error-sparse distribution.Source: §2.1 · p. 3
• Text-only ASR error detection is motivated by settings where raw audio is unavailable, extending prior work beyond acoustic-transcript consistency models.Source: §2.2 · p. 3
• Two-stage pipelines that separate error localization from rewriting and fine-grained supervision improve correction performance under data scarcity.Source: §2.2 · p. 4
• Low-resource post-editing studies indicate that problem-specific inductive bias and domain-adaptive training can matter more than model scaling alone.Source: §2.3 · p. 4

Dataset construction

• 296,100 seconds of recordings from approximately 370 counselors were transcribed and manually corrected into corresponding dialogue references.The recordings average 2 minutes and 30 seconds per call and total 82 hours and 15 minutes.Source: §3.1.1 · p. 5
• Two experienced call-center personnel performed annotation and anonymization, followed by consensus discussion for disagreements.The annotation process involved independent expert work and later validation.Source: §3.1.1 · p. 5
• Cohen’s κ was 0.73, while exact-match agreement between jointly identified erroneous utterances was approximately 85.5%.The double-annotation sample contained 1,000 test utterances.Source: §3.1.1 · p. 5

Detector

• DCSC first uses an encoder-based detector to identify token-level errors and decide whether an utterance requires correction.The detector produces a binary mask over tokenized transcription tokens.Source: §4 · p. 8, §4.1.1 · p. 8, §4.1.2 · p. 8
• The detector labels substitutions, deletions, and insertions as erroneous tokens, masking the preceding or following token for insertion cases.Matched tokens receive 0, while erroneous tokens receive 1 in the mask sequence.Source: §4.1.1 · p. 8

Span-level correction

• The corrector converts erroneous word-level regions into span pairs linking noisy text to corrected text, separated by [SEP], or outputs “No Error” for clean utterances.Consecutive erroneous tokens are merged into spans before constructing the correction target.Source: §4.2.1 · p. 9

Dialogue-level context augmentation

• Dialogue augmentation concatenates up to 10 preceding ground-truth utterances with the target utterance to resolve errors requiring discourse context.Special separator tokens distinguish the dialogue context from the target utterance.Source: §4.2.2 · pp. 9–10

Gated inference

• At inference, the detector gates the corrector: clean utterances are returned unchanged, while flagged utterances are corrected using the full utterance rather than only predicted error tokens.The corrector receives dialogue context when context augmentation is enabled.Source: §4.3 · p. 10, §4.2.3 · p. 10
• Corrector training combines genuinely erroneous samples, detector false positives, and a balanced random sample of error-free utterances to reduce spurious downstream corrections.The curated set is fine-tuned with the fine-grained span-level correction scheme.Source: §4.3 · p. 10

Baseline comparison

• Direct utterance-to-utterance generation over-corrects clean utterances, whereas explicit detection gating better preserves already-correct transcripts.The pkoT5+Uttr2Uttr baseline reaches Bal-WER 25.33 because its N-WER remains high.Source: §5.1.2 · p. 13

Performance overview

• DCSC reduces E-WER from 29.35 to 26.27 and Bal-WER from 14.67 to 13.30 while maintaining low N-WER between 0.23 and 0.43.All DCSC configurations outperform the Zero-rule baseline on detection and correction quality.Source: §5.1.2 · p. 13

K-Seq2Seq vs. LLMs

• LLMs provide strong detection with dialogue context, but pkoT5 produces the lowest E-WER and Bal-WER, indicating more stable rewriting after correction is triggered.Qwen2.5 reaches Recall 75.32, while pkoT5 reaches F1 72.91 and the strongest correction metrics.Source: §5.1.2 · p. 13

Comparison within K-Seq2Seq models

• Among encoder–decoder models, pkoT5 balances Precision and Recall better than mT5 and KoBart, achieving F1 72.91, EM 85.16, and Bal-WER 13.30.mT5 and KoBart have higher Precision but substantially lower Recall, missing more erroneous utterances.Source: §5.1.2 · p. 13

Model selection

• pkoT5 achieves the best overall correction performance, with EM 85.16, E-WER 26.27, and Bal-WER 13.30, while requiring less computation than substantially larger LLMs.Its detection F1 is 72.91, higher than Qwen2.5 despite the smaller model size.Source: §5.1.2 · p. 14

Effect of token-level detection

• Token-level supervision improves KoElectra’s Recall from 76.11 to 81.19 and F1 from 69.85 to 70.61, with the largest gains from fine-grained error localization.The effect is not uniform across aggregate metrics or detector backbones.Source: §5.2.2 · pp. 14–15

Supervision granularity and context

• Span-level targets reduce harmful rewriting and instability, while dialogue context further improves error identification and correction of context-dependent errors.For Llama-3.1, context raises F1 from 42.93 to 54.00 and lowers E-WER from 29.69 ± 0.62 to 26.62 ± 1.62.Source: §5.3.2 · p. 16

Detector–corrector interaction

• The corrector reduces false positives from 911 to 436, improving Precision from 62.46 to 74.64 and decreasing N-WER by 0.16 points.It recovers 475 of 911 falsely forwarded clean samples unchanged, but Recall falls from 81.20 to 68.72 after unresolved corrections.Source: §6.2.1 · p. 18

Detector operating points

• At the default detector threshold of 0.5, Recall is 81.20%, with 351 of 1867 erroneous utterances filtered out before correction.Lowering the threshold to 0.1 raises Recall to approximately 94.5%, while aggressive routing lowers standalone Precision.Source: §6.2.2 · p. 19

Context prediction

• Predicted context changes detection little but increases Bal-WER by 0.45 points for Llama-3.1 and 0.29 points for pkoT5.The corresponding Bal-Acc reductions are 0.45 and 0.01 points, indicating measurable but limited error propagation under predicted context.Source: §6.3 · p. 19

Cross-domain transfer

• Out-of-domain DCSC reduces Bal-WER from 22.66 to 18.19 with Llama-3.1 and 18.15 with pkoT5 without Hyper-BTS training data.Mixed-domain DSC further reaches 12.91 and 11.36, approaching in-domain SFT results of 11.08 and 10.52, respectively.Source: §6.4 · p. 20

Performance by speaker role

• Customer utterances achieve higher Bal-Acc than counselor utterances, 76.25 versus 75.14, while counselor utterances achieve lower Bal-WER, 10.81 versus 15.83.The analysis associates this pattern with customer pronunciation variability and counselors’ more structured, less error-prone speech.Source: §6.5 · p. 21

Scope and modality

• The text-only setting is designed for cases where raw audio is inaccessible, while multimodal approaches remain a future direction when audio is available.The study primarily focuses on Korean ASR transcript post-editing, leaving extension to other languages and tasks for future evaluation.Source: §7.1 · p. 23

Dataset generalizability

• DasanCallDial may not generalize perfectly to open-domain conversations because it reflects call-center protocols and frequent turntaking.The dataset’s topic and frequency diversity nevertheless indicates some task transferability.Source: §7.1 · p. 23

Industrial impact

• The framework provides a practical blueprint for audio-free post-editing in customer service systems, CRM analysis, and other accuracy-sensitive applications.This implication is framed for real-world, high-noise environments with resource constraints.Source: §7.1 · p. 23

Interface granularity

• The detector–corrector interface remains coarse because token-level detector masks still trigger correction at the utterance level.Passing token indices directly could enable more targeted edits but risks detection-error propagation, boundary sensitivity, and optimization instability.Source: §7.1 · p. 23

Under-correction

• Under-correction arises when the detector filters erroneous utterances or when the corrector fails to resolve forwarded errors.Lowering the detector threshold can reduce the first source but does not address corrector-side failures, motivating better calibration and training objectives.Source: §7.1 · p. 23

Contextual limitations

• Finer granularity can reduce immediate context and limit correction of errors requiring long-term dialogue cues outside the input window.Compressed dialogue-context embeddings or more sophisticated context augmentation are proposed to capture broader discourse dependencies.Source: §7.1 · pp. 23–24

Metadata and personalization

• The dataset lacks explicit speaker metadata such as regional background or age group, limiting opportunities for personalized or dialect-aware correction.Future integration of such metadata must continue to follow privacy protocols.Source: §7.1 · p. 24

Data governance

• The dataset’s anonymization and ground-truth annotation were handled securely by trained Call Foundation employees with direct audio access.Only the anonymized dataset was shared with Chung-Ang University.Source: §7.2 · p. 24

• DasanCallDial contains 1,974 call-center dialogues and 115,460 utterance lines with manually verified references.Source: §8 · p. 24
• DCSC combines token-level detection, detector-guided routing, dialogue-context augmentation, and span-level correction in a two-stage framework.Source: §8 · p. 24
• The primary KoElectra–pkoT5 model achieves state-of-the-art performance, outperforming zero-rule baselines and general-purpose LLMs in a label-imbalanced setting.The experiments also examine learning dynamics, transferability, and robustness.Source: §8 · p. 24

• Yonghyun Jun contributed to writing, visualization, validation, methodology, investigation, formal analysis, and conceptualization.Source: CRediT authorship contribution statement · p. 24
• Jimin Lee and Hwan Chang contributed to writing, methodology, and conceptualization, with Hwan Chang also contributing to validation and review and editing.Source: CRediT authorship contribution statement · p. 24
• Dongho Shin and Seolah Kim contributed to data curation, while Hwanhee Lee contributed to writing, supervision, project administration, funding acquisition, and conceptualization.Source: CRediT authorship contribution statement · p. 24

• Claude-Opus-4.8 and GPT-5.6-Sol were used to improve readability during preparation of the work.Source: Declaration of Generative AI and… · p. 24
• The authors reviewed and edited the content after using the tools or services.Source: Declaration of Generative AI and… · p. 24
• The authors state that they take full responsibility for the publication’s content.Source: Declaration of Generative AI and… · p. 24

• Figure 12 presents concrete examples from the original dialogue-level dataset.Source: Appendix A · p. 28
• Figure 13 presents examples from the utterance-level version of the dataset.Source: Appendix A · p. 28
• Side-by-side tables illustrate dataset structure, typical content, and the outcome of de-identification.Source: Appendix A · p. 28

• Table 9 contains the detailed annotation guideline for Section 3.1.1.Source: Appendix B · p. 28
• The annotation guideline is presented as part of Section 3.1.1.Source: Appendix B · p. 28
• Table 9 is the referenced source for the section’s annotation details.Source: Appendix B · p. 28

• 20.90% utterance error rate for Whisper exceeds HAIV’s 17.95%, while approximately four out of five Whisper utterances remain error-free.Source: Appendix C · p. 28
• Noun errors dominate for Whisper and HAIV, at 50.2% and 56.3%, respectively.Source: Appendix C · p. 28
• Consonant-related errors remain dominant, comprising 82.6% for Whisper versus 84.5% for HAIV.Source: Appendix C · p. 28
• Omission and insertion errors are most frequent for Whisper and HAIV, at 80.9% and 75.2%, respectively.Source: Appendix C · p. 28
• The small error-rate gap and consistent dominant patterns suggest the error-sparse setting is not unique to HAIV.Source: Appendix C · p. 28

• Tables 10 and 11 provide the detailed experimental setup for Section 5.1.1.Source: Appendix D · p. 28
• Table 10 summarizes backbone architectures and computational requirements.Source: Appendix D · p. 28, Table 10 · p. 30
• Table 11 reports training and inference configurations for each model family.Source: Appendix D · p. 28, Table 11 · p. 30
• Figures 12 and 13 present dialogue-level and utterance-level dataset examples, respectively.Source: Figure 12 · p. 29, Figure 13 · p. 29
• Figure 14 shows error distributions from re-transcribing DasanCallDial audio with a Korean-fine-tuned Whisper model.Source: Figure 14 · p. 30